前情概要: compass是以 Lucene 为底层的搜寻引擎介面, 它可以提供你对搜寻需求的满足,藉由预先建立起索引档, 可以把全文检索所需的搜寻优化到能接受的程度. 底下是边看边写写下来的结论 , 多余的想法跟对话也是忠实呈现 , 希望对於有相关需求的使用者能有所介绍...-.-;; This is NOT a step-by-step guideline. 这不是一份[下一步-下一步-下一步]的指引, 而是笔记下来所需要的东西, 如果您无法理解,可能是我表达能力太差,也可能是您需要阅读一些相关知识. ────────────────────────────────── 目前看下来的一点结论 ────────────────────────────────── 1.compass 在界面上是采用与hibernate相似的介面 一样是 Factory Session Transation http://0rz.tw/8b4nh 摘录自上述网页说明 Compass API As you will learn in this chapter, Compass high level API looks strangely familiar. If you used an ORM framework (Hibernate, JDO or JPA), you should feel right at home. This is of-course, intentional. The aim is to let the developer learn as less as possible in terms of interaction with Compass. Also, there are so many design patterns of integrating this type of API with different applications models, that it is a shame that they won't be used with Compass as well. For Hibernate users, Compass maps to SessionFactory, CompassSession maps to Session, and CompassTransaction maps to Transaction. 相关使用范例请参照原本的资料, 由於hibernate会建立与资料库的链结 (connection), 看到这里我心中的疑惑是那资料库从何而来 . 基於效率考量 , 显然不会是去读取原本的db , 就先前的所知是会建立index档, 相信在接下来的章节会有所解答. ────────────────────────────────── 果不其然, 接下来指引谈论到的就是 Connection! 但仍然不是预期中的索引档,而是从何读取. http://0rz.tw/864n5 这一章主要提到的都是connection的设定档如何撰写, 由於这章本身很长,讲述各种的方式, 而且显然Jdbc由於要考虑到资料库的各项设定,所以篇幅稍长, 各位看官可根据自己需要的方式去选择需要的章节, 在这里我所需要的是File System Store的部分. 不过我还是将章节表列如下. 4.1. File System Store 4.2. RAM Store 4.3. Jdbc Store 4.3.1. Managed Environment 4.3.2. Data Source Provider 4.3.2.1. Driver Manager 4.3.2.2. Jakarta Commons DBCP 4.3.2.3. c3p0 4.3.2.4. JNDI 4.3.2.5. External 4.3.3. File Entry Handler 4.3.4. DDL 4.4. Lock Factory 4.5. Local Directory Cache 4.6. Lucene Directory Wrapper 4.6.1. SyncMemoryMirrorDirectoryWrapperProvider 4.6.2. AsyncMemoryMirrorDirectoryWrapperProvider 这里定义的是索引档要怎麽存放在什麽地方, 也定义要从哪里读取索引档. ────────────────────────────────── 接下来让我们快速的进到下一步... ────────────────────────────────── 各位看官!!!四张Ａ!!!(被巴) (不小心拿错剧本...让我们切回原本主题..) 光看这章的章节名称也知道我们终於要进入搜寻引擎的主题了. 这章就没什麽好跳过的了...努力的啃吧 ...都是原理解释居多... ────────────────────────────────── Chapter 5. Search Engine http://0rz.tw/6b4oU ---------------------------------------------------- 5.1. Introduction Compass Core provides an abstraction layer on top of the wonderful Lucene Search Engine. Compass also provides several additional features on top of Lucene, like two phase transaction management, fast updates, and optimizers. When trying to explain how Compass works with the Search Engine, first we need to understand the Search Engine domain model. 要解释如何使用 Compass 及底层的Lucene 来达到目标, 我们必须要了解搜寻引擎的基础模型. ---------------------------------------------------- 5.2. Alias, Resource and Property Resource represents a collection of properties. You can think about it as a virtual document - a chunk of data, such as a web page, an e-mail message, or a serialization of the Author object. A Resource is always associated with a single Alias and several Resources can have the same Alias. The alias acts as the connection between a Resource and its mapping definitions (OSEM/XSEM/RSEM). A Property is just a place holder for a name and value (both strings). A Property within a Resource represents some kind of meta-data that is associated with the Resource like the author name. Resource(资源) 代表一群属性, 你可以想像它是个虚拟文件, 一大片的资料, 像一个网页,一个email讯息,或者一连串的 Author 物件. 一个 Resource 总会关系着一个 Alias(别名) , 并且可能会有多个Resource 共用同一个别名. Alias 扮演着在资源及其映射*的定义中的连线角色. 一个 Property (属性) 是放置一个name 与 value (两者皆为字串). *TonyQ 注: 这部分由於部分属於基础知识,所以我不特别在此着墨, 在网上找到能解释些名词的blog,可以参考. http://blogger.org.cn/blog/more.asp?name=lhwork&id=18505 Property可以对应到 System.property 的概念, OSEM = Object /Search Engine Mapping XSEM = XML / Search Engine Mapping RSEM = Resource / Search Engine Mapping Every Resource is associated with one or more id properties. They are required for Compass to manage Resource loading based on ids and Resource updates (a well known difficulty when using Lucene directly). Id properties are defined either explicitly in RSEM definitions or implicitly in OSEM/XSEM definitions. 每个Resource 关系着一到多个id 属性, id对 Compass 来讲是必须的, 因为它采用 id 来作为管理读取与更新的属性. id属性在 RSEM 被明确的定义,也被包含在 OSEM/XSEM 中. For Lucene users, Compass Resource maps to Lucene Document and Compass Property maps to Lucene Field. 对Lucene 使用者而言, Compass Resource可以看做是 Lucene Document, 且 Compass Peroperty 也对应到 Lucene Field. ---------------------------------------------------- 5.2.1. Using Resource/Property When working with RSEM, resources acts as your prime data model. They are used to construct searchable content, as well as manipulate it. When performing a search, resources be used to display the search results. 当使用 RSEM时, 资源就像是你的原始资料模组(一个资料存放的单位), 他们不但被用在建构可搜寻的内容，而且也巧妙的被处理. 当执行一个查询,资源被用在展示这些查询的结果. Another important place where resources can be used, which is often ignored, is with OSEM/XSEM. When manipulating search content through the use of the application domain model (in case of OSEM), or through the use of xml data structures (in case of XSEM), resources are rarely used. They can be used when performing search operations. Based on your mapping definition, the semantic model could be accessed in a uniformed way through resources and properties. 另一个重点是在於资源可被运用在何处, 在 OSEM/XSEM 的模式下 , 资源常常被忽略不用. 当操作搜寻内容透过使用 OSEM的 程式资料模组* , 或 XSEM的xml资料结构时 , 资源几乎不使用, TonyQ 注: application domain model 应指程式执行时的Object model Lets simplify this statement by using an example. If our application has two object types, Recipe and Ingredient, we can map both recipe title and ingredient title into the same semantic meta-data name, title (Resource Property name). This will allow us when searching to display the search results (hits) only on the Resource level, presenting the value of the property title from the list of resources returned. 我们来透过一个范例来简化这些叙述 , 如果我们的应用程式(简称ap) 有两个物件型态, 食谱(Recipe) 跟 原料(Ingredient) ,我们可以 将食谱标题跟原料标题对应到同样语义的原始资料名称,也就是标题("title"). 这将会允许我们,当搜寻到要显示搜寻结果(称作hits)时, 可以只停留在资源层级,不需了解更底层的资料来源. 而是展示从资源回传的清单中 , property "title"的值, ---------------------------------------------------- 5.3. Analyzers Analyzers are components that pre-process input text. They are also used when searching (the search string has to be processed the same way that the indexed text was processed). Therefore, it is usually important to use the same Analyzer for both indexing and searching. 分析者是对输入资料做预先处理的一群原件, 他们也被用在搜寻时 . (搜寻的字串也同样需要透过跟建立索引的字串一样的方法被处理*) 因此, 重点在於通常使用同样的分析者来建立索引与查询. TonyQ注: 像 "hello hi" 可能需要分成两关键字hello 跟hi Analyzer is a Lucene class (which qualifies to org.apache.lucene.analysis.Analyzer class). Lucene core itself comes with several Analyzers and you can configure Compass to work with either one of them. If we take the following sentence: "The quick brown fox jumped over the lazy dogs", we can see how the different Analyzers handle it: 分析者是一个Lucene class . (限定是 org.apache.lucene.analysis.Analyzer类别). Lucene 核心自带有数个分析者,你可以设定Compass去使用其中任何一个, 如果我们使用底下的叙述 "The quick brown fox jumped over the lazy dogs" 我们将会看到不同的分析者如何处理它 (表列如下) whitespace (org.apache.lucene.analysis.WhitespaceAnalyzer): [The] [quick] [brown] [fox] [jumped] [over] [the] [lazy] [dogs] simple (org.apache.lucene.analysis.SimpleAnalyzer): [the] [quick] [brown] [fox] [jumped] [over] [the] [lazy] [dogs] stop (org.apache.lucene.analysis.StopAnalyzer): [quick] [brown] [fox] [jumped] [over] [lazy] [dogs] standard (org.apache.lucene.analysis.standard.StandardAnalyzer): [quick] [brown] [fox] [jumped] [over] [lazy] [dogs] TonyQ按:显而易见的差异,stop跟stanard少了两个the , simple则不考虑 The跟the的大小写差异(一律小写). Lucene also comes with an extension library, holding many more analyzer implementations (including language specific analyzers). Compass can be configured to work with all of them as well. Lucene 也拥有一个延伸的资源库(library), 拥有许多分析者实作, (包括特定语言的分析者) , Compass 亦可被设定成使用这些分析者. ---------------------------------------------------- 底下这些章节请自行阅读...主要是设定档...目前先跳过 5.3.1. Configuring Analyzers 5.3.2. Analyzer Filter 5.3.3. Handling Synonyms (同义字处理) 5.4. Query Parser 5.5. Index Structure (索引结构) 5.6. Transaction (这章谈到一些交易的细节) 5.6.1. Locking 5.6.2. Isolation 5.6.2.1. read_committed 5.6.2.2. serializable 5.6.2.3. lucene 5.6.3.2. FS Transaction Log 5.7. All Support (设定是否支援 OSEM, RSEM, XSEM) 5.8. Sub Index Hashing (子索引建表方案) 5.8.1. Constant Sub Index Hashing 5.8.2. Modulo Sub Index Hashing 5.8.3. Custom Sub Index Hashing 5.9. Optimizers 最佳化 5.9.1. Scheduled Optimizers 5.9.2. Aggressive Optimizer 5.9.3. Adaptive Optimizer 5.9.4. Null Optimizer 5.10. Merge 5.10.1. Merge Policy 5.10.2. Merge Scheduler 5.11. Index Deletion Policy 5.12. Spell Check / Did You Mean 语法检查 5.12.1. Spell Index spell check 是个值得着墨的功能, 透过开启这个功能可以有[建议结果]. 就类似google的[你是不是要查 xxxx ]的提示语, 有兴趣的可以翻原文这个章节... btw 我用的版本没有支援的样子,推测是2.0以後才支援的项目.. 5.13. Direct Lucene 5.13.1. Wrappers 5.13.2. Searcher And IndexReader ---------------------------------------------------- 总结, 第五章是解释搜寻原理,以及如何 tuning 搜寻结果的部分, 并且为Lucene user解释 Compass与Lucene之间的关连.. ────────────────────────────────── 接下来六七八章分别是 OSEM , XSEM , RSEM的操作方式 , 我需要处理的主要是OSEM的部分(因为有用hibernate转成物件) , 所以接下来只着墨在 OSEM的部分 .(也就是 How to write .cpm.xml) ------------------------------------------------------- 接下来参考 http://0rz.tw/514pS 的资料当hbm来写. 不同类别的东西可以再下一个meta constant来补, 在querystring的时候再针对那个meta搜寻(+type=xxxx +(:keyword)"就好. 需要注意的地方 (我一开始弄反了) <class name="Category" alias="category" root="false"> <id name="sid"> ^^^^^^^^^^^^^^^ class栏位(field)名称 <meta-data>cat_sid</meta-data> ^^^^^^^索引值名称 (properties name) </id> .. .. </class> 另外如果有需要对结果作排序 可考虑... (当然也要排序的栏位须要先行准备好!!) compass.openSession().queryBuilder().queryString("test").toQuery(). addSort("description",CompassQuery.SortPropertyType.STRING).hits(); 查询用的query string 细节: http://0rz.tw/514pR -- 结束得有点虎头蛇尾, 不过基本上 cpm就是写id跟要蒐的栏位,还有一些可能需要的常数或栏位, 之後就进行build索引的动作 , 这里我是交给Gps去处理 , Compass 在 Spring有不错的整合 (文件中称Compass::Spring), 所以这边我也就先略去不写 , 请参阅原文件 . 我能阅读文件的时间用的差不多了, 有兴趣研究的再讨论吧 ,有问题欢迎指正/批评/讨论 . -- I am a person, and I am always thinking . Thinking in love , Thinking in life , Thinking in why , Thinking in worth. I can't believe any of what , I am just thinking then thinking , but worst of all , most of mine is thinking not actioning... --

※ 发信站: 批踢踢实业坊(ptt.cc) ◆ From: 220.128.219.202