Mapping

Your domain classes need to be mapped to the search index in order to make them searchable, and to get them back out again for search results. The easiest way to do this is to delcare static searchable = true in your domain class(es):

class User {
    static searchable = true

    // ... properties etc
}

This maps the class in question with default rules.

Custom mappings are possible by setting the searchable property to a Map or Closure.

The plugin doesn't yet expose all mapping configuration options so you can also use either Compass annotations or Compass Mapping XML instead if you need something special.

Only specific properties

You might want to exclude some properties from the index because the data is sensitive, or you only need some properties to be searchable:

class Post {
    static searchable = [only: ['category', 'title']]

    // ...
}

class Post {
    static searchable = [except: 'createdAt']

    // ...
}

The value of except or only can be a String (with wildcards since 0.4) or List of Strings as shown above. Wilcards (since 0.4) are shell-like patterns - * means any number of characters and ? means any single character:

// map, eg, 'addressLine1', 'addressLine2', 'addressPostcode', etc...
static searchable = [only: 'address*']

// do not map, eg, 'screenX' and 'screenY' and 'version'
static searchable = [except: ['screen?', 'version']]

However read this caveat and alternative approach.

Compass mapping concepts

Let's learn some essential Compass concepts: root vs non-root class mappings and property mapping strategies (searchable-property, searchable-reference and searchable-component).

Root mappings

Each class mapping can be either a root or non-root mapping.

If a class has a root mapping, then searches will return instances of that class. If a class has a non-root mapping searches will never return instances of that class.

So why then would you want non-root mappings? Because you may wish to store objects in the index so that when other objects are returned as search results, you get a complete object graph.

Searchable-Property, Searchable-Reference and Searchable-Component

Properties of your domain classes will be mapped with one of the following strategies:

Searchable-Property

This is used for simple types (or collections of); basically anything that can be represented as a string in the index, eg, strings, numbers, dates, immutable value types, enums, etc.

When these types of domain class properties are saved to the index their text representation is made searchable in the index in a Lucene field named after the property, so for a String property called title you can search specifically for text in that property with a query string like "title:attack".

Additionally Compass creates a property in the index called "all" which contains all searchable text for a domain class. And if you do not prefix search query terms, they hit the "all" field. Eg a query string "attack" will match the word "attack" in any searchable property.

Searchable-Reference

This is used for complex types (or collections of); typically other domain classes.

A searchable-reference class type must be searchable itself (whether it is root or not).

When a search matches a class Post that has a searchable-reference property comments of type Set<Comment>, then the Post will be returned from the index with the comments populated from data saved in the index - it doesn't hit the database.

Storage of searchable-references in the index is efficient; in the class declaring the searchable-reference property, a reference (the id) is stored but only for the purposes of re-creating the object graph when returning an item from the index. The searchable-reference instance's own data is stored in a separate index dedicated to that class type.

Searchable-Component

This is used for complex types (or collections of); typically other domain classes or custom types.

A searchable-component class type must be searchable itself (whether it is root or not).

The searchable data in a searchable-component property is added to the searchable data of the class instance declaring the property.

This means that when a class Post that has a searchable-component property comments and a search matches some text in one of those associated Comment instances then the Post is returned. Additionally the Post will be rehydrated from the index with its comments property populated from data saved in the index - it doesn't hit the database to do this.

Default mapping rules

The easiest way to map a class to the search index is by declaring static searchable = true which maps all "mappable" properties in the class using built-in rules:

Simple property types (typically simple types like numbers, dates and strings, or a collection of) are mapped as searchable-properties and are added to the index as a searchable piece of text for the instance, in a field named after the property.

This allows you to search for text in certain properties, eg, "title:grails" would match Posts with "grails" in the title.

Additionally, Compass adds all searchable text to an "all" property in the index and makes this the default place to look for text search queries. This means when you do a text search without prefixing terms, eg, "omfug" what you are actually searching is the "all" property meaning your query will match domain class instances with the word "omfug" in any of their searchable properties.
Non-embedded domain class properties (or element type if a collection) are mapped as searchable-reference as long as they are searchable too.

This means that when an object is returned from the index as a search hit, it can be returned along with it's associated domain objects (and domain object collections) set on it.
Embedded domain class properties are mapped as searchable-component

If the other domain class does not declare a searchable property it is automatically mapped as non-root, with default mapping rules, otherwise the other domain class's own searchable property defines the mapping for that class.

To prevent this automatic searchable-component mapping you can declare the other class as searchable = false.

Default mapping example

Let's use this example to explain the default mapping rules:

class Post {
    static searchable = true
    static embedded = ['metadata']
    static hasMany = [comments: Comment]
    String category
    String title
    String post
    User author
    Metadata metadata
    Date createdAt
}

The simple type properties category, title, post and createdAt are mapped as searchable-properties named after the property, so you can target specific properties in your search criteria, eg, "title:grails" or "category:javascript"
The non-embedded domain class properties user and comments are mapped as searchable-reference so they are returned along with your domain object in search results, as long as those classes are also searchable.

This means that when an object is returned from the index as a search hit, it can be returned along with it's associated domain objects (and domain object collections) set on it.
The embedded domain class property metadata is mapped as searchable-component so any search query that matches the Post's metadata will return that Post instance.

It would not be necessary to declare the Metadata class as searchable, in which case the class is mapped as non-root mapping with default rules, otherwise if you have declared a searchable property in Metadata then that defines whether Metadata is a root mapping (when searchable != false) or not mapped at all (when searchable == false).

Per-property mapping options

You can override the default mapping rules and define per domain class property mappings:

class Post {
    static searchable = {
        // Note: "createdAt" is not mentioned so it is mapped as per default rules,
        // in this case as a searchable-property

        // Note: "metadata" is not mentioned so it is mapped as per default rules,
        // in this case as a searchable-component if appropriate

        category index: 'un_tokenized', excludeFromAll: true  // Implied Compass "searchable property"
        title boost: 2.0, analyzer: "myAnalyzer"              // Implied Compass "searchable property"
        post store: 'compress'                                // Implied Compass "searchable property"
        author cascade: 'create,delete'                       // Implied Compass "searchable reference"
        comments component: true                              // Explicit Compass "searchable component"
    }

    // ...
}

You can use the options supported by the Compass class property mapping strategies in a searchable Closure declaration.

The supported options for each type are defined by their Compass annotations: @SearchableProperty, @SearchableReference and @SearchableComponent. (Note that Searchable Plugin does not use the annotations internally, but they make a good API doc reference.)

However, where the annotation attributes have Enum values, the mapping requires a lower-cased string equivalent. For example Index.TOKENIZED becomes "tokenized". To clarify, here's a comparison between what this mapping would look like with Java annotations and Searchable Plugin's closure config:

Searchable Plugin's closure mapping	Native Compass Annotations
// implied searchable property category index: 'un_tokenized', excludeFromAll: true	@SearchableProperty(index = Index.UN_TOKENIZED, excludeFromAll = true) public String getCategory()
// implied searchable property title boost: 2.0, analyzer: "myAnalyzer"	@SearchableProperty(boost = 2.0f, analyzer = "myAnalyzer") public String getTitle()
// implied searchable property post store: 'compress'	@SearchableProperty(store = Store.COMPRESS) public String getPost()
// implied searchable reference author cascade: 'create,delete'	@SearchableReference(cascade = {Cascade.CREATE, Cascade.DELETE}) public User getAuthor()
// explicit searchable component comments component: true	@SearchableComponent public List<Comment> getComments()

Let's explore the above example. (Please note this is just an example demonstrating a few options with contrived reasoning for using them. You should understand the options before using them yourself.)

The category property is a "searchable property" (simple type). Here we say index it (add it to the index) but don't break it up into terms: store the value as a single term. We also tell Compass to exclude the property from the searchable "all" field:

category index: 'un_tokenized', excludeFromAll: true  // Implied Compass "searchable property"

The title property is important so we give it a boost, meaning any matches in the title field have a higher score, and use a custom analyzer (configured with Compass settings) to index the property:

title boost: 2.0, analyzer: "myAnalyzer"             // Implied Compass "searchable property"

The post property may be very large so we tell Compass (Lucene actually) to compress it in the index:

post store: 'compress'                                // Implied Compass "searchable property"

The author property would normally be mapped as a searchable reference. We don't alter that behavoir but we do change the default "cascade" option to 'create,delete', whereas normally it would be 'all'.

author cascade: 'create,delete'                       // Implied Compass "searchable reference"

We would like the comments to be searchable as part of the Post. In other words we would like the searchable text in comments to be part of our Post searchable text, so when searching for Posts we also get hits when the text is in the comments. For this we need to make comments a searchable component rather than searchable reference:

comments component: true                              // Explicit Compass "searchable component"

The value of 'reference' or 'component' can be true or a Map of options, eg:

comments reference: true // or

comments reference: [cascade: 'create,delete'] // or

comments component: true // or

comments component: [cascade: 'create,delete']

Any other properties that are not mentioned in the closure mapping are mapped using default rules.

Per-property mappings, only certain properties

You can combine the closure mapping with the only or except options:

class Post {
    static searchable = {
        except = ["version", "createdAt"]                     // version and createdAt will not be mapped to the index
        category index: 'un_tokenized', excludeFromAll: true  // Implied Compass "searchable property"
        title boost: 2.0, analyzer: "myAnalyzer"              // Implied Compass "searchable property"
        post store: 'compress'                                // Implied Compass "searchable property"
        author cascade: 'create,delete'                       // Implied Compass "searchable reference"
        comments component: true                              // Explicit Compass "searchable component"
    }

    // ...
}

The except or only value can be either a String (with wildcards since 0.4) or List of Strings. See only/except mapping option for more.

Mapping DSL

The mapping DSL allows you to provide additional mapping information and/or override defaults.

The searchable mapping DSL is not complete.

Eventually it will support all of Compass's class mapping configuration options.

Class mapping

There are a number of mapping options that affect the class (or provide defaults for property mappings).

You can define these options in a couple of different formats:

static searchable = {
    alias "foobar"
    subIndex "fb"
    constant name: "type", value: "some foobar"
    constant name: "noise", values: ["squawk", "shriek"]

    // property mappings follow if required
}

If the option names clash with your own class's properties, an alternative format is (since 0.5):

static searchable = {
    mapping alias: "foobar", subIndex: "fb", {
        constant name: "type", value: "some foobar"
        constant name: "noise", values: ["squawk", "shriek"]
    }

    // property mappings follow if required
}

You can supply a Map and/or Closure to the mapping declaration, eg:

static searchable = {
    mapping alias: "foobar", subIndex: "fb"

    // property mappings follow if required
}

static searchable = {
    mapping {
        alias "foobar"
        subIndex "fb"
    }

    // property mappings follow if required
}

The class mapping options are:

alias - the symbolic name given to each entity in the index
subIndex - the name of the directory/table Lucene index
constant - defines additional constant meta data for class instances in the index

alias

The alias is a symbolic name used to identify the class type in the search index. (You can think of the alias as the same as the table name in an ORM mapping.)

You can override the default alias by specifying the alias in the searchable mapping:

static searchable = {
    alias "posts"
}

static searchable = {
    mapping alias: "posts"
}

subIndex

(Since 0.5)

The subIndex is the name of the Lucene index created for each searchable domain class. If you are using a file-connection (the default) then this is the name of the Lucene index directory. If using a JDBC-connection this forms part of the table name.

You can override the default with:

static searchable = {
    subIndex "posts_idx"
}

static searchable = {
    mapping subIndex: "posts_idx"
}

constant

You can provide constants for each instance of the domain class in the index:

static searchable = {
    constant name: "type", value: "person"
}

static searchable = {
    mapping {
        constant name: "type", value: "person"
    }
}

This is a way to add meta-data to the search index on a per-class basis and is typically done so you can search on it later, eg:

searchableService.search("+type:person bob")

The value of the constant may be a List, and you can use the word "value" or "values" as you prefer, eg:

constant name: "media", value: "cd"

constant name: "media", value: ["cassette", "tape"] // or
constant name: "media", values: ["cassette", "tape"]

To add multiple constants just add more constant declarations:

constant name: "media", value: "cd"
constant name: "capacity", value: 80

This feature uses Compass's Searchable-Constant functionality and supports the same optional attributes as @SearchableConstant except Enum values become lower-case strings, eg:

constant name: "mimeType", value: "text/plain", index: "un_tokenized", store: "no", boost: 1.5

Default exclusions

The plugin has a list of property names to exclude from mapping by default. Out of the box this list is simply ["password"], so when using searchable = true Searchable Plugin will map all Class properties except properties named "password". This list can be configured by you and this might be a good approach if you never want properties like "version" mapped: see Configuration.

When using only or except Searchable Plugin ignores the default property exclusion list and maps all properties as indicated. [ Request for feedback: is this ok? Should it always exclude these default excluded properties, even when the user defines only/except? ]

Default formats

You can control the format of stringified properties in the index. This might be necessary for sorting on number types, for instance. Searchable Plugin does not apply any special formatting rules (over and above what Compass does) but allows you to configure the format per class type with the configuration file.

Excluding certain fields from being searched

Note that using only or except means that the index will only contain those properties as indicated, so when an object is returned from the index, some properties may be null, even if they have non-null values in the database.

You can do better than this by still mapping those properties to the index, but having them not actually indexed for search purposes. To do this, map the class with the closure mapping technique described above, and declare the proeprty with an index: 'no' option.

This method may become default behavoir for properties excluded from mapping with only/except in future.

Compass annotations

In order to use Compass annotations just define them in your classes, eg:

import org.compass.annotations.*

@Searchable(alias = 'user')
class User {
    static hasMany = [friends: User]

    @SearchableId
    Long id

    @SearchableProperty
    String name

    @SearchableReference(refAlias = 'user')
    Set friends
}

(Normally with Compass you need a master XML config file - typically compass.cfg.xml - in which you declare these mapped classes, but with the Searchable Plugin you don't need that - it detects them automatically.)

Note that when using annotations, Compass (well, Java really) needs actual properties for the annotations to annotate, so you might find you have to add things like the id property of the class and any Collection properties that are normally created dynamically by Grails.

See the Compass manual, API docs and Searchable Plugin integration tests for more examples.

Compass Mapping XML

To use Compass Mapping XML, add a DomainClassName.cpm.xml file to your classpath.

See the Compass manual and the Searchable Plugin integration tests for examples.

(The plugin currently configures the Compass mappings by generating this XML at runtime, so if you would like to see it, enable debug logging.)

Previous - Searching

Up - Searchable Plugin

Next - Managing the index

Led & Sustained by

Developed with

Powered by

Searchable Plugin - Mapping