Multiple Elasticsearch Parsers for a Single Field

Question

I store different types of documents in one index with a strict predefined display. All of them have a field (say, "body"), but I would like them to be analyzed a little differently when indexing (for example, to use different token filters for certain documents) and be treated equally in the search. As far as I know, analyzers cannot be specified for each document.

What I also considered using:

• Fields of objects with different analysis of subfields for document types, so each document has only one filled subfield (for example, "body.mail", "body.html"). The problem is that I could not perform a search on the entire "body" that would look through all its subfields (so as not to violate the existing application).
• New reincarnation of multipoles (to have a body field with a common analyzer and analyze mail, html, etc. inside it). Hovewer, I'm not sure if you can use them directly during indexing and indirectly during searches (for example, to save an object using {"mail":"smth"} to use a specific index analyzer, and then search for "query":{"body":"smth"} to use the universal search analyzer).
• To split the body into several fields with different mappings, remove them from _all and set copy_to to one body field. I'm not sure, but a significant invoice index will be added due to copying.

Answer 1

As I mentioned in the comments, what you want is not possible. Your requirement in one sentence is that: the same data is analyzed in several ways, but viewed as one field, because it will violate the existing application.

  -- body.html -- body.email body field ---- body.content --- all searched as "body" ... -- body.destination -- body.whatever 

• Your first option is multi-field fields that have this specific purpose: the same data is analyzed in several ways. The problem is that you cannot search for "body" and expect the ES to search for body.html , body.email ... Even if it is possible, you want to be searched in different analyzers. Again, this is not possible. For this parameter, you need to change the application and search for each field in multi_match or in query_string .

• The second option - reincarnation of multi-fields - will not work again, because you cannot link to body and ES in the background to match mail , content , etc.

• The third option - using copy_to - will not work, because copying to another “X” field means that the indexing of the copied data will be analyzed using the X analyzer , and this interrupts your requirement that the same data be analyzed in different ways.

• There may be a fourth option - "path": "just_name" from multi_fields - which should work at first glance. So, you can have 3 multi-user fields (email, content, html), which all three have a sub-field body . Having "path": "just_name" , you can only search for body , even if body is a subfield of several other fields. But this is impossible, because this type of multi-field will not accept different analyzers for the same body .

In any case, you need to change something in your requirements, because they will not work the way you want it.

---

Having said that, I am interested to know what queries you use in your application. This would be a simple change (yes, you will need to change your application) from querying the body field to querying body.* In multi_match .

And I have another solution for you: create multiple indexes, one index for each parser of your body . For example, for mail , content and html you define three indexes:

 PUT /multi_fields1 { "mappings": { "test": { "properties": { "body": { "type": "string", "index_analyzer": "whitespace", "search_analyzer": "standard" } } } } } PUT /multi_fields2 { "mappings": { "test": { "properties": { "body": { "type": "string", "index_analyzer": "standard", "search_analyzer": "standard" } } } } } PUT /multi_fields3 { "mappings": { "test": { "properties": { "body": { "type": "string", "index_analyzer": "keyword", "search_analyzer": "standard" } } } } } 

You see that they all have the same type and the same field name - body - but different index_analyzer s. Then you define an alias:

 POST _aliases { "actions": [ {"add": { "index": "multi_fields1", "alias": "multi"}}, {"add": { "index": "multi_fields2", "alias": "multi"}}, {"add": { "index": "multi_fields3", "alias": "multi"}} ] } 

Name your alias the same as your current index. The application does not need to be changed, it will use the same name to search by index, but this name will not point to the index, but to an alias, which, in turn, refers to your multiple indexes. What needs to be changed is how you index documents, because html documents need to be searched in multi_fields1 index, for example, an email document should be an index in multi_fields2 index, etc.

No matter what solution you find / choose, your requirements must change, because the way you want it is impossible.

Answer 2

I think you can use multi-user fields. With a multi-field field, you can define analyzers (both indexing and search) for each of the subfields and search the corresponding fields in accordance with the requirements of the applications. In general, an index analyzer may differ from field to field, the same for a search analyzer.

 {
   "your_type": {   
     "properties": {
         "body": {
             "type": "string",
             "index": "analyzed",
             "index_analyzer": "index_body_analyzer",
             "search_analyzer": "search_body_analyzer",
             "fields": {
                 "mail": {
                     "type": "string",
                     "index": "analyzed",
                     "index_analyzer": "index_bodymail_analyzer",
                     "search_analyzer": "search_bodymail_analyzer"
                 },
                 "html": {
                     "type": "string",              
                     "index": "analyzed",
                     "index_analyzer": "index_bodyhtml_analyzer",
                     "search_analyzer": "search_bodyhtml_analyzer"
                 }
             }
         }
     }
 }