Create an index with multiple field matching syntax using NEST 2.x

Question

I just can't understand that the syntax is correct for displaying multiple fields in NEST 2.0 - if that is the correct terminology. Each example I found for comparison seems to be equal to the <= 1.x version of NEST. I am new to Elasticsearch and NEST, and I read their documentation, but the NEST documentation has not been completely updated for 2.x.

Basically, I don't need to index or store the whole type. Some fields that I need only for indexing, some fields I need to index and retrieve, and some I do not need for indexing, just for retrieval.

MyType { // Index this & allow for retrieval. int Id { get; set; } // Index this & allow for retrieval. // **Also**, in my searching & sorting, I need to sort on this **entire** field, not just individual tokens. string CompanyName { get; set; } // Don't index this for searching, but do store for display. DateTime CreatedDate { get; set; } // Index this for searching BUT NOT for retrieval/displaying. string CompanyDescription { get; set; } // Nest this. List<MyChildType> Locations { get; set; } } MyChildType { // Index this & allow for retrieval. string LocationName { get; set; } // etc. other properties. } 

I have the ability to index the entire object, and the child as-is uses the following as an example:

 client.Index(item, i => i.Index(indexName)); 

However, the actual object is much larger than this, and I really don't need most of it. I found this, which is similar to what I think I want to do, but in an older version: elasticsearch multi-user mapping

I think “matching” is what I'm going to, but, as I said, I'm new to Elasticsearch and NEST, and I'm trying to learn terminology.

Be careful! :) This is my first time to ask a question about SO. Thanks!

Answer 1

As far as I can see, you do not have the complex types that you are trying to display. This way you can easily use NEST attributes to map your objects.

Check this:

 [Nest.ElasticsearchType] public class MyType { // Index this & allow for retrieval. [Nest.Number(Store=true)] int Id { get; set; } // Index this & allow for retrieval. // **Also**, in my searching & sorting, I need to sort on this **entire** field, not just individual tokens. [Nest.String(Store = true, Index=Nest.FieldIndexOption.Analyzed, TermVector=Nest.TermVectorOption.WithPositionsOffsets)] string CompanyName { get; set; } // Don't index this for searching, but do store for display. [Nest.Date(Store=true, Index=Nest.NonStringIndexOption.No)] DateTime CreatedDate { get; set; } // Index this for searching BUT NOT for retrieval/displaying. [Nest.String(Store=false, Index=Nest.FieldIndexOption.Analyzed)] string CompanyDescription { get; set; } [Nest.Nested(Store=true, IncludeInAll=true)] // Nest this. List<MyChildType> Locations { get; set; } } [Nest.ElasticsearchType] public class MyChildType { // Index this & allow for retrieval. [Nest.String(Store=true, Index = Nest.FieldIndexOption.Analyzed)] string LocationName { get; set; } // etc. other properties. } 

After this declaration, in order to create this mapping in elasticsearch, you need to make a call similar to:

 var mappingResponse = elasticClient.Map<MyType>(m => m.AutoMap()); 

With an AutoMap () call, NEST will read your attributes from your POCO and create a matching request accordingly.

Also see the Attribute-Based Mapping section of here .

Hooray!

Answer 2

In addition to the answers of Colin and Selcuk, you can also completely control the display through freely (and the object initialization API). Here is an example based on your requirements

 void Main() { var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); var connectionSettings = new ConnectionSettings(pool); var client = new ElasticClient(connectionSettings); client.Map<MyType>(m => m .Index("index-name") .AutoMap() .Properties(p => p .String(s => s .Name(n => n.CompanyName) .Fields(f => f .String(ss => ss .Name("raw") .NotAnalyzed() ) ) ) .Date(d => d .Name(n => n.CreatedDate) .Index(NonStringIndexOption.No) ) .String(s => s .Name(n => n.CompanyDescription) .Store(false) ) .Nested<MyChildType>(n => n .Name(nn => nn.Locations.First()) .AutoMap() .Properties(pp => pp /* properties of MyChildType */ ) ) ) ); } public class MyType { // Index this & allow for retrieval. public int Id { get; set; } // Index this & allow for retrieval. // **Also**, in my searching & sorting, I need to sort on this **entire** field, not just individual tokens. public string CompanyName { get; set; } // Don't index this for searching, but do store for display. public DateTime CreatedDate { get; set; } // Index this for searching BUT NOT for retrieval/displaying. public string CompanyDescription { get; set; } // Nest this. public List<MyChildType> Locations { get; set; } } public class MyChildType { // Index this & allow for retrieval. public string LocationName { get; set; } // etc. other properties. } 

This leads to display

 { "properties": { "id": { "type": "integer" }, "companyName": { "type": "string", "fields": { "raw": { "type": "string", "index": "not_analyzed" } } }, "createdDate": { "type": "date", "index": "no" }, "companyDescription": { "type": "string", "store": false }, "locations": { "type": "nested", "properties": { "locationName": { "type": "string" } } } } } 

The .AutoMap() call causes NEST to display a mapping based on property types and any attributes that apply to them. Then .Properties() overrides any of the displayed mappings. for instance

• CompanyName displayed as multi_field with the CompanyName field parsed using a standard parser, and companyName.raw not parsed. You can reference the latter in your queries using .Field(f => f.CompanyName.Suffix("raw"))
• Locations displayed as a nested type (automatic conversion by default displays this as an object type mapping). You can then define any specific mappings for MyChildType with .Properties() inside the Nested<MyChildType>() call.

Answer 3

At the time of this writing, Nest does not offer a way to map a property in your class to multiple fields when matching documents using built-in attributes. However, it does provide the tools you need for something, with your mappings, which you could do if you yourself wrote JSON.

Here is the solution I put together for my needs. It should not be difficult to use as a starting point for everything you need to do.

Firstly, here is a display example that I want to generate

 { "product":{ "properties":{ "name":{ "type":"string", "index":"not_analyzed", "fields":{ "standard":{ "type":"string", "analyzer":"standard" } } } } } } 

The product document will have a name field that is indexed but not parsed, and a name.standard field that uses a standard parser.

The C # class I create looks like this:

 [ElasticsearchType] public class Product { [WantsStandardAnalysisField] public string Name { get; set; } } 

Note the WantsStandardAnalysisField attribute. This is a custom attribute added without special properties. Literally simple:

 public class WantsStandardAnalysisField : Attribute {} 

If I used AutoMap as-is, my custom attribute would be ignored and I would get a mapping to the name field, but not name.standard . Fortunately, AutoMap accepts an instance of IPropertyVisitor . A base class called NoopPropertyVisitor implements the interface and does nothing, so you can subclass it and override only the methods you need. When you use the property visitor using AutoMap, it will create a document mapping for you, but it will give you the opportunity to change it before it is submitted to Elastic Search. All we need to do is find the properties marked with our custom attribute and add a field to them.

Here is an example that does this:

 public class ProductPropertyVisitor : NoopPropertyVisitor { public override void Visit(IStringProperty type, PropertyInfo propertyInfo, ElasticsearchPropertyAttributeBase attribute) { base.Visit(type, propertyInfo, attribute); var wsaf = propertyInfo.GetCustomAttribute<WantsStandardAnalysisField>(); if (wsaf != null) { type.Index = FieldIndexOption.NotAnalyzed; type.Fields = new Properties { { "standard", new StringProperty { Index = FieldIndexOption.Analyzed, Analyzer = "standard" } } }; } } } 

As you can see, we can do whatever we want with the generated property, including disabling the analysis for the main property and adding a new field with its own settings. For fun, you can add a couple of properties to a custom attribute to specify the name of the field and the analyzer. You can even change the code to see if the attribute has been added multiple times, allowing you to add as many fields as you want.

If you must run this using any method that generates a mapping using AutoMap, for example:

 new TypeMappingDescriptor<Product>().AutoMap(new ProductPropertyVisitor()) 

You will get the required multi-field mapping. Now you can configure the mappings for your heart content. Enjoy it!

Answer 4

I think you have at least 2 possibilities to solve your problem:

• By indexing: create something like a metadata model that is stored for retrieval only. See _ source field to limit the return to this field.
• When searching: specify the fields you want to request: if you do not want to request the created file, just do not include it in your search.

In my case, I use both of these approaches to get very quick results :-)