Query multilevel fields in Elastic Search

Question

I am new to Elastic Search and the non-SQL paradigm. I am following the ES tutorial, but there is one thing that I could not work for.

In the following code (I'me using PyES to interact with ES) I create a single document with a nested field (items) that contains another nested field (concepts).

from pyes import * conn = ES('127.0.0.1:9200') # Use HTTP # Delete and Create a new index. conn.indices.delete_index("documents-index") conn.create_index("documents-index") # Create a single document. document = { "docid": 123456789, "title": "This is the doc title.", "description": "This is the doc description.", "datepublished": 2005, "author": ["Joe", "John", "Charles"], "subjects": [{ "subjectname": 'subject1', "subjectid": [210, 311, 1012, 784, 568], "subjectkey": 2, "concepts": [ {"name": "concept1", "score": 75}, {"name": "concept2", "score": 55} ] }, { "subjectname": 'subject2', "subjectid": [111, 300, 141, 457, 748], "subjectkey": 0, "concepts": [ {"name": "concept3", "score": 88}, {"name": "concept4", "score": 55}, {"name": "concept5", "score": 66} ] }], } # Define the nested elements. mapping1 = { 'subjects': { 'type': 'nested' } } mapping2 = { 'concepts': { 'type': 'nested' } } conn.put_mapping("document", {'properties': mapping1}, ["documents-index"]) conn.put_mapping("subjects", {'properties': mapping2}, ["documents-index"]) # Insert document in 'documents-index' index. conn.index(document, "documents-index", "document", 1) # Refresh connection to make queries. conn.refresh() 

I can request the subject of a nested field:

 query1 = { "nested": { "path": "subjects", "score_mode": "avg", "query": { "bool": { "must": [ { "text": {"subjects.subjectname": "subject1"} }, { "range": {"subjects.subjectkey": {"gt": 1}} } ] } } } } results = conn.search(query=query1) for r in results: print r # as expected, it returns the entire document. 

but I can't figure out how to query based on the concepts of a nested field.

ES documentation means that

Multilevel nesting is automatically maintained and detected, as a result, the internal internal nested query automatically corresponds to the corresponding nesting level (rather than root) if it exists inside another nested query.

So, I tried to create a query in the following format:

 query2 = { "nested": { "path": "concepts", "score_mode": "avg", "query": { "bool": { "must": [ { "text": {"concepts.name": "concept1"} }, { "range": {"concepts.score": {"gt": 0}} } ] } } } } 

which returned 0 results.

I cannot understand what is missing, and I did not find any example with queries based on two levels of nesting.

Answer 1

Well, having tried the tone of the combinations, I finally got it using the following query:

 query3 = { "nested": { "path": "subjects", "score_mode": "avg", "query": { "bool": { "must": [ { "text": {"subjects.concepts.name": "concept1"} } ] } } } } 

Thus, the nested attribute path (themes) is always the same, regardless of the level of nested attributes, and in the definition of the request, I used the full attribute path (subject.concepts.name).

Answer 2

A shot in the dark, since I have not tried it personally, but have you tried the full path to the concepts?

 query2 = { "nested": { "path": "subjects.concepts", "score_mode": "avg", "query": { "bool": { "must": [ { "text": {"subjects.concepts.name": "concept1"} }, { "range": {"subjects.concepts.score": {"gt": 0}} } ] } } } } 

Answer 3

I have a question for JCJS answer. why should your comparison not like it?

 mapping = { "subjects": { "type": "nested", "properties": { "concepts": { "type": "nested" } } } } 

I am trying to determine that two types of mappings may not work, but they will smooth data; I think we should nest the nested properties.

Finally ... if we use this mapping, then the subquery should be like ...

 { "query": { "nested": { "path": "subjects.concepts", "query": { "term": { "name": { "value": "concept1" } } } } } } 

It is vital to use the full path attribute for the path attribute ... but not for the term key there may be a full path or relative path.