Documentation on Mapping and Nested Queries

[alikleitcr7]

Elastic.Clients.Elasticsearch version:
8.9.3 - (downgraded from latest 8.14.6 as the documentation does not comply with it)
Elasticsearch version:
8.14.2
.NET runtime version:
net7.0
Operating system version:
window 11

Description of the problem including expected versus actual behavior:

I wanted to post this as a question in the discussion board but I got "An error occurred: Sorry you cannot post a link to that host".

Therefore I'm writing this here.

I'm using the latest elasticsearch-net client, I could not find documentation or hints related to using nested queries or mapping but the deprecated ones (NEST).

What I tried:

searchResponse = await elasticsearchClient.SearchAsync<ElasticSearchTestDocument>(s => s
    .Index(indexName)
    .From(0)
    .Size(10)
    .Query(q =>
    {
        q.Nested(n =>
        {
            n.Path(p => p.Items.First())
            .Query(nq =>
            {
                nq.Bool(b =>
                {
                    b.Must(m =>
                    {
                        m.Match(mt =>
                        {
                            mt.Field(f => f.Items.First().ItemId)
                            .Query(item2Id.ToString());
                        });
                    });
                });
            });
        });
    })
);

this will throw an error "failed to find nested object under path"

added the ignore flag under Nested:

q.Nested(n =>
  {
   n.IgnoreUnmapped();

Query had no error but no documents.

I understand the Nested attribute was there in NEST but could not find it in the mentioned version.

    public class ElasticSearchTestDocument
    {
        public string Title { get; set; }
        [Nested]
        public List<ElasticSearchTestItemDocument> Items { get; set; }
    }

Is it experimental that there is only very simple/little CRUD documentation on the new API or am I missing something?

Any take on this would be appreciated.

[flobernd]

Hi @alikleitcr7, could you please post the JSON request produced by the client?

To print the JSON to the console, you could initialize the client like this:

var settings = new ElasticsearchClientSettings(new Uri("<redacted>"))
    .Authentication(new BasicAuthentication("elastic", "<redacted>"))
    .DisableDirectStreaming()
    .EnableDebugMode(cd =>
    {
        Console.WriteLine(cd.DebugInformation);
    });

var client = new ElasticsearchClient(settings);

[alikleitcr7]

Hi @flobernd, Here's the debug information:

Successful (200) low level call on POST: /test-index/_search?pretty=true&error_trace=true

# Audit trail of this API call:
 - [1] HealthyResponse: Node: http://localhost:9200/ Took: 00:00:00.0213502
# Request:
{
  "query": {
    "nested": {
      "query": {
        "bool": {
          "must": {
            "match": {
              "items.itemId": {
                "query": "e92d5dbb-0ed4-4d29-8850-23b0f934f94f"
              }
            }
          }
        }
      },
      "ignore_unmapped": true,
      "path": "items"
    }
  },
  "from": 0,
  "size": 10
}
# Response:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

# TCP states:
  Established: 56
  TimeWait: 24
  CloseWait: 5
  SynSent: 1

# ThreadPool statistics:
  Worker: 
    Busy: 2
    Free: 32765
    Min: 20
    Max: 32767
  IOCP: 
    Busy: 0
    Free: 1000
    Min: 1
    Max: 1000

Any comment on the documentation for this version?

[flobernd]

Hi @alikleitcr7, the produced JSON response looks correct to me, so I guess this might be a mapping problem. It probably makes sense trying to post in the community forums again (in the server category and not the client one) as I can't really help you with this.

8.9.3 - (downgraded from latest 8.14.6 as the documentation does not comply with it)

The nested query you posted still works in the latest version of the client. Mapping must be done manually like e.g.:

await client.Indices.PutMappingAsync<Person>(m => m.Properties(p => p.Nested(n => n.Data)));

Any comment on the documentation for this version?

The documentation is very sparse at the moment, but the new client (starting from version 8.13) closely maps to the structure of the REST JSON structure. Most method names can be infered by looking at the REST documentation.

If there are any concrete questions, feel free to ask them in the .NET client section of the community forums, or - as a last resort - open an issue here. Always happy to help!

[alikleitcr7]

@flobernd Thank you for the response. By community forumes you mean https://discuss.elastic.co/ ? could not find the section you are referring to, is it a category or tag? I saw one net related question posted under language-clients tag and ElasticSearch category

I will check with the mapping, there was an attribute [Nested] that seemed to be removed, which maps the field automatically without having to go though the above code. Do you by any chance know if we have an alternative on the class level for auto mapping?

Appreciated! 😄

[flobernd]

@alikleitcr7 Posting it here should probably be the right thing to do.

Do you by any chance know if we have an alternative on the class level for auto mapping?

Auto mapping is currently not implemented in the new client. It's on the roadmap, but I can't give you an ETA for this sadly.

[alikleitcr7]

Thank you @flobernd for the response,

Auto mapping is not implemented in the client.

IndexAsync does auto map the document contract, but the array Items ends up as type Object. I tried your suggestion (corrected with index name parameter) :

await client.Indices.PutMappingAsync<ElasticSearchTestDocument>(indexName, m => m.Properties(p => p.Nested(n => n.Items)));

Which did not affect the auto created dynamic index.

Thanks to this StackOverflow Post, What worked for me is deleting and recreating the index with CreateAsync and Mappings:

await elasticsearchClient.Indices.CreateAsync<ElasticSearchTestDocument>(indexName, im => im.Mappings(m =>
{
    m.Properties(p => p.Nested(n => n.Items));
}));

and after indexing a document it appended the rest of the properties to the map.

[flobernd]

@alikleitcr7 Yes, most of the times, index mappings can't be changed if there are already documents indexed.

Closing this issue for now as I think your question has been answered. I as well left you a comment on your other question in the community forums.