Impossible to set dynamic templates in mappings when creating an index template

[lucacremonesi]

Elastic.Clients.Elasticsearch version:
8.18.0

Elasticsearch version:
8.18.0

.NET runtime version:
.NET Framework 4.8.9310.0

Operating system version:
Microsoft Windows 10.0.19045

Description of the problem including expected versus actual behavior:
When creating an index template with mappings that contain dynamic templates you will get the following server error:
Type: illegal_argument_exception Reason: "composable template [template_name] template after composition is invalid" CausedBy: "Type: illegal_argument_exception Reason: "invalid composite mappings for [template_name]" CausedBy: "Type: mapper_parsing_exception Reason: "Failed to parse mapping: A dynamic template must be defined with a name" CausedBy: "Type: mapper_parsing_exception Reason: "A dynamic template must be defined with a name""""

The client generates the request as follows:

{
    "index_patterns": "pattern*",
    "template": {
        "mappings": {
            "dynamic_templates": [
                {
                    "key": "copy_to_all_fields",
                    "value": {
                        "match_mapping_type": "string",
                        "unmatch": "all_fields",
                        "mapping": {
                            "copy_to": "all_fields",
                            "type": "text"
                        }
                    }
                }
            ],
            ...

Instead the client should generate the request as follows:

{
    "index_patterns": "pattern*",
    "template": {
        "mappings": {
            "dynamic_templates": [
                {
                    "copy_to_all_fields": {
                        "match_mapping_type": "string",
                        "unmatch": "all_fields",
                        "mapping": {
                            "copy_to": "all_fields",
                            "type": "text"
                        }
                    }
                }
            ],
            ...

Steps to reproduce:

1. Define DynamicTemplates: Dictionary<string, DynamicTemplate> DynamicTemplates;
2. Initialize DynamicTemplates:

var dynamicTemplate = DynamicTemplate.Mapping(new TextProperty() { CopyTo = "all_fields" });
dynamicTemplate.MatchMappingType = new[] { "string" };
dynamicTemplate.Unmatch = new[] { "all_fields" };

DynamicTemplates = new Dictionary<string, DynamicTemplate>
    {
        { "copy_to_all_fields", dynamicTemplate }
    };

3. Start creating an index template with dynamic templates in mappings:

    client.Indices.PutIndexTemplateAsync("template_name", pitr => pitr
        .IndexPatterns(new[] { "pattern*" })
        .Template(itm => itm
            .Settings(s => s
                ...
            )
            .Mappings(tm => tm
                .DynamicTemplates(DynamicTemplates)
                ...
            )
        )
    )

Expected behavior
The index template should be successfully created with the correct dynamic templates in mappings.

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

[flobernd]

Hi @lucacremonesi, thanks for reporting!

I just verified that this bug is fixed in 9.x. I plan to backport all bugfixes and improvements from 9.x for the upcoming 8.19 version.

[lucacremonesi]

Hi @flobernd,
Thanks for the quick reply.

I just wanted to add that the issue doesn't occur in version 8.17.4 of the Elasticsearch client, so it appears to be a regression in version 8.18.0.
Do you have an idea of when version 8.19 might be released?

[flobernd]

@lucacremonesi The 8.19 release of Elasticsearch (server) will be in the next few months, but I plan to release a -preview version of the 8.19 client as soon as it's ready. Should hopefully not take me much longer than 1-2 weeks.

[lucacremonesi]

@flobernd In the meanwhile, can we use the client 8.17.4 with the server 8.18.1? According to the documentation that should be possible but I am wondering if there are any known issues.

[flobernd]

@lucacremonesi That should work fine 🙂 The client is forward compatible with all minor versions and even with up to +1 major version (including all minors of the +1 major).