`@JsonAnyGetter` serialization order change from 2.18.4 to 2.19.0

[eddumelendez]

Search before asking

• I searched in the issues and found nothing similar.

Describe the bug

Using JsonAnyGetter used to return property's value from this source instead of the field annotated with JsonProperty until 2.18.4. Starting with 2.19.0, the behavior changed and now value from property is always returned.

Version Information

2.19.1

Reproduction

I am attaching a reproducer with Java 17 https://github.com/eddumelendez/jackson-deserializer-issue with the very exact configuration I use in my project.

Expected behavior

read the value from JsonAnyGetter

Additional context

No response

[pjfanning]

sounds a bit like #5204 (but not definitely the same issues)

[pjfanning]

https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.19 mentions a couple of changes in the area but don't jump out as related

[JooHyukKim]

JsonAnyGetter is for serialization. I saw ur test qnd it seems to do round-trip.

Could u try out again with hard coded JSON so we can see what the output for each of serialzation and deserialization looks like?

[pjfanning]

In #5205 (comment) - we also see a case where the serialization changed and the reporter thought that the issue was a deserialization issue.

[JooHyukKim]

Observation

Okay did some digging.
With .enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY) on the mapper, test passes.

It turns out 2.18.4 version serialization output is...

{"Property1":"value1","Property2":null,"Property2":"value2"}

... this, test, passing with "Property2":null coming first.

But in 2.19 the serialization output is ...like

{"Property2":"value2","Property1":"value1","Property2":null}

... this, test fails. "Property2":null is comes second.

Thoughts

I vaguely recall Jackson does not guarantee order of serialization, unless configured so (e.g. via MapperFeature.SORT_PROPERTIES_ALPHABETICALLY). So even though regression is caused by underlying @JsonAnyGetter implementation change, fix might have to be done on deserialization side.

Reproduction

• Best effort to bring reproduction here in one piece :-/ from remote fork repo.

import java.io.IOException;
import java.io.Serializable;
import java.util.*;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

import com.fasterxml.jackson.annotation.*;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier;
import com.fasterxml.jackson.databind.deser.std.DelegatingDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;

import static org.assertj.core.api.Assertions.assertThat;

class JacksonJsonAnyGetterTest {

    @Test
    void test() throws JsonProcessingException {
        Child child = new Child();
        child.setProperty1("value1");
        Accessor.overrideRawValue(child, "Property2", "value2");

        ObjectMapper objectMapper = new ObjectMapper()
                .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
                .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)
                ;
        ObjectMapper originalObjectMapper = objectMapper.copy();
        objectMapper.registerModule(new SimpleModule("docker-java") {
            @Override
            public void setupModule(SetupContext context) {
                super.setupModule(context);
                context.addBeanDeserializerModifier(new BeanDeserializerModifier() {
                    @Override
                    public JsonDeserializer<?> modifyDeserializer(DeserializationConfig config, BeanDescription beanDescription, JsonDeserializer<?> originalDeserializer) {
                        if (!beanDescription.getType().isTypeOrSubTypeOf(Parent.class)) { return originalDeserializer;}
                        return new DockerObjectDeserializer(originalDeserializer, beanDescription,originalObjectMapper
                        );
                    }
                });
            }
        });

//        String json = objectMapper.writeValueAsString(child);
        // Without
        // org.opentest4j.AssertionFailedError:
//        Assertions.assertEquals("{\"Property1\":\"value1\",\"Property2\":null,\"Property2\":\"value2\"}", json);
        Child child1 = objectMapper.readValue("{\"Property1\":\"value1\",\"Property2\":\"value2\",\"Property2\":null}\n", Child.class);
        assertThat(child1.getProperty2()).isEqualTo(null);

        Child child2 = objectMapper.readValue("{\"Property1\":\"value1\",\"Property2\":null,\"Property2\":\"value2\"}\n", Child.class);
        assertThat(child2.getProperty2()).isEqualTo("value2");
    }

    static abstract class Parent {
        HashMap<String, Object> rawValues = new HashMap<>();
        @JsonAnyGetter
        public Map<String, Object> getRawValues() {
            return Collections.unmodifiableMap(this.rawValues);
        }
    }

    static class Child extends Parent implements Serializable {
        @JsonProperty("Property1")
        private String property1;
        @JsonProperty("Property2")
        private String property2;
        public String getProperty1() {return property1;}
        public void setProperty1(String property1) {this.property1 = property1;}
        public String getProperty2() {return property2;}
        public void setProperty2(String property2) {this.property2 = property2;}
    }

    static class DockerObjectDeserializer extends DelegatingDeserializer {
        private final BeanDescription beanDescription;
        private final ObjectMapper originalMapper;
        DockerObjectDeserializer(JsonDeserializer<?> delegate, BeanDescription beanDescription, ObjectMapper originalMapper) {
            super(delegate);
            this.beanDescription = beanDescription;
            this.originalMapper = originalMapper;
        }

        @Override
        protected JsonDeserializer<?> newDelegatingInstance(JsonDeserializer<?> newDelegatee) {
            return new DockerObjectDeserializer(newDelegatee, beanDescription, originalMapper);
        }

        @Override
        @SuppressWarnings({"deprecation", "unchecked"})
        public Object deserialize(JsonParser p, DeserializationContext ctxtx) throws IOException {
            JsonNode jsonNode = p.readValueAsTree();
            Object deserializedObject = originalMapper.treeToValue(jsonNode, beanDescription.getBeanClass());

            if (deserializedObject instanceof Parent) {
                Accessor.overrideRawValues(((Parent) deserializedObject), originalMapper.convertValue(jsonNode, HashMap.class));
            }
            return deserializedObject;
        }

    }

    static class Accessor {
        static void overrideRawValues(Parent o, HashMap<String, Object> rawValues) {
            o.rawValues = rawValues != null ? rawValues : new HashMap<>();
        }
        public static void overrideRawValue(Parent o, String key, Object value) {
            o.rawValues.put(key, value);
        }
    }
}

[JooHyukKim]

Okay I modified the assertions to use hardcoded JSON like below and both 2.18 and 2.19 versions pass.

Child child1 = objectMapper.readValue("{\"Property1\":\"value1\",\"Property2\":\"value2\",\"Property2\":null}\n", Child.class);
assertThat(child1.getProperty2()).isEqualTo(null);

Child child2 = objectMapper.readValue("{\"Property1\":\"value1\",\"Property2\":null,\"Property2\":\"value2\"}\n", Child.class);
assertThat(child2.getProperty2()).isEqualTo("value2");

So basically serialization order is changed from 2.18 to 2.19.
Deserializer behavior same (title may be changed accordingly)

Not sure if we should support case of writing duplicating JSON keys because the behavior happens to work...

[ajacob]

Following a recent migration from jackson 2.18.4 to 2.19, I noticed this behavior change.
This issue was not opened when I started to work on it so I did an investigation and wanted to share this for you in case it can help:

Test case

• 2.18.4: pass ✅
• 2.19.0: fails 🔴

package com.fasterxml.jackson.databind.ajacob;

import com.fasterxml.jackson.annotation.JsonAnyGetter;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.databind.MapperFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import org.junit.jupiter.api.Test;

import java.util.LinkedHashMap;
import java.util.Map;

import static org.junit.jupiter.api.Assertions.assertEquals;

public class AnyGetterSerializationOrderChangeTest
{

    static class DynaBean {
        public String l;
        public String j;
        public String a;

        protected Map<String, Object> extensions = new LinkedHashMap<>();

        @JsonAnyGetter
        public Map<String, Object> getExtensions() {
            return extensions;
        }

        @JsonAnySetter
        public void addExtension(String name, Object value) {
            extensions.put(name, value);
        }
    }

    /*
    /**********************************************************
    /* Test cases
    /**********************************************************
     */

    private final ObjectMapper MAPPER = JsonMapper.builder()
            .enable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY)
            .configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true)
            .build();

    @Test
    public void testDynaBean() throws Exception
    {
        DynaBean b = new DynaBean();
        b.a = "1";
        b.j = "2";
        b.l = "3";
        b.addExtension("z", "5");
        b.addExtension("b", "4");
        assertEquals("{\"a\":\"1\",\"j\":\"2\",\"l\":\"3\",\"b\":\"4\",\"z\":\"5\"}", MAPPER.writeValueAsString(b));
    }
}

My understanding is that in 2.18.4 the props from the @JsonAnyGetter field were added at the end (after ordering of fields from host classes). Now in 2.19.0 the props are added following the order based on the name of the @JsonAnyGetter field.
In my test case the field is named extensions, meaning that the props are put per alphabetical order from this field name.

Reverting the following PR on 2.19.0 makes the test pass: #4775

I don't know if this behavior change was intended, if this should be the newly accepted default, or if it should be considered as a bug

[JooHyukKim]

@ajacob Thank you for your report! Let's see if we can get this fixed

[JooHyukKim]

Wrote simple #5216 PR to start off...