Deduction deserialization - add support for pojo-hierarchies such that the absence of child fields infers a supertype

[pakorcz]

Currently, deduction deserialization requires all values to have unique field. Because of that, there is no way to deserialize values to superclass and subclasses, when defaultImpl is different than superclass.

Example:

For json:

 "name": "Italy",
 "places": {
   "mountains": {
     "name": "The Alps"
   },
   "cites": [
     {
       "name": "Rome",
       "places": {
         "colosseum": {
           "name": "The Colosseum"
         },
         "romanForum": {
           "name": "The Roman Forum"
         }
       }
     },
     {
       "name": "Venice",
       "places": {
         "bridges": []
       }
     }
   ]
 }
}

and pojo-hierarchy:

public static class Place implements WorthSeeing {  
  public String name;  
}  
  
public static class CompositePlace extends Place implements WorthSeeing {  
  
  public Map<String, WorthSeeing> places;  
}  
  
static class ListOfPlaces extends ArrayList<WorthSeeing> implements WorthSeeing {  
}

and deduction deserialization with defaultImpl differ then supertype:

@JsonTypeInfo(use = DEDUCTION, defaultImpl = ListOfPlaces.class)  
@JsonSubTypes( {@Type(ListOfPlaces.class), @Type(CompositePlace.class), @Type(Place.class)})  
interface WorthSeeing {}

the MismatchedInputException is thrown, because the values with only name fields are not classified to Place .

Desired behaviour:

• value contains only fields from superclass -> it's mapped to an object of superclass
• value contains fields from superclass and a subclass -> it's mapped to an object of the supclass
• value contains fields from superclass and fields that don't exist in any of subclasses -> the UnrecognizedPropertyException is thrown

[cowtowncoder]

I am not sure whether this is a valid idea, will have to defer to @drekbour.

[drekbour]

Acknowledged, yet to dig and and respond.

[drekbour]

Hi @pakorcz. I refer you to #2976 (comment) and the comments which follow it. The given example (great to see something not involving animals!) makes assumptions on a vanilla usecase that mean it's ok to infer the supertype. Those assumptions will not always hold true and the logic will be chaotic in other configurations.

@cowtowncoder Much as it pains me to say, I don't think the change is stable to include.

I really opened a can o' worms with this didn't I! We should keep collecting these requirements (I think they are all the same requirement really but expressed in usefully different ways as users try to hack around the limitations) and perhaps they will weave a clearer picture of where a general-case solution can safely add value.

I take from this issue the reasonable expectation that supertypes might be ok when they are explicitly included in the set of JsonSubTypes.

PS. There exists an alternative implementation of the deduction code which I wrote after the initial PR. It should be faster and would better handle terminal conditions - though nothing can get away from the fact that deser code doesn't why a field is absent given the many options. I'll PR it so it's not lost.

[cowtowncoder]

Yeah I was guessing it would not be something we'd want to do for general usage -- there tends to lots of logic that needs to be added to cover all kinds of cases that "make sense" to one user or use case.

+1 for collecting requirements if anyone wants to volunteer -- one possible place would

https://github.com/FasterXML/jackson-future-ideas/wiki

either new JSTEP-x for "improved deduction", or just another wiki page linked.
Or an issue.

Ideally I think there should be alternative way to add Type Id detection, given JsonNode (which does require full buffering but it is what it is), and let users implement more advanced logic.
But that's not easy given existing API design.

[drekbour]

I'll raise a new feature because I think I have a workable solution in mind now. It justs needs some meditation before I raise

[cowtowncoder]

Creating an issue as Work-in-Progress sounds reasonable; can always edit/change/close/re-create if need be.

[pakorcz]

Thank you guys for explanation and taking an interest in this case!

The point of it is exactly what @drekbour wrote:
“the expectation that supertypes might be ok when they are explicitly included in the set of JsonSubTypes”.

I would really appreciate this feature, because thanks to it – if set of possible json data reflects some kind of hierarchy – then it could be deserialized out of the box to proper class of pojo-hierarchy ( without further mapping and checking of null values).

[barkefors]

👋 I wonder if it would be an option to allow opting in to this behavior by explicitly declaring these disambiguating expectations. (Let me first say that while I've been using Jackson since at least as long as I can remember, I've never ventured out here before so apologies if this idea has already been discussed or is fundamentally broken in some way!)

In my understanding the presence of a property is in a natural way a fulfillment of the requiredness of a property, so what if we could resolve cases like the example here by putting the onus on the user, letting them declare CompositePlace as

class CompositePlace extends Place implements WorthSeeing {  
  @JsonProperty(required = true)
  public Map<String, WorthSeeing> places;  
}

// or
public CompositePlace(@JsonProperty(value="places", required=true) Map<String, WorthSeeing> places) {...}

// or
record CompositePlace(@JsonProperty(required=true) Map<String, WorthSeeing> places) implements WorthSeeing {}

which given {"name": "BarkingDuck"} would then discard CompositePlace as a candidate since places is missing. I'd argue that doing it this way would to some extent address concerns re. Include.NON_NULL, by requiring the user to decide whether an X without a Y is allowed to be an X or not. (Bird/Wingspan, CompositePlace/Place, OncallEngineer/PagerNumber...)

I believe that this could be implemented in a quite similar way to what #3290 does:

0. When building fingerprints, also construct a Map<BitSet, BitSet> of required properties per type keyed by fingerprint
1. Keep track of seen, known properties while iterating over fields in BitSet actual
2. After iterating over fields, iterate over candidates removing if required.andNot(actual) is not empty
3. Check again if candidates.size() == 1, otherwise proceed

Thanks for your great work on this amazing library! ❤️

[drekbour]

Thankyou for thinking through the scenario and code. Honouring required=true is a good addition to the deduction process and goes onto my list.

[TCke83]

+1 would love to see this in the next version, can't use the deduction now because the supertypes don't get deducted