Excluding and defining rules for missing fields

[martonvago]

Missing fields are a special case for rules and exclusions because they are not in the object.

Let's say we have the resource

{
    # "name": "woolly-dormice-2015",
    "title": "Body fat percentage in the hibernating woolly dormouse",
    "path": "resources/woolly-dormice-2015/data.parquet",
    # "version": "1.0.0",
}

• If version is missing, I can't define a rule for it, not even a "version must exist" rule. This is because there is nothing to apply the rule to.
  • Solution: apply it to None
• If name, a required field, is missing, I can't exclude it with jsonpath="...name" because it is not in the descriptor.
  • Solution: handle excluding required issues separately in a few lines of code

It wouldn't be very difficult to support these use cases. Should we?

[lwjohnst86]

Try it out! And yea, I was also noticing similar "edge cases".

[martonvago]

Hi @joelostblom ! Luke mentioned that you've worked with jsonschema before and fought with how it handles missing required fields. I'm not sure in what context exactly, but if you have any advice then don't hold back!

It's probably quite hard to understand what's going on in this particular case/issue without spending some time with the code, so I'm just thinking of generic tips maybe, not anything too concrete ☺️

[martonvago]

Should have been closed 🎉

[joelostblom]

Sorry for never replying here @martonvago. I totally missed that you tagged me back in Oct and only saw it now when going through some recently closed issues.

[martonvago]

It would have been too complicated to explain properly anyway, no worries at all!!