Looking for everyday JSON examples

[ross-spencer]

Working on RO-CRATE recently I have submitted a new proposal for PRONOM for identification. Thinking about JSON more deeply, the identification of the specific characteristics of an object or document feels like a tricky problem, e.g. depending on how an object is formatted, we can see equivalent structures with different amounts of whitespace:

{
    "@context": [
        "https://w3id.org/ro/crate/1.1/context"
    ],

{
    "@context": ["https://w3id.org/ro/crate/1.1/context" ],

{
    "@context": [      "https://w3id.org/ro/crate/1.1/context"      ],

PRONOM deals with this in one example using a wildcard: 7B*2269645F737472223A*22726574776565746564223

I can't recall the impact of * in a signature, although I believe ID tools will optimize for the existence of { in the case of JSON which works well if the file doesn't have any whitespace at the beginning.

Additionally, encoding more complex rules, such as identifying keys where keys aren't guaranteed an order by the JSON spec gets complicated very quickly, just two keys will require two signatures in PRONOM.

{
    "key 1": "value1",
    "key 2": "value2"
}

{
    "key 2": "value2",
    "key 1": "value1"
}

I have written a small prototype identifier in Python that proposes an ideal ruleset for identifying JSON more reliably.

I have started recording different rules for JSON in a registry module.

And described its potential relationship to PRONOM here.

Like other discussions, this is to do a couple of things:

1. raise the idea, and seek discussion, e.g.
   i. comments on the approach,
   ii. discussion about PRONOM hosting more records, but potentially without signature patterns for JSON so that there are PUIDs available.
2. Collect more about day-to-day examples of JSON that folks are seeing in their collections to build more of an inventory that can be used to create jsonid signatures and potentially feed these back into PRONOM as entries and/or byte signatures.

Looking forward to talking more. If folks have examples of JSON that would be great! If you can paste examples in your replies, I can use them to build signatures. Links to specifications are also helpful. Largely just hoping to gather more data at the moment.

[petervwyatt]

Not exactly samples, but some interesting work on the difficulties and ambiguities of parsing JSON!:

• Nicolas Seriot (no date) seriot.ch - Parsing JSON is a Minefield 💣. http://seriot.ch/parsing_json.php.

• Miller, J. (no date) An Exploration of JSON Interoperability Vulnerabilities. https://labs.bishopfox.com/tech-blog/an-exploration-of-json-interoperability-vulnerabilities

[ross-spencer]

Very nice, thank you Peter!