zarr

#zarr considered as a data format

Some projects are using zarr storage as a data format since it explicitly is centered around chunking support (though I have not found any specific ones on searching!?)

Zarr actually supports a multitude of storage representations, the main two types being:

• Each chunk goes into a separate file, but the organization of files with directories has multiple options.
• Chunks are stored in a database, with a number of database options provided.

We will only focus on the first option for this comparison.

Zarr storage does have support in multiple languages.

We will start on highlighting the support for how data can be organized as this is probably the biggest difference between zarr and ASDF.

Zarr allows attributes to be provided as a JSON object. S. presumably nesting in the JSON is permitted and thus one may have an arbitrarily deep hierarchy of attributes, though there is no mechanism for automatically turning these into special objects in the supporting language. Essentially you are provided a tree of dictionaries, lists, and basic types. These attributes are associated with an array.