Skip to content

Commit 4d4d833

Browse files
authored
v2 spec: add optional dimension_separator (see #707) (#715)
* v2 spec: add optional dimension_separator (see #707) Various implementations allow for defining the separator between the dimension indexes when writing chunks: * n5-zarr defines a `dimensionSeparator` parameter; * zarr-python's NestedDirectoryStore does so by default * and FSStore provides a `key_separator` parameter; * tensorstore has a `key_encoding` parameter; and * jzarr is looking to add the same functionality. When writing an array, it is straight-forward to set this separator and have arrays properly configured. Consumers of such arrays, however, must either know *a priori* if their arrays use a non-default separator or must loop through all possible chunks keys searching for the right one. By defining adding an optional metadata key to the .zarray, we: * preserve the efficient configuration of arrays * while keeping the v2 spec backwards compatible. The primary downsides are that this will be the first optional metadata value in the v2 spec and therefore we don't have a strong understanding of how that will play out, and datasets which were previously written with non-default separators will need updating in order to enable the detection though that is no worse than the current situation. * Update dim. sep. description after feedback * Remove `MUST NOT` restriction for other keys
1 parent 648154d commit 4d4d833

File tree

1 file changed

+12
-1
lines changed

1 file changed

+12
-1
lines changed

docs/spec/v2.rst

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,18 @@ filters
7878
filters are to be applied. Each codec configuration object MUST contain a
7979
``"id"`` key identifying the codec to be used.
8080

81-
Other keys MUST NOT be present within the metadata object.
81+
The following keys MAY be present within the object:
82+
83+
dimension_separator
84+
If present, either the string ``"."`` or ``"/""`` definining the separator placed
85+
between the dimensions of a chunk. If the value is not set, then the
86+
default MUST be assumed to be ``"."``, leading to chunk keys of the form "0.0".
87+
Arrays defined with ``"/"`` as the dimension separator can be considered to have
88+
nested, or hierarchical, keys of the form "0/0" that SHOULD where possible
89+
produce a directory-like structure.
90+
91+
Other keys SHOULD NOT be present within the metadata object and SHOULD be
92+
ignored by implementations.
8293

8394
For example, the JSON object below defines a 2-dimensional array of 64-bit
8495
little-endian floating point numbers with 10000 rows and 10000 columns, divided

0 commit comments

Comments
 (0)