feat: Add support for all relevant keywords to get_schema_fields. Add name, deprecated_self, pattern, merge_by_id to Field.

- Rename definition_path to definition
- Remove support for null schema, which are invalid JSON Schema and prohibited by OCDS
- Remove definition_pointer, definition_pointer_components, definition_path_components from Field
- Remove support for adding fields to Field.__dict__

test: Add benchmark and regression tests. Add manage.py script to generate regression scenarios.

Author: jpmckinney

.github/workflows/ci.yml

@@ -34,4 +34,9 @@ jobs:
           pip install orjson
           coverage run --source=ocdskit --append -m pytest -W error -W default::ocdsmerge.exceptions.DuplicateIdValueWarning -W ignore::DeprecationWarning:requests_cache.models.response
           pip uninstall -y orjson
+      - name: Save libcove performance
+        env:
+          BENCHMARK_SAVE: 1
+        run: pytest --benchmark-save=libcove --benchmark-only
+      - run: pytest --benchmark-only --benchmark-compare=0001 --benchmark-compare-fail=median:0.001
       - uses: coverallsapp/github-action@v2

.gitignore

@@ -1,4 +1,5 @@
 /*.egg-info
+/.benchmarks
 /.coverage
 /build
 /dist

docs/changelog.rst

@@ -1,6 +1,34 @@
 Changelog
 =========
 
+1.2.1
+-----
+
+Added
+~~~~~
+
+-  :class:`ocdskit.schema.Field`: Add ``name``, ``deprecated_self``, ``pattern``, ``merge_by_id``.
+-  :func:`ocdskit.schema.get_schema_fields`:
+
+   -  Add support for ``allOf``, ``anyOf``, ``oneOf``, ``then``, ``else``, ``$defs``.
+   -  Add support for ``items`` within ``definitions``.
+   -  Add support for ``patternProperties`` and ``items`` within ``items``.
+   -  Add support for ``properties``, ``patternProperties`` and ``items`` within ``patternProperties``.
+
+Changed
+~~~~~~~
+
+-  :class:`ocdskit.schema.Field`: Rename ``definition_path`` to ``definition``.
+
+Removed
+~~~~~~~
+
+-  :func:`ocdskit.schema.get_schema_fields`: Remove support for ``null`` schema, which is invalid.
+-  :class:`ocdskit.schema.Field`:
+
+   -  Remove ``definition_pointer``, ``definition_pointer_components``, ``definition_path_components``.
+   -  Remove support for adding fields to ``__dict__``.
+
 1.2.0 (2024-09-15)
 ------------------
 

docs/contributing.rst

@@ -44,23 +44,46 @@ And run, for example:
 
 .. code-block:: bash
 
-    python -c 'import json; print("\n".join(json.dumps({"releases": [{"ocid": str(y), "date": ""} for x in range(100)]}) for y in range(10000)))' | ocdskit compile --package > /dev/null
-    python -c 'print("\n".join(map(str, range(5000000))))' | ocdskit package-records > /dev/null
-    python -c 'print("\n".join(map(str, range(5000000))))' | ocdskit package-releases > /dev/null
-    python -c 'import json; print("\n".join(json.dumps({"records": list(range(500))}) for x in range(10000)))' | ocdskit echo --root-path records.item | ocdskit package-records --size 999 > /dev/null
-    python -c 'import json; print("\n".join(json.dumps({"releases": list(range(500))}) for x in range(10000)))' | ocdskit echo --root-path releases.item | ocdskit package-releases --size 999 > /dev/null
+   python -c 'import json; print("\n".join(json.dumps({"releases": [{"ocid": str(y), "date": ""} for x in range(100)]}) for y in range(10000)))' | ocdskit compile --package > /dev/null
+   python -c 'print("\n".join(map(str, range(5000000))))' | ocdskit package-records > /dev/null
+   python -c 'print("\n".join(map(str, range(5000000))))' | ocdskit package-releases > /dev/null
+   python -c 'import json; print("\n".join(json.dumps({"records": list(range(500))}) for x in range(10000)))' | ocdskit echo --root-path records.item | ocdskit package-records --size 999 > /dev/null
+   python -c 'import json; print("\n".join(json.dumps({"releases": list(range(500))}) for x in range(10000)))' | ocdskit echo --root-path releases.item | ocdskit package-releases --size 999 > /dev/null
 
 To test whether commands stream input, you can run, for example:
 
 .. code-block:: bash
 
-    echo 'cat tests/fixtures/realdata/record-package_versioned.json tests/fixtures/realdata/record-package_versioned.json; sleep 3; cat tests/fixtures/record-package_minimal.json' > input.sh
-    sh input.sh | ocdskit upgrade 1.0:1.1
-    sh input.sh | ocdskit split-record-packages 1
+   echo 'cat tests/fixtures/realdata/record-package_versioned.json tests/fixtures/realdata/record-package_versioned.json; sleep 3; cat tests/fixtures/record-package_minimal.json' > input.sh
+   sh input.sh | ocdskit upgrade 1.0:1.1
+   sh input.sh | ocdskit split-record-packages 1
 
 .. code-block:: bash
 
-    echo 'cat tests/fixtures/realdata/release-package-1-2.json tests/fixtures/realdata/release-package-1-2.json; sleep 7; cat tests/fixtures/release-package_minimal.json' > input.sh
-    sh input.sh | ocdskit split-release-packages 1
+   echo 'cat tests/fixtures/realdata/release-package-1-2.json tests/fixtures/realdata/release-package-1-2.json; sleep 7; cat tests/fixtures/release-package_minimal.json' > input.sh
+   sh input.sh | ocdskit split-release-packages 1
 
 You can run ``sh input.sh | tee`` to compare the timing of ``tee`` to the timings above.
+
+Performance
+-----------
+
+pytest-benchmark tests ensure that :func:`ocdskit.schema.get_schema_fields` is faster than `libcove <https://pypi.org/project/libcove/>`__.
+
+Before making changes:
+
+.. code-block:: bash
+
+   pytest --benchmark-only --benchmark-min-rounds=10 --benchmark-save=tmp
+
+After making changes:
+
+.. code-block:: bash
+
+   pytest --benchmark-only --benchmark-min-rounds=10 --benchmark-compare=0002 --benchmark-compare-fail=mean:0%
+
+Clean up saved benchmarks:
+
+.. code-block:: bash
+
+   rm -f .benchmarks/*/*_tmp.json