Tests for the test suite #468

m-mohr · 2023-10-12T12:17:41Z

See the README for details.

This PR includes the changes from PR #490.

tests/README.md

soxofaan · 2024-01-05T09:13:10Z

I have the impression a lot of changes in the diff come from PR #490
Is that intended?

Co-authored-by: Stefaan Lippens <soxofaan@users.noreply.github.com>

m-mohr · 2025-08-07T18:51:05Z

Thank you @soxofaan. I've rebased and the diff looks significantly cleaner now.

soxofaan

This PR is too large to give it a full in-depth review, but here are some random notes.

I guess it's more important to get this merged as "good enough" so that implementations can start integrate it, than to reach perfection.

absolute.json

soxofaan · 2025-08-22T13:57:10Z

apply_kernel.json

@@ -1,7 +1,7 @@
 {
    "id": "apply_kernel",
    "summary": "Apply a spatial convolution with a kernel",
-    "description": "Applies a 2D convolution (i.e. a focal operation with a weighted kernel) on the horizontal spatial dimensions (axes `x` and `y`) of a raster data cube.\n\nEach value in the kernel is multiplied with the corresponding pixel value and all products are summed up afterwards. The sum is then multiplied with the factor.\n\nThe process can't handle non-numerical or infinite numerical values in the data cube. Boolean values are converted to integers (`false` = 0, `true` = 1), but all other non-numerical or infinite values are replaced with zeroes by default (see parameter `replace_invalid`).\n\nFor cases requiring more generic focal operations or non-numerical values, see ``apply_neighborhood()``.",
+    "description": "Applies a 2D convolution (i.e. a focal operation with a weighted kernel) on the horizontal spatial dimensions (axes `x` and `y`) of a raster data cube.\n\nEach value in the kernel is multiplied with the corresponding pixel value and all products are summed up afterwards. The sum is then multiplied with the factor.\n\nThe process can't handle non-numerical or infinite numerical values in the data cube. Boolean values are converted to integers (`false` = 0, `true` = 1), but all other non-numerical, NaN, no-data, or infinite values are replaced with zeroes by default (see parameter `replace_invalid`).\n\nFor cases requiring more generic focal operations or non-numerical values, see ``apply_neighborhood()``.",


these changes look unrelated to the scope of this PR, right?

Well, they were spotted when creating the tests. They should've been in #490, but it seems I didn't catch them all.

tests/README.md

soxofaan · 2025-08-22T14:08:19Z

tests/README.md

+- The input and output values for no-data values are `null` by default unless otherwise specified by a runner.
+- Input that is not valid according to the schemas, will be rejected upfront and will not be checked on. For example, the absolute process only tests against the data types `number` and `null`. There are no tests for a boolean or string input.
+- Numerical data types such as uint8 don't matter, i.e. tests don't check for overflows etc. This suite can't provide such tests as the underlying data type is not known.
+- If not otherwise specified for numbers, a precision of 10 decimals is checked so return values should have at least 11 decimals.


I'm not sure I understand the goal of this. If only up to 10 decimals must be checked, why care about the 11th decimal?

For whatever reaons I sometimes ran into rounding errors or imprecisions, so the 11th number was there to ensure that's actually happening in the 11th digit, not the 10th.

tests/README.md

soxofaan · 2025-08-22T14:15:42Z

tests/schema/schema.json

+        },
+        "experimental": {
+            "type": "boolean",
+            "description": "Declares that the process is experimental, tests may fail.",


why attaching this "experimental" label to the tests, if we already label the processes directly as "experimental".
Or is it to label the test suite as experimental?

To keep them for the tests self-contained. The test-suite reads only the test files if I remember correctly and makes use of the flag (warning vs error, I think).

tests/schema/schema.json

soxofaan · 2025-08-22T14:53:51Z

tests/any.json5

+      "arguments": {
+        "data": []
+      },
+      "returns": {"type": "nodata"}


FYI:

Suggested change

"returns": {"type": "nodata"}

"returns": false

per #494

Yeah, but #550 has not been merged yet. We need to merge it first.

soxofaan · 2025-08-22T14:54:51Z

tests/all.json5

+      "arguments": {
+        "data": []
+      },
+      "returns": {"type": "nodata"}


FYI:

Suggested change

"returns": {"type": "nodata"}

"returns": true

per #494

Waiting for #550.

Co-authored-by: Stefaan Lippens <soxofaan@users.noreply.github.com>

m-mohr · 2025-08-22T15:35:24Z

I merged in from draft, but the tests still need some updates to reflect the latest changes in there. Need to go through the changes and adapt the tests. And also add tests for text_find.

m-mohr force-pushed the add-tests branch 3 times, most recently from f7affe8 to dafc6d9 Compare October 13, 2023 16:39

soxofaan reviewed Oct 16, 2023

View reviewed changes

tests/README.md Outdated Show resolved Hide resolved

m-mohr force-pushed the add-tests branch 3 times, most recently from 7a70a55 to 0155298 Compare October 30, 2023 17:26

m-mohr mentioned this pull request Dec 8, 2023

OpenEO processes validation suite #204

Open

m-mohr linked an issue Dec 8, 2023 that may be closed by this pull request

OpenEO processes validation suite #204

Open

m-mohr force-pushed the add-tests branch 10 times, most recently from 51b2834 to 1982451 Compare December 15, 2023 17:28

m-mohr force-pushed the add-tests branch 2 times, most recently from 0e979e4 to 4069c31 Compare December 22, 2023 20:30

m-mohr force-pushed the add-tests branch from 47366f4 to 9fd49f9 Compare January 3, 2024 14:05

m-mohr requested a review from soxofaan January 3, 2024 14:05

m-mohr marked this pull request as ready for review January 3, 2024 14:05

m-mohr and others added 22 commits August 7, 2025 20:50

Fix apply test

c24c1ad

Add profile levels

67917fa

Fix test in apply

895b40c

Add tests for reduce_dimension and apply_dimension

1967425

Add more tests

3b77850

Fix various test cases

25384af

Special no-data handling, fix tests

75fb59b

Add filter_bands and filter_temporal

2e4c55d

Add mask, merge_cubes, refactor datacube object

1b475fc

Mark experimental processes

0702c08

Add additional test for apply_dimension

fe5f948

Add tests for apply_kernel

4e5ae0d

Add tests for aggregate_temporal

5248a13

Add tests for aggregate_temporal_period, fix aggregate_temporal

6e21fc0

Add tests for filter_bbox, filter_spatial and mask_polygon

17f5a6b

Use nodata type in tests properly

81834d6

Add changelog entry

6508be2

Fix invalid datetimes

20ef8bc

Apply suggestions from code review

788955d

Co-authored-by: Stefaan Lippens <soxofaan@users.noreply.github.com>

Add comment

65c8534

Add consistency checks for test files and fix issues

412b6e8

Remove list of processes from README

b88ff6a

m-mohr force-pushed the add-tests branch from 87e3363 to b88ff6a Compare August 7, 2025 18:50

soxofaan reviewed Aug 22, 2025

View reviewed changes

m-mohr and others added 4 commits August 22, 2025 17:24

Merge branch 'draft' into add-tests

1ec59e8

Update tests/README.md

7d9c1ec

Co-authored-by: Stefaan Lippens <soxofaan@users.noreply.github.com>

Update tests/schema/schema.json

6442258

Co-authored-by: Stefaan Lippens <soxofaan@users.noreply.github.com>

Update tests/README.md

7f4e7b8

Tests for the test suite #468

Are you sure you want to change the base?

Tests for the test suite #468

Uh oh!

Conversation

m-mohr commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

soxofaan commented Jan 5, 2024

Uh oh!

m-mohr commented Aug 7, 2025

Uh oh!

soxofaan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-mohr Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-mohr commented Aug 22, 2025

Uh oh!

Uh oh!

m-mohr commented Oct 12, 2023 •

edited

Loading

m-mohr Aug 22, 2025 •

edited

Loading