Merge pull request #262 from Open-EO/draft

Release openEO processes v1.1.0

Author: m-mohr

.github/workflows/docs.yml

@@ -0,0 +1,55 @@
+name: Deploy Documentation
+on:
+  release:
+    types: [published]
+  push:
+    branches:
+      - draft
+      - master
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version: [14.x]
+    steps:
+      - name: Inject env variables
+        uses: rlespinasse/github-slug-action@v3.x
+      - uses: actions/setup-node@v1
+      - uses: actions/checkout@v2
+      - run: |
+          npm install
+          npm run generate
+        working-directory: tests
+      - name: clone gh-pages and clean-up
+        if: ${{ env.GITHUB_REF_SLUG == 'master' }}
+        run: |
+          git clone --branch gh-pages https://github.com/Open-EO/openeo-processes.git gh-pages
+          find gh-pages -maxdepth 1 -type f -delete
+          rm -rf gh-pages/examples/
+          rm -rf gh-pages/meta/
+          rm -rf gh-pages/proposals/
+      - name: create empty gh-pages folder
+        if: ${{ env.GITHUB_REF_SLUG != 'master' }}
+        run: mkdir gh-pages
+      - run: |
+          cp tests/docs.html index.html
+          cp tests/processes.json processes.json
+          rsync -vrm --include='*.json' --include='*.html' --include='examples/***' --include='meta/***' --include='proposals/***' --exclude='*' . gh-pages
+      - name: deploy to root (master)
+        uses: peaceiris/actions-gh-pages@v3
+        if: ${{ env.GITHUB_REF_SLUG == 'master' }}
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: gh-pages
+          user_name: 'openEO CI'
+          user_email: openeo.ci@uni-muenster.de
+      - name: deploy to ${{ env.GITHUB_REF_SLUG }}
+        uses: peaceiris/actions-gh-pages@v3
+        if: ${{ env.GITHUB_REF_SLUG != 'master' }}
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: gh-pages
+          destination_dir: ${{ env.GITHUB_REF_SLUG }}
+          user_name: 'openEO CI'
+          user_email: openeo.ci@uni-muenster.de

.github/workflows/tests.yml

@@ -0,0 +1,16 @@
+name: Test Processes
+on: [push, pull_request]
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version: [14.x]
+    steps:
+      - uses: actions/setup-node@v1
+      - uses: actions/checkout@v2
+      - name: Run tests
+        run: |
+          npm install
+          npm run test
+        working-directory: tests

.travis.yml

@@ -6,8 +6,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased / Draft
 
+## [1.1.0] - 2021-06-29
 
-## 1.0.0 - 2020-07-31
+### Added
+- New processes in proposal state
+    - `array_append`
+    - `array_concat`
+    - `array_create`
+    - `array_create_labeled`
+    - `array_find_label`
+    - `array_interpolate_linear` [#173](https://github.com/Open-EO/openeo-processes/issues/173)
+    - `array_modify`
+    - `date_shift`
+    - `is_infinite`
+    - `nan`
+    - `reduce_spatial`
+- Added return value details (property `returns`) for the schemas with the subtype `process-graph`. [API#350](https://github.com/Open-EO/openeo-api/issues/350)
+- `apply_neighborhood`: Clarify behavior for data cubes returned by the child processes and for that add the exception `DataCubePropertiesImmutable`.
+- Added a guide for implementors that describes numerours implementation details for processes that could not be covered in the specifications itself, for example a recommended implementation for the `if` process. [#246](https://github.com/Open-EO/openeo-processes/issues/246)
+
+### Changed
+- Added `proposals` folder for experimental processes. Experimental processes are not covered by the CHANGELOG and MAY include breaking changes! [#196](https://github.com/Open-EO/openeo-processes/issues/196), [#207](https://github.com/Open-EO/openeo-processes/issues/207), [PSC#8](https://github.com/Open-EO/PSC/issues/8)
+    - Moved the experimental process `run_udf_externally` to the proposals.
+    - Moved the rarely used and implemented processes `cummax`, `cummin`, `cumproduct`, `cumsum`, `debug`, `filter_labels`, `load_result`, `load_uploaded_files`, `resample_cube_temporal` to the proposals.
+- Exception messages have been aligned always use ` instead of '. Tooling could render it with CommonMark.
+- `load_collection`  and `mask_polygon`: Also support multi polygons instead of just polygons. [#237](https://github.com/Open-EO/openeo-processes/issues/237)
+- `run_udf` and `run_udf_externally`: Specify specific (extensible) protocols for UDF URIs.
+- `resample_cube_spatial` and `resample_spatial`: Aligned with GDAL and added `rms` and `sum` options to methods. Also added better descriptions.
+- `resample_cube_temporal`: Process has been simplified and only offers the nearest neighbor method now. The `process` parameter has been removed, the `dimension` parameter was made less restrictive, the parameter `valid_within` was added. [#194](https://github.com/Open-EO/openeo-processes/issues/194)
+
+### Deprecated
+- `GeometryCollection`s are discouraged in all relevant processes.
+
+### Removed
+
+- Removed the experimental processes `aggregate_spatial_binary` and `reduce_dimension_binary`. [#258](https://github.com/Open-EO/openeo-processes/issues/258)
+
+### Fixed
+- Clarify that the user workspace is server-side. [#225](https://github.com/Open-EO/openeo-processes/issues/225)
+- Clarify that the `condition` parameter for `array_filter` works also on indices and labels.
+- Clarify contradicting statements in `filter_temporal` for the default value of the `dimension` parameter. By default *all* temporal dimensions are affected by the process. [#203](https://github.com/Open-EO/openeo-processes/issues/203)
+- Clarify how the parameters passed to the overlap resolver correspond to the data cubes. [#184](https://github.com/Open-EO/openeo-processes/issues/184)
+- Improve and clarify specifications for `is_nan`, `is_nodata`, `is_valid`. [#189](https://github.com/Open-EO/openeo-processes/issues/189)
+- Improve and clarify specifications for `all` and `any`. [#189](https://github.com/Open-EO/openeo-processes/issues/199)
+- `array_element`: Clarify that `ArrayNotLabeled` exception is thrown when parameter `label` is specified and the given array is not labeled.
+- `array_apply`, `array_element`, `array_filter`: Added the `minimum: 0` constraint to all schemas describing zero-based indices (parameter `index`).
+- `array_labels`: Clarified the accepted data type for array elements passed to the parameter `data`.
+- `merge_cubes`: Clarified the dimension label order after the merge. [#212](https://github.com/Open-EO/openeo-processes/issues/212)
+- `merge_cubes`: Clarified the fourth example. [#266](https://github.com/Open-EO/openeo-processes/issues/266)
+- Fixed typos, grammar issues and other spelling-related issues in many of the processes.
+- Fixed the examples `array_contains_nodata` and `array_find_nodata`.
+- Fixed links to openEO glossary and added links to data cube introduction. [#216](https://github.com/Open-EO/openeo-processes/issues/216)
+- Fixed description of `apply_dimension` with regards to reference systems. Made description easier to understand, too. [#234](https://github.com/Open-EO/openeo-processes/issues/234)
+- Clarified disallowed characters in subtype `file-path`.
+- Clarified that UDF source code must contain a newline/line-break (affects `run_udf`).
+- `aggregate_spatial`, `aggregate_spatial_binary`: Clarified that Features, Geometries and GeometryCollections are a single entity in computations. Only FeatureCollections are multiple entities. [#252](https://github.com/Open-EO/openeo-processes/issues/252)
+- `aggregate_spatial`: Clarified that the values have no predefined order and reducers such as `first` and `last` return unpredictable results. [#260](https://github.com/Open-EO/openeo-processes/issues/260)
+- `load_collection`, parameter `spatial_extent`: Clarified that all pixels that are inside the bounding box of the given polygons but do not intersect with any polygon have to be set to no-data (`null`). [#256](https://github.com/Open-EO/openeo-processes/issues/256)
+- `load_collection`: Clarified that the parameters are recommended to be used in favor of `filter_*` processes.
+- `aggregate_temporal` and `aggregate_temporal_period`: Clarified that reducers are also executed for intervals/periods with no data. [#263](https://github.com/Open-EO/openeo-processes/issues/263)
+- `dimension_labels`: Clarified that the process fails with a `DimensionNotAvailable` exception, if a dimension with the specified name does not exist.
+
+## [1.0.0] - 2020-07-31
 
 ### Added
 - `subtype-schemas.json`: A list of predefined subtypes is available as JSON Schema; Moved over from openeo-api.
@@ -41,13 +101,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `aggregate_temporal`: Fixed outdated message for exception `TooManyDimensions`.
 - `clip`: Fixed examples.
 - `linear_scale_range`: Clarify that the process implicitly clips the values. [#159](https://github.com/Open-EO/openeo-processes/issues/159)
-- `mean`: Clarify behaviour for arrays with `null`-values only.
-- `mod`: Clarified behaviour. [#168](https://github.com/Open-EO/openeo-processes/issues/168)
-- `resample_*`: Clarified behaviour.
+- `mean`: Clarify behavior for arrays with `null`-values only.
+- `mod`: Clarified behavior. [#168](https://github.com/Open-EO/openeo-processes/issues/168)
+- `resample_*`: Clarified behavior.
 - `first`, `last`, `max`, `mean`, `median`, `min`, `sd`, `variance`: Clarify behavior for arrays with `null`-values only.
 - Clarified (and fixed if necessary) for all processes in the "cubes" category the descriptions for the returned data cube. [#149](https://github.com/Open-EO/openeo-processes/issues/149)
 
-## 1.0.0-rc.1 - 2020-01-31
+## [1.0.0-rc.1] - 2020-01-31
 
 ### Added
 - Processes:
@@ -159,3 +219,13 @@ First version which is separated from the openEO API. Complete rework of all pro
 
 ## Legacy versions
 Older versions of the processes were released as part of the openEO API, see the corresponding changelog for more information.
+
+
+[Unreleased]: <https://github.com/Open-EO/openeo-processes/compare/1.1.0...HEAD>
+[1.1.0]: <https://github.com/Open-EO/openeo-processes/compare/1.0.0...1.1.0>
+[1.0.0]: <https://github.com/Open-EO/openeo-processes/compare/1.0.0-rc.1...1.0.0>
+[1.0.0-rc.1]: <https://github.com/Open-EO/openeo-processes/compare/0.4.2...1.0.0-rc.1>
+[0.4.2]: <https://github.com/Open-EO/openeo-processes/compare/0.4.1...0.4.2>
+[0.4.1]: <https://github.com/Open-EO/openeo-processes/compare/0.4.0...0.4.1>
+[0.4.0]: <https://github.com/Open-EO/openeo-processes/tree/0.4.0>
+

CHANGELOG.md

@@ -6,12 +6,15 @@ openEO develops interoperable processes for big Earth observation cloud processi
 
 ## Versions / Branches
 
-The [master branch](https://github.com/Open-EO/openeo-processes/tree/master) is the 'stable' version of the openEO processes specification. The latest release is version **1.0.0**. The [draft branch](https://github.com/Open-EO/openeo-processes/tree/draft) is where active development takes place.
+The [master branch](https://github.com/Open-EO/openeo-processes/tree/master) is the 'stable' version of the openEO processes specification. An exception is the [`proposals`](proposals/) folder, which provides experimental new processes currently under discussion. They may still change, but everyone is encouraged to implement them and give feedback.
+
+The latest release is version **1.1.0**. The [draft branch](https://github.com/Open-EO/openeo-processes/tree/draft) is where active development takes place. PRs should be made against the draft branch.
 
 | Version / Branch                                             | Status                    | openEO API versions |
 | ------------------------------------------------------------ | ------------------------- | ------------------- |
 | [unreleased / draft](https://processes.openeo.org/draft)     | in development            | 1.x.x               |
-| [**1.0.0** / master](https://processes.openeo.org/1.0.0/)    | **latest stable version** | 1.x.x               |
+| [**1.1.0** / master](https://processes.openeo.org/1.1.0/)    | **latest stable version** | 1.x.x               |
+| [1.0.0](https://processes.openeo.org/1.0.0/)                 | legacy version            | 1.x.x               |
 | [1.0.0 RC1](https://processes.openeo.org/1.0.0-rc.1/)        | legacy version            | 1.x.x               |
 | [0.4.2](https://processes.openeo.org/0.4.2/)                 | legacy version            | 0.4.x               |
 | [0.4.1](https://processes.openeo.org/0.4.1/)                 | legacy version            | 0.4.x               |
@@ -25,7 +28,18 @@ See also the [changelog](CHANGELOG.md) for the changes between versions and the
 
 This repository contains a set of files formally describing the openEO Processes:
 
-* The `*.json` files provide the process specifications as defined by the openEO API.
+* The `*.json` files provide stable process specifications as defined by openEO. Stable processes need at least two implementations and a use-case example added to the [`examples`](examples/) folder *or* consensus from the openEO PSC.
+* The `*.json` files in the [`proposals`](proposals/) folder provide proposed new process specifications that are still experimental and subject to change, including breaking changes. Everyone is encouraged to base their work on the proposals and give feedback so that eventually the processes evolve into stable process specifications.
+* [implementation.md](meta/implementation.md) in the `meta` folder provide some additional implementation details for back-ends. For back-end implementors, it's highly recommended to read them.
 * [subtype-schemas.json](meta/subtype-schemas.json) in the `meta` folder defines common data types (`subtype`s) for JSON Schema used in openEO processes.
 * The [`examples`](examples/) folder contains some useful examples that the processes link to. All of these are non-binding additions.
-* The [`tests`](tests/) folder can be used to test the process specification for validity and and consistent "style". It also allows to render the processes in a web browser.
+* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser.
+
+
+## Process
+
+* All new processes must be added to the [`proposals`](proposals/) folder.
+* Processes will only be moved from proposals to the stable process specifications once there are at least two implementations and an example process in the [`examples`](examples/) folder showing it in a use case. This doesn't require a PSC vote individually as it's not a breaking change, just an addition.
+* The [`proposals`](proposals/) folder allows breaking changes without a PSC vote and without increasing the major version number (i.e. a breaking change in the proposals doesn't require us to make the next version number 2.0.0).
+* The proposals are released as experimental processes with the other processes.
+* Each release and all breaking changes in the stable process specifications must go through PSC vote.

README.md

@@ -1,7 +1,7 @@
 {
     "id": "add",
     "summary": "Addition of two numbers",
-    "description": "Sums up the two numbers `x` and `y` (*x + y*) and returns the computed sum.\n\nNo-data values are taken into account so that `null` is returned if any element is such a value.\n\nThe computations follow [IEEE Standard 754](https://ieeexplore.ieee.org/document/8766229) whenever the processing environment supports it.",
+    "description": "Sums up the two numbers `x` and `y` (*`x + y`*) and returns the computed sum.\n\nNo-data values are taken into account so that `null` is returned if any element is such a value.\n\nThe computations follow [IEEE Standard 754](https://ieeexplore.ieee.org/document/8766229) whenever the processing environment supports it.",
     "categories": [
         "math"
     ],

add.json

@@ -1,7 +1,7 @@
 {
     "id": "add_dimension",
     "summary": "Add a new dimension",
-    "description": "Adds a new named dimension to the data cube.\n\nAfterwards, the dimension can be referenced with the specified `name`. If a dimension with the specified name exists, the process fails with a `DimensionExists` error. The dimension label of the dimension is set to the specified `label`.",
+    "description": "Adds a new named dimension to the data cube.\n\nAfterwards, the dimension can be referred to with the specified `name`. If a dimension with the specified name exists, the process fails with a `DimensionExists` exception. The dimension label of the dimension is set to the specified `label`.",
     "categories": [
         "cubes"
     ],