Skip to content

Commit 3744511

Browse files
m-mohrclausmichele
m-mohr
and
clausmichele
authored
STAC, Collection and Workspace handling in openEO (#485)
* Add `export_collection`, `export_workspace`, `stac_update`; `save_results` returns the STAC resource instead of boolean `true` Open-EO/openeo-api#376 * Update stac_update/modify * Added details about STAC support. * Update meta/implementation.md Co-authored-by: Matthias Mohr <m.mohr@uni-muenster.de> --------- Co-authored-by: Michele Claus <31700619+clausmichele@users.noreply.github.com>
1 parent 9532cb1 commit 3744511

File tree

7 files changed

+191
-4
lines changed

7 files changed

+191
-4
lines changed

‎CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
66

77
## Unreleased / Draft
88

9+
### Added
10+
11+
- `export_collection`
12+
- `export_workspace`
13+
- `stac_modify`
14+
915
### Changed
1016

1117
- `clip`: Throw an exception if min > max [#472](https://github.com/Open-EO/openeo-processes/issues/472)
18+
- `save_results`: Returns the STAC resource instead of boolean `true` [API#376](https://github.com/Open-EO/openeo-api/issues/376)
1219
- Added a uniqueness contraint to various array-typed parameters (e.g. lists of dimension names or labels)
1320
- Renamed `create_data_cube` to `create_cube`. [#68](https://github.com/Open-EO/openeo-processes/issues/68)
1421
- `apply_polygon`: Renamed `polygons` parameter to `geometries` for better alignment with other geometry handling processes. [#511](https://github.com/Open-EO/openeo-processes/issues/511)

‎meta/implementation.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,3 +230,24 @@ We have found some libraries that can be used for an implementation:
230230
- Julia: [Statistics.quantile](https://docs.julialang.org/en/v1/stdlib/Statistics/#Statistics.quantile!), type 7 is the default.
231231
- Python: [numpy](https://numpy.org/doc/stable/reference/generated/numpy.quantile.html), [pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.quantile.html), [xarray](http://xarray.pydata.org/en/stable/generated/xarray.DataArray.quantile.html) - type 7 (called 'linear' for the interpolation parameter) is the default for all of them.
232232
- R: [quantile](https://stat.ethz.ch/R-manual/R-patched/library/stats/html/quantile.html) - type 7 is the default.
233+
234+
## STAC support
235+
236+
The subtype `stac` is an abstract type that refers to a STAC resource of any type (Catalog, Collection, or Item).
237+
It can refer to:
238+
- static STAC resources, e.g. hosted on cloud storage
239+
- dynamic STAC resources made available via a STAC API
240+
- a STAC JSON representation embedded as an argument into an openEO user-defined process
241+
242+
### stac_modify
243+
244+
The process `stac_modify` updates a given STAC resource based on
245+
[RFC 7386: JSON Merge Patch](https://www.rfc-editor.org/rfc/rfc7386.html).
246+
For static STAC resources, the content of the JSON files shall be updated according to RFC 7386.
247+
248+
If the underlying STAC resource is part of an API, the following HTTP endpoints shall be used for the updates:
249+
- For STAC Items: `PATCH /collections/{collectionId}/items/{featureId}`
250+
according to the [Transaction Extension](https://github.com/stac-api-extensions/transaction)
251+
- For STAC Collections: `PATCH /collections/{collectionId}`
252+
according to the [Collection Transaction Extension](https://github.com/stac-api-extensions/collection-transaction)
253+
- For STAC Catalogs there is no API support for updates.

‎meta/subtype-schemas.json

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,6 +278,23 @@
278278
"description": "A raster data cube, which is a data cube with two dimension of type spatial (x and y). This has been deprecated in favour of `datacube`.",
279279
"deprecated": true
280280
},
281+
"stac": {
282+
"type": "object",
283+
"subtype": "stac",
284+
"title": "STAC resource",
285+
"description": "A STAC Catalog, Collection, or Item, as defined by the [STAC specification](https://stacspec.org) version 0.9.0 or later.",
286+
"oneOf": [
287+
{
288+
"$ref": "http://schemas.stacspec.org/v1.0.0/catalog-spec/json-schema/catalog.json"
289+
},
290+
{
291+
"$ref": "http://schemas.stacspec.org/v1.0.0/collection-spec/json-schema/collection.json"
292+
},
293+
{
294+
"$ref": "http://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json"
295+
}
296+
]
297+
},
281298
"temporal-interval": {
282299
"type": "array",
283300
"subtype": "temporal-interval",
@@ -413,6 +430,13 @@
413430
"title": "WKT2 definition",
414431
"description": "Specifies details about cartographic projections as WKT2 string. Refers to the latest WKT2 version (currently [WKT2:2018](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html) / ISO 19162:2018) unless otherwise stated by the process."
415432
},
433+
"workspace-id": {
434+
"type": "string",
435+
"subtype": "workspace-id",
436+
"title": "Workspace ID",
437+
"description": "A workspace identifier from the list of available workspaces.",
438+
"pattern": "^[\\w\\-\\.~]+$"
439+
},
416440
"year": {
417441
"type": "integer",
418442
"subtype": "year",

‎proposals/export_collection.json

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
{
2+
"id": "export_collection",
3+
"summary": "Export data to an openEO collection",
4+
"description": "Exports the given processing results made available through a STAC resource (e.g., a STAC Collection or Item) to the given openEO collection. The STAC resource itself is exported with all STAC resources and assets underneath.",
5+
"categories": [
6+
"export",
7+
"stac"
8+
],
9+
"experimental": true,
10+
"parameters": [
11+
{
12+
"name": "data",
13+
"description": "The data to export to the openEO collection.",
14+
"schema": {
15+
"type": "object",
16+
"subtype": "stac"
17+
}
18+
},
19+
{
20+
"name": "collection",
21+
"description": "The identifier of the collection to export to. If the collection **exists**, the data will be added to the collection. If the collection **does not exist** yet, it will be created based on the given STAC metadata.",
22+
"schema": [
23+
{
24+
"title": "New Collection",
25+
"type": "string",
26+
"pattern": "^[A-Za-z0-9_\\-\\.~/]+$"
27+
},
28+
{
29+
"title": "Existing Collection",
30+
"type": "string",
31+
"subtype": "collection-id",
32+
"pattern": "^[A-Za-z0-9_\\-\\.~/]+$"
33+
}
34+
]
35+
}
36+
],
37+
"returns": {
38+
"description": "Returns the collection identifier.",
39+
"schema": {
40+
"type": "string",
41+
"subtype": "collection-id",
42+
"pattern": "^[A-Za-z0-9_\\-\\.~/]+$"
43+
}
44+
}
45+
}

‎proposals/export_workspace.json

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
{
2+
"id": "export_workspace",
3+
"summary": "Export data to a cloud user workspace",
4+
"description": "Exports the given processing results made available through a STAC resource (e.g., a STAC Collection) to the given user workspace. The STAC resource itself is exported with all STAC resources and assets underneath.",
5+
"categories": [
6+
"export",
7+
"stac"
8+
],
9+
"experimental": true,
10+
"parameters": [
11+
{
12+
"name": "data",
13+
"description": "The data to export to the user workspace as a STAC resource.",
14+
"schema": {
15+
"type": "object",
16+
"subtype": "stac"
17+
}
18+
},
19+
{
20+
"name": "workspace",
21+
"description": "The identifier of the workspace to export to.",
22+
"schema": {
23+
"type": "string",
24+
"pattern": "^[\\w\\-\\.~]+$",
25+
"subtype": "workspace-id"
26+
}
27+
},
28+
{
29+
"name": "merge",
30+
"description": "Provides a cloud-specific path identifier to a STAC resource to merge the given STAC resource into. If not provided, the STAC resource is kept separate from any other STAC resources in the workspace.",
31+
"schema": {
32+
"type": [
33+
"string",
34+
"null"
35+
]
36+
},
37+
"optional": true,
38+
"default": null
39+
}
40+
],
41+
"returns": {
42+
"description": "Returns the potentially updated STAC resource.",
43+
"schema": {
44+
"type": "object",
45+
"subtype": "stac"
46+
}
47+
}
48+
}

‎proposals/stac_modify.json

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
{
2+
"id": "stac_modify",
3+
"summary": "Updates an existing STAC resource",
4+
"description": "Modifies the given STAC resource (e.g., a STAC Collection or Item) based on the given changeset.",
5+
"categories": [
6+
"stac"
7+
],
8+
"experimental": true,
9+
"parameters": [
10+
{
11+
"name": "data",
12+
"description": "The existing STAC resource.",
13+
"schema": {
14+
"type": "object",
15+
"subtype": "stac"
16+
}
17+
},
18+
{
19+
"name": "changes",
20+
"description": "A potentially incomplete STAC resource that should be merged with the existing STAC resource. It follows the [RFC 7386: JSON Merge Patch](https://www.rfc-editor.org/rfc/rfc7386.html) specification.",
21+
"schema": {
22+
"type": "object"
23+
}
24+
}
25+
],
26+
"returns": {
27+
"description": "Returns the modified STAC resource.",
28+
"schema": {
29+
"type": "object",
30+
"subtype": "stac"
31+
}
32+
},
33+
"links": [
34+
{
35+
"href": "https://www.rfc-editor.org/rfc/rfc7386.html",
36+
"title": "RFC 7386: JSON Merge Patch",
37+
"type": "text/html",
38+
"rel": "about"
39+
}
40+
]
41+
}

‎save_result.json

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44
"description": "Makes the processed data available in the given file format to the corresponding medium that is relevant for the context this processes is applied in:\n\n* For **batch jobs** the data is stored on the back-end. STAC-compatible metadata is usually made available with the processed data.\n* For **synchronous processing** the data is sent to the client as a direct response to the request.\n* **Secondary web services** are provided with the processed data so that it can make use of it (e.g., visualize it). Web service may require the data in a certain format. Please refer to the documentation of the individual service types for details.",
55
"categories": [
66
"cubes",
7-
"export"
7+
"export",
8+
"stac"
89
],
910
"parameters": [
1011
{
@@ -35,10 +36,10 @@
3536
}
3637
],
3738
"returns": {
38-
"description": "Always returns `true` as in case of an error an exception is thrown which aborts the execution of the process.",
39+
"description": "Returns the STAC resource that was created in the process of saving the result.",
3940
"schema": {
40-
"type": "boolean",
41-
"const": true
41+
"type": "object",
42+
"subtype": "stac"
4243
}
4344
},
4445
"exceptions": {

0 commit comments

Comments
 (0)