Skip to content

Commit 808644b

Browse files
authored
Merge branch 'draft' into issue508-canonical-signed-url-expiry
2 parents aa657a8 + d5ba280 commit 808644b

File tree

18 files changed

+808
-181
lines changed

18 files changed

+808
-181
lines changed

.github/workflows/docs.yml

+5-5
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,11 @@ jobs:
1111
runs-on: ubuntu-latest
1212
steps:
1313
- name: Inject env variables
14-
uses: rlespinasse/github-slug-action@v3.x
15-
- uses: actions/setup-node@v3
14+
uses: rlespinasse/github-slug-action@v5
15+
- uses: actions/setup-node@v4
1616
with:
1717
node-version: 'lts/*'
18-
- uses: actions/checkout@v3
18+
- uses: actions/checkout@v4
1919
- run: |
2020
npm install
2121
npm run build
@@ -33,7 +33,7 @@ jobs:
3333
cp errors.json gh-pages/errors.json
3434
cp -rv assets/. gh-pages/assets/
3535
- name: deploy to root (master)
36-
uses: peaceiris/actions-gh-pages@v3
36+
uses: peaceiris/actions-gh-pages@v4
3737
if: ${{ env.GITHUB_REF_SLUG == 'master' }}
3838
with:
3939
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -42,7 +42,7 @@ jobs:
4242
user_name: 'openEO CI'
4343
user_email: openeo.ci@uni-muenster.de
4444
- name: deploy to ${{ env.GITHUB_REF_SLUG }}
45-
uses: peaceiris/actions-gh-pages@v3
45+
uses: peaceiris/actions-gh-pages@v4
4646
if: ${{ env.GITHUB_REF_SLUG != 'master' }}
4747
with:
4848
github_token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/tests.yml

+13-3
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,21 @@ jobs:
44
deploy:
55
runs-on: ubuntu-latest
66
steps:
7-
- uses: actions/setup-node@v3
7+
- uses: actions/setup-node@v4
88
with:
99
node-version: 'lts/*'
10-
- uses: actions/checkout@v3
11-
- name: Run tests
10+
- uses: actions/checkout@v4
11+
- name: Run core tests
1212
run: |
13+
npm install
14+
npm test
15+
- name: Run commercial-data tests
16+
run: |
17+
cd extensions/commercial-data
18+
npm install
19+
npm test
20+
- name: Run workspaces tests
21+
run: |
22+
cd extensions/workspaces
1323
npm install
1424
npm test

.spectral.yml

+1
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ rules:
88
tag-description: true
99
oas3-parameter-description: true
1010
oas3-unused-component: false # Broken: https://github.com/stoplightio/spectral/issues/1271
11+
array-items: false
1112
operation-summary-formatted:
1213
description: Operation `summary` should start with upper case and not end with a dot.
1314
given: '$.paths.*[?( @property === ''get'' || @property === ''put'' || @property === ''post'' || @property === ''delete'' || @property === ''options'' || @property === ''head'' || @property === ''patch'' || @property === ''trace'' )]'

CHANGELOG.md

+11-2
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
66

77
## Unreleased / Draft
88

9+
### Added
10+
11+
- **New extensions:**
12+
- [Remote Process Definition Extension](./extensions/remote-process-definition/README.md)
13+
- Added `version` property to `GET /processes` [#517](https://github.com/Open-EO/openeo-api/issues/517)
14+
915
### Fixed
1016

1117
- `GET /file_formats`: Base paramater on top of normal JSON Schema, not Process JSON Schema
1218
- `PATCH /services/{service_id}` and `PATCH /jobs/{job_id}`: Explicitly allow updating back-end specific properties (as in `POST`)
1319
- `GET /services/{service_id}` and `GET /jobs/{job_id}`: Explicitly allow listing back-end specific properties (as provided in `POST`)
1420
- `GET /jobs/{job_id}/results`: Clarify that signed URLs (used as "canonical" link) should be regenerated each time.
21+
- Clarified for log levels which default values apply
22+
- Clarified how the relation types `license`, `version-history` and `author` can be used to enrich the process metadata. [#531](https://github.com/Open-EO/openeo-api/issues/531)
23+
- Clarified the behaviour of `federation:backends` for `POST /validation`
1524

1625
## [1.2.0] - 2021-05-25
1726

1827
### Added
1928

2029
- **New extensions:**
21-
- [Commercial Data Extension](./extensions/commercial-data/README.md)
22-
- [Federation Extension](./extensions/federation/README.md)
30+
- [Commercial Data Extension](./extensions/commercial-data/README.md)
31+
- [Federation Extension](./extensions/federation/README.md)
2332
- `GET /`: New Relation types: [#404](https://github.com/Open-EO/openeo-api/issues/404)
2433
- `create-form` to link to the registration page
2534
- `recovery-form` to link to the credentials recovery page.

README.md

+6-5
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,11 @@ See also the [changelog](CHANGELOG.md) and the [milestones](https://github.com/O
2929

3030
## Extensions
3131

32-
| Name | Version | Stability | Description |
33-
| ---------------------------------------------- | ------- | ------------ | ----------- |
34-
| [Commercial Data](extensions/commercial-data/) | 0.1.0 | experimental | Provides an interface for discovering, ordering and using commercial data. |
35-
| [Federation](extensions/federation/) | 0.1.0 | experimental | Covers federation aspects, i.e. where multiple back-ends are exposed as a single API. |
32+
| Name | Version | Stability | Description |
33+
| ------------------------------------------------------------------ | ------- | ------------ | ----------- |
34+
| [Commercial Data](extensions/commercial-data/) | 0.1.0 | experimental | Provides an interface for discovering, ordering and using commercial data. |
35+
| [Federation](extensions/federation/) | 0.1.0 | experimental | Covers federation aspects, i.e. where multiple back-ends are exposed as a single API. |
36+
| [Remote Process Definition](extensions/remote-process-definition/) | 0.1.0 | experimental | Load user-defined processes that are hosted externally through the process namespace into process graphs. |
3637

3738
## Repository
3839

@@ -43,7 +44,7 @@ This repository contains a set of files formally describing the openEO API, each
4344
* The [assets](assets/) folder contains some useful additional files such as examples or schemas. All of these are non-binding additions. The source of truth are the top-level specification files.
4445
* The [extensions](extensions/) folder contains extensions to the openEO API.
4546

46-
# Development
47+
## Development
4748

4849
The `draft` branch is the latest version and is the one to create Pull Requests against.
4950

errors.json

+3-3
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,7 @@
200200
]
201201
},
202202
"ProcessInvalid": {
203-
"description": "The process given is invalid, which ususlly means that the process metadata is invalid.",
203+
"description": "The given process definition is invalid, which usually means that the process metadata is invalid.",
204204
"message": "Invalid process specified.",
205205
"http": 400,
206206
"tags": [
@@ -241,8 +241,8 @@
241241
]
242242
},
243243
"ProcessGraphComplexity": {
244-
"description": "The process graph is too complex for synchronous processing and will likely time out. Please use a batch job instead.",
245-
"message": "The process is too complex for for synchronous processing. Please use a batch job instead.",
244+
"description": "The process graph is computationally too heavy for synchronous processing and will likely time out. Please use a batch job instead.",
245+
"message": "The process graph is too heavy for synchronous processing. Please use a batch job instead.",
246246
"http": 400,
247247
"tags": [
248248
"Data Processing"
+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
extends: "spectral:oas"
2+
rules:
3+
contact-properties: true
4+
tag-description: true
5+
oas3-parameter-description: true
6+
oas3-unused-component: true
7+
operation-id-kebab-case:
8+
given: "$..operationId"
9+
then:
10+
function: pattern
11+
functionOptions:
12+
match: ^[a-z][a-z0-9\-]*$

extensions/commercial-data/README.md

+9-8
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Commercial Data Extension
22

3-
The Commercial Data API extension provides an interface for discovering, ordering and using commercial data in the openEO API.
3+
The Commercial Data Extension to the openEO API provides an interface for discovering, ordering and using commercial data in openEO.
44

55
- Version: **0.1.0**
66
- Stability: **experimental**
@@ -12,7 +12,7 @@ Extensions can not change or break existing behavior of the openEO API.
1212

1313
## Overview of the workflow
1414

15-
All the available datasets provided by a backend are listed on the `GET /collections` endpoint. The collections are normally freely accessible. This extension adds capabilities for providing collections that are not free of charge and require purchasing data products that can thereupon be used in processing. Commercial data collections usually allow purchasing small subsets of the data (products), for example, a single observation of an area.
15+
All the available datasets provided by a back-end are listed on the `GET /collections` endpoint. The collections are normally freely accessible. This extension adds capabilities for providing collections that are not free of charge and require purchasing data products that can thereupon be used in processing. Commercial data collections usually allow purchasing small subsets of the data (products), for example, a single observation of an area.
1616

1717
Therefore, the client must have an ability to search the available products that match their desired criteria and inspect their metadata to decide whether the products should be purchased.
1818

@@ -22,7 +22,7 @@ When the order is completed, the data is ingested in a collection and its ID is
2222

2323
### Collection discovery
2424

25-
A backend should add general information about a commercial data collection to the `/collections` and `/collections/{collection_id}` endpoints, the same as with freely available collections. Only the metadata about the entire dataset needs to be provided, not about the specific data products that a user has already purchased.
25+
A back-end should add general information about a commercial data collection to the `/collections` and `/collections/{collection_id}` endpoints, the same as with freely available collections. Only the metadata about the entire dataset needs to be provided, not about the specific data products that a user has already purchased.
2626

2727
Commercial data collections are distinguished from freely available collections by including `"order:status": "orderable"` as specified in the [STAC Order extension](https://github.com/stac-extensions/order/tree/v1.0.0).
2828

@@ -82,7 +82,7 @@ Commercial data collections must also include human-readable pricing information
8282

8383
### Filtering parameters discovery
8484

85-
Searching for products can support refining the search by filtering with general or collection-specific attributes. Backends should implement a top level `/queryables` endpoint for attributes available for all collections, and collection-specific attributes should be provided at `/collections/{collection_id}/queryables` according to [OGC Queryables specification](https://portal.ogc.org/files/96288#filter-queryables) and [STAC Filter extension](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/fragments/filter).
85+
Searching for products can support refining the search by filtering with general or collection-specific attributes. Back-ends should implement a top level `/queryables` endpoint for attributes available for all collections, and collection-specific attributes should be provided at `/collections/{collection_id}/queryables` according to [OGC Queryables specification](https://portal.ogc.org/files/96288#filter-queryables) and [STAC Filter extension](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/fragments/filter).
8686

8787
#### Example
8888

@@ -125,7 +125,7 @@ Example response from `GET /collections/PLEIADES/queryables`:
125125

126126
### Searching available products
127127

128-
Backends should implement the top-level `GET /search` endpoint as specified in the [STAC Item Search API specification](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/item-search). This should include the [Filter Extension](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/fragments/filter), which enables filtering the available products by attributes specified in `GET /queryables` and `GET /collections/{collection-id}/queryables`.
128+
Back-ends should implement the top-level `GET /search` endpoint as specified in the [STAC Item Search API specification](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/item-search). This should include the [Filter Extension](https://github.com/radiantearth/stac-api-spec/tree/v1.0.0-rc.1/fragments/filter), which enables filtering the available products by attributes specified in `GET /queryables` and `GET /collections/{collection-id}/queryables`.
129129
The endpoint returns a list of [STAC Items](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md) that match the criteria. The products should be aligned with the STAC specification, utilising the existing [STAC extensions](https://github.com/radiantearth/stac-spec/blob/v1.0.0/extensions/README.md) as much as possible, and trying avoiding custom attributes, if a generally accepted definition does not exist.
130130

131131
#### Example
@@ -155,29 +155,30 @@ Example request payload to `GET /search` for `PLEIADES` products from "Living Li
155155

156156
### Ordering products
157157

158-
Backends must implement the following endpoints:
158+
Back-ends must implement the following endpoints:
159159

160160
- `GET /orders`: Get a list of all created orders
161161
- `POST /orders`: Create an order
162162
- `GET /orders/{order_id}`: Get full metadata of a specific order
163163
- `POST /orders/{order_id}`: Confirm a created order
164164

165165
Optionally, they can also implement:
166+
166167
- `DELETE /orders/{order_id}`: Delete an order
167168

168169
See the [OpenAPI document](openapi.yaml) for details.
169170

170171
### Product metadata
171172

172-
The backend can provide full product metadata as STAC Items following [STAC API Features specification](https://github.com/radiantearth/stac-api-spec/tree/main/ogcapi-features). This requires implementing two additional endpoints, `/collections/{collection_id}/items` and `/collections/{collection_id}/items/{item_id}`.
173+
The back-end can provide full product metadata as STAC Items following [STAC API Features specification](https://github.com/radiantearth/stac-api-spec/tree/main/ogcapi-features). This requires implementing two additional endpoints, `/collections/{collection_id}/items` and `/collections/{collection_id}/items/{item_id}`.
173174

174175
`/collections/{collection_id}/items/{item_id}` may return an error if the data has not been ingested yet.
175176

176177
Items should contain links to the respective orders that made them available using relation type `parent_order`.
177178

178179
### Payment
179180

180-
Payment should be done in the currency used by the backend, listed at `GET /` under `billing`. When an order is created, the backend should return the full final cost of the order.
181+
Payment should be done in the currency used by the back-end, listed at `GET /` under `billing`. When an order is created, the back-end should return the full final cost of the order.
181182

182183
### Example usage with Python client
183184

extensions/commercial-data/openapi.yaml

+7-7
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
openapi: 3.0.2
22
info:
3-
title: openEO Commercial Data API extension
3+
title: openEO API - Commercial Data Extension
44
version: 0.1.0
55
description: >-
6-
The Commercial Data API extension provides an interface for discovering,
7-
ordering and using commercial data in the openEO API.
6+
The Commercial Data Extension to the openEO API provides an interface for discovering,
7+
ordering and using commercial data in openEO.
88
contact:
99
name: openEO Consortium
1010
url: 'https://openeo.org'
@@ -13,7 +13,7 @@ info:
1313
name: Apache 2.0
1414
url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
1515
externalDocs:
16-
url: https://github.com/Open-EO/openeo-api/blob/draft/extensions/federation/README.md
16+
url: https://github.com/Open-EO/openeo-api/blob/draft/extensions/commercial-data/README.md
1717
tags:
1818
- name: Orders
1919
description: Management of commercial data orders
@@ -87,7 +87,7 @@ paths:
8787
The order can contain additional parameters that specify how the items should be delivered.
8888
For example, it might be possible to set the projection, resampling method, bit depth etc. of the delivered data.
8989
90-
Backends SHOULD expose the available ordering parameters in `GET /collections/{collection_id}` in the `order_parameters` field, following the `process_parameters` schema of [`GET /service_types`](https://openeo.org/documentation/1.0/developers/api/reference.html#tag/Secondary-Services/operation/list-service-types).
90+
Back-ends SHOULD expose the available ordering parameters in `GET /collections/{collection_id}` in the `order_parameters` field, following the `process_parameters` schema of [`GET /service_types`](https://openeo.org/documentation/1.0/developers/api/reference.html#tag/Secondary-Services/operation/list-service-types).
9191
9292
Returns `Location` and `OpenEO-Identifier` header with the link to the detailed information about the order.
9393
tags:
@@ -148,7 +148,7 @@ paths:
148148
description: |-
149149
Get full metadata of the order.
150150
151-
Backends can optionally link to the spatial and temporal extent information and other metadata about the items using the relation type `item`, preferrably by implementing and linking to `GET /collections/{collection_id}/items/{item_id}`.
151+
Back-ends can optionally link to the spatial and temporal extent information and other metadata about the items using the relation type `item`, preferrably by implementing and linking to `GET /collections/{collection_id}/items/{item_id}`.
152152
tags:
153153
- Orders
154154
security:
@@ -176,7 +176,7 @@ paths:
176176
177177
When an order is created through `POST /order`, the data isn't yet ordered from the commercial data provider. The client MUST explicitly confirm the order whereupon the order is executed, the costs are deducted from the client's account and the data is ingested in the target collection.
178178
179-
The target collection is the collection through which the ordered data is made available. The source and the target collection can be the same or different, it's up to the backend if the ordered data id ingested into a new target collection or the existing source collection. Additionally, if the backend creates new target collections, it can provide access to the union of all purchased products through the source collection.
179+
The target collection is the collection through which the ordered data is made available. The source and the target collection can be the same or different, it's up to the back-end if the ordered data id ingested into a new target collection or the existing source collection. Additionally, if the back-end creates new target collections, it can provide access to the union of all purchased products through the source collection.
180180
181181
This endpoint only has an effect if `order:status` is `orderable`.
182182

extensions/commercial-data/package.json

+3-3
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,12 @@
1313
"url": "git+https://github.com/Open-EO/openeo-api.git"
1414
},
1515
"devDependencies": {
16-
"@stoplight/spectral": "^5.9.1",
17-
"redoc-cli": "^0.13.18"
16+
"@stoplight/spectral-cli": "^6.6.0",
17+
"redoc-cli": "^0.13.21"
1818
},
1919
"scripts": {
2020
"start": "redoc-cli serve openapi.yaml --watch --options.expandResponses \"200,201,202,203,204\" --options.pathInMiddlePanel true",
21-
"build": "redoc-cli bundle openapi.yaml -o redoc.html --title \"openEO API\" --cdn --options.expandResponses \"200,201,202,203,204\" --options.pathInMiddlePanel true",
21+
"build": "redoc-cli bundle openapi.yaml -o redoc.html --title \"openEO API - Commercial Data Extension\" --cdn --options.expandResponses \"200,201,202,203,204\" --options.pathInMiddlePanel true",
2222
"test": "spectral lint openapi.yaml"
2323
}
2424
}

0 commit comments

Comments
 (0)