Updated examples and more preparations for the 0.4.0 release.

Author: m-mohr

docs/arch.md

@@ -11,7 +11,7 @@ The openEO API defines a language how clients communicate to back-ends in order
 7. The openEO API may define *profiles* in order to group specific functionality.
 
 ![Architecture - openEO API shown in dark blue](img/arch.png)
-*Figure: Architecture - openEO API shown in dark blue*
+*Figure: Architecture*
 
 # Microservices
 

docs/collections.md

@@ -1,6 +1,6 @@
 # Data Discovery (Collections)
 
-openEO strives for compatibility with [STAC](https://github.com/radiantearth/stac-spec) and [OGC WFS](https://github.com/opengeospatial/WFS_FES) as far as possible. Implementing the data discovery endpoints of openEO should also produce valid STAC 0.6 and WFS 3.0 responses, including an incomplete compatibility with their APIs (see below).
+openEO strives for compatibility with [STAC](https://github.com/radiantearth/stac-spec) and [OGC WFS](https://github.com/opengeospatial/WFS_FES) as far as possible. Implementing the data discovery endpoints of openEO should also produce valid STAC 0.6 and WFS 3.0 responses, including a partial compatibility with their APIs (see below).
 
 **WARNING**: STAC and OGC WFS 3, as well as openEO, are still under development.
 Therefore, it is very likely that further changes and adjustments will be made in the future.
@@ -15,7 +15,8 @@ Some commonly used extensions that are integrated in the openEO API specificatio
 - SAR extension
 - Commons extension
 - Scientific extension
-- Data Cube extension (draft)
+- Data Cube extension
+- Non-Common Properties extension
 
 ## Links
 

docs/errors.md

@@ -48,7 +48,7 @@ The openEO API usually uses the following HTTP status codes for successful reque
 
 The openEO API often uses the following HTTP status codes for failed requests: 
 
-- **400 Bad request**:
+- **400 Bad Request**:
   The back-end responds with this error code whenever the error has its origin on client side and no other HTTP status code in the 400 range is suitable.
 
 - **401 Unauthorized**:
@@ -64,7 +64,7 @@ The openEO API often uses the following HTTP status codes for failed requests:
 - **500 Internal Server Error**:
   The error has its origin on server side and no other status code in the 500 range is suitable.
 
-- **501 Not implemented**:
+- **501 Not Implemented**:
   An endpoint is specified in the openEO API, but is not supported.
 
 

docs/examples-poc.md

@@ -82,21 +82,20 @@ Deriving minimum NDVI measurements over pixel time series of Sentinel 2 imagery.
         "loadco1": {
           "process_id": "load_collection",
           "arguments": {
-            "id": "Sentinel-2"
-          }
-        },
-        "filter1": {
-          "process_id": "filter_temporal",
-          "arguments": {
-            "data": {"from_node": "loadco1"},
-            "start": "2017-01-01",
-            "end": "2017-02-01"
+            "id": "Sentinel-2",
+            "spatial_extent": {
+              "west": {"variable_id": "spatial_extent_west"},
+              "east": {"variable_id": "spatial_extent_east"},
+              "north": {"variable_id": "spatial_extent_north"},
+              "south": {"variable_id": "spatial_extent_south"}
+            },
+            "temporal_extent": ["2017-01-01", "2017-02-01"]
           }
         },
         "ndvi1": {
           "process_id": "ndvi",
           "arguments": {
-            "data": {"from_node": "filter1"}
+            "data": {"from_node": "loadco1"}
           }
         },
         "reduce1": {
@@ -176,33 +175,20 @@ Create a monthly aggregated Sentinel 1 product from a custom Python script.
         "loadco1": {
           "process_id": "load_collection",
           "arguments": {
-            "id": "Sentinel-1"
-          }
-        },
-        "filter1": {
-          "process_id": "filter_bbox",
-          "arguments": {
-            "data": {"from_node": "loadco1"},
-            "extent": {
+            "id": "Sentinel-1",
+            "spatial_extent": {
               "west": 16.1,
               "east": 16.6,
               "north": 48.6,
               "south": 47.2
-            }
-          }
-        },
-        "filter2": {
-          "process_id": "filter_temporal",
-          "arguments": {
-            "data": {"from_node": "filter1"},
-            "start": "2017-01-01",
-            "end": "2017-02-01"
+            },
+            "temporal_extent": ["2017-01-01", "2017-02-01"]
           }
         },
         "reduce1": {
           "process_id": "reduce",
           "arguments": {
-            "data": {"from_node": "filter2"},
+            "data": {"from_node": "loadco1"},
             "dimension": "temporal",
             "reducer": {
               "callback": {
@@ -307,40 +293,21 @@ Compute time series of zonal (regional) statistics of Sentinel 2 imagery over us
         "loadco1": {
           "process_id": "load_collection",
           "arguments": {
-            "id": "Sentinel-2"
-          }
-        },
-        "filter1": {
-          "process_id": "filter_bbox",
-          "arguments": {
-            "data": {"from_node": "loadco1"},
-            "extent": {
+            "id": "Sentinel-2",
+            "spatial_extent": {
               "west": 16.1,
               "east": 16.6,
               "north": 48.6,
               "south": 47.2
-            }
-          }
-        },
-        "filter2": {
-          "process_id": "filter_temporal",
-          "arguments": {
-            "data": {"from_node": "filter1"},
-            "start": "2017-01-01",
-            "end": "2017-02-01"
-          }
-        },
-        "filter3": {
-          "process_id": "filter_bands",
-          "arguments": {
-            "data": {"from_node": "filter2"},
+            },
+            "temporal_extent": ["2017-01-01", "2017-02-01"],
             "bands": ["B8"]
           }
         },
         "reduce1": {
           "process_id": "reduce",
           "arguments": {
-            "data": {"from_node": "filter3"},
+            "data": {"from_node": "loadco1"},
             "dimension": "spectral"
           }
         },

docs/gettingstarted-backends.md

@@ -8,17 +8,17 @@ If your preferred technology has no back-end driver yet, you may consider writin
 
 You certainly need to understand the [architecture](arch.md) of openEO and concepts behind [jobs](jobs.md), [processes](processes.md) and [process graphs](processgraphs.md). This helps you read and understand the [API specification](apireference.md). Technical API related documents like [CORS](cors.md) and [error handing](errors.md) should be read, too.
 
-If you do not want to start from scratch, you could try to generate a server stub from the [OpenAPI 3.0](https://www.openapis.org/)-based [API specification](apireference.md) with the [Swagger code generator](https://github.com/swagger-api/swagger-codegen). If you are using Python or NodeJS to implement your driver you may re-use some common modules of existing driver implementations:
+If you do not want to start from scratch, you could try to generate a server stub from the [OpenAPI 3.0](https://www.openapis.org/)-based [API specification](apireference.md) with the [OpenAPI Generator](https://github.com/OpenAPITools/openapi-generator). If you are using Python implement your driver you may re-use some common modules of existing driver implementations:
 
-* [Python Driver](https://github.com/Open-EO/openeo-python-driver)
-* [NodeJS Driver](https://github.com/Open-EO/openeo-nodejs-driver)
+* [Python Driver Commons](https://github.com/Open-EO/openeo-python-driver)
 
 You can implement a back-end in iterations. It is recommended to start by implementing the [Capabilities](apireference.md#tag/Capabilities) microservice. [EO Data Discovery](apireference.md#tag/EO-Data-Discovery), [Process Discovery](apireference.md#tag/Process-Discovery) are important for the client libraries to be available, too. Afterwards you should implement [Batch Job Management](apireference.md#tag/Batch-Job-Management) or [synchronous data processing](apireference.md#/paths/~1result/post). All other microservices can be added later and are not strictly required to run openEO services. Keep in mind that you don't need to implement all endpoints in the first iteration and that you can specify in the Capabilities, which endpoints you are supporting.
 
 For example, you could start by implementing the following endpoints in the first iteration:
 
+* Well-Known Document: `GET /.well-known/openeo`
 * Capabilities: `GET /` and `GET /output_formats`
-* Data discovery: `GET /collections` and `GET /collections/{name}`
+* Data discovery: `GET /collections` and `GET /collections/{collection_id}`
 * Process discovery: `GET /processes`
 * Data processing: `POST /result`
 * Authentication (if required): `GET /credentials/basic`

docs/gettingstarted-clients.md

@@ -8,7 +8,7 @@ If your preferred programming language is not part of the [available client libr
 
 Get started by reading the [guidelines to develop client libraries](guidelines-clients.md), which have been written to ensure the client libraries provide a consistent feel and behavior across programming languages. You certainly need to understand the concepts behind [jobs](jobs.md), [processes](processes.md) and [process graphs](processgraphs.md). This helps you understand the [API specification](apireference.md) and related documents.
 
-If you do not want to start from scratch, you could try to generate a client library stub from the [OpenAPI 3.0](https://www.openapis.org/)-based [API specification](apireference.md) with the [Swagger code generator](https://github.com/swagger-api/swagger-codegen). Make sure the generated code complies to the client library guidelines mentioned above.
+If you do not want to start from scratch, you could try to generate a client library stub from the [OpenAPI 3.0](https://www.openapis.org/)-based [API specification](apireference.md) with the [OpenAPI Generator](https://github.com/OpenAPITools/openapi-generator). Make sure the generated code complies to the client library guidelines mentioned above.
 
 *More information will follow soon, for example about client testing.*
 

docs/glossary.md

@@ -19,7 +19,7 @@ A **process graph** chains specific process calls. Similarly to scripts in the c
 
 ## EO data (Collections)
 
-In our domain, different terms are used to describe EO data(sets). Within openEO, a **granule** (sometimes also called *item* or *asset* in the specification) typically refers to a limited area and a single overpass leading to a very short observation period (seconds) or a temporal aggregation of such data (e.g. for 16-day MODIS composites) while a **collection** is an aggregation of granules sharing the same product specification. It typically corresponds to the series of products derived from data acquired by a sensor on board a satellite and having the same mode of operation.
+In our domain, different terms are used to describe EO data(sets). Within openEO, a **granule** (sometimes also called *item* or *asset* in the specification) typically refers to a limited area and a single overpass leading to a very short observation period (seconds) or a temporal aggregation of such data (e.g. for 16-day MODIS composites) while a **collection** is an aggregation of granules sharing the same product specification. It typically corresponds to the series of products derived from data acquired by a sensor on board a satellite and having the same mode of operation.
 
 The [CEOS OpenSearch Best Practice Document v1.2](http://ceos.org/ourwork/workinggroups/wgiss/access/opensearch/) lists synonyms used (by organizations) for:
 
@@ -36,12 +36,12 @@ The figure below shows the data of
 a four-dimensional (8 x 8 x 2 x 2) raster data cube, with dimension names
 and values:
 
-| #| dimension name | dimension values                     |
-|--|----------------|--------------------------------------|
-| 1| x              | 288790.5, 288819, 288847.5, 288876, 288904.5, 288933, 288961.5, 288990 |
-| 2| y              | 9120747, 9120718, 9120690, 9120661, 9120633, 9120604, 9120576, 9120547 |
-| 3| band           | `red`, `green` |
-| 4| time           | `2018-02-10`, `2018-02-17` |
+| #    | dimension name | dimension values |
+| ---- | -------------- | ---------------- |
+| 1    | x              | 288790.5, 288819, 288847.5, 288876, 288904.5, 288933, 288961.5, 288990 |
+| 2    | y              | 9120747, 9120718, 9120690, 9120661, 9120633, 9120604, 9120576, 9120547 |
+| 3    | band           | `red`, `green` |
+| 4    | time           | `2018-02-10`, `2018-02-17` |
 
 dimensions x and time are aligned along the x-axis; y and band are aligned along the y-axis.
 
@@ -128,4 +128,12 @@ When the target grid or time series has a lower resolution (larger grid cells) o
 
 ### User-defined function (UDF)
 
-The abbreviation **UDF** stands for **user-defined function**. With this concept, users are able to upload custom code and have it executed e.g. for every pixel of a scene, or applied to a particular dimension or set of dimensions, allowing custom server-side calculations. See the section on [UDFs](udfs.md) for more information.
+The abbreviation **UDF** stands for **user-defined function**. With this concept, users are able to upload custom code and have it executed e.g. for every pixel of a scene, or applied to a particular dimension or set of dimensions, allowing custom server-side calculations. See the section on [UDFs](udfs.md) for more information.
+
+# Data Processing modes
+
+Process graphs can be processed in three different ways:
+
+1. Results can be pre-computed by creating a ***batch job*** using  `POST /jobs`.  They are submitted to the back-end's processing system, but will remain inactive until `POST /jobs/{job_id}/results` has been called. They will run only once and store results after execution. Results can be downloaded. Batch jobs are typically time consuming so that user interaction is not possible. This is the only mode that allows to get an estimate about time, volume and costs beforehand.
+2. A more dynamic way of processing and accessing data is to create a **secondary web service**. They allow web-based access using different protocols such as [OGC WMS](http://www.opengeospatial.org/standards/wms), [OGC WCS](http://www.opengeospatial.org/standards/wcs) or [XYZ tiles](https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames). These protocols usually allow users to change the viewing extent or level of detail (zoom level). Therefore, computations often run *on demand* so that the requested data is calculated during the request. Back-ends should make sure to cache processed data to avoid additional/high costs and reduce waiting times for the user.
+3. Process graphs can also be executed **on-demand** (i.e. synchronously) by sending the process graph to `POST /result`. Results are delivered with the request itself and no job is created. Only lightweight computations, for example small previews, should be executed using this approach as timeouts are to be expected for [long-polling HTTP requests](https://www.pubnub.com/blog/2014-12-01-http-long-polling/).

docs/guidelines-api.md

@@ -14,13 +14,13 @@ In the specification the key words “MUST”, “MUST NOT”, “REQUIRED”, 
 
 Unless otherwise stated the API works case sensitive.
 
-All names SHOULD be written in snake case, i.e. words are separated with one underscore character (_) and no spaces, with all letters lowercased. Example: `hello_world`. This applies particularly to endpoints and JSON property names. HTTP header fields follow their respective casing conventions, e.g. `Content-Type` or `OpenEO-Costs`, despite being case-insensitive according to [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.2).
+All names SHOULD be written in snake case, i.e. words are separated with one underscore character (_) and no spaces, with all letters lowercased. Example: `hello_world`. This applies particularly to endpoints and JSON property names. HTTP header fields follow their respective casing conventions, e.g. `Content-Type` or `OpenEO-Costs`, despite being case-insensitive according to [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.2).
 
 ## Technical requirements
 
 ### HTTP
 
-The API developed by the openEO project uses [HTTP REST](https://en.wikipedia.org/wiki/Representational_state_transfer)[Level 2](https://martinfowler.com/articles/richardsonMaturityModel.html#level2) for communication between client and back-end server.
+The API developed by the openEO project uses [HTTP REST](https://en.wikipedia.org/wiki/Representational_state_transfer) [Level 2](https://martinfowler.com/articles/richardsonMaturityModel.html#level2) for communication between client and back-end server.
 
 Public APIs MUST be available via HTTPS only and all inbound calls MUST be HTTPS.