Merge remote-tracking branch 'origin/master' into hv_issue719-job-manager-threaded-job-start

Author: HansVRP

CHANGELOG.md

@@ -9,11 +9,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+### Changed
+
+### Removed
+
+### Fixed
+
+
+## [0.40.0] - 2025-04-14
+
+### Added
+
 - `sar_backscatter`: try to retrieve coefficient options from backend ([#693](https://github.com/Open-EO/openeo-python-client/issues/693))
+- Improve error message when OIDC provider is unavailable ([#751](https://github.com/Open-EO/openeo-python-client/issues/751))
+- Added `on_response_headers` argument to `DataCube.download()` and related to handle (e.g. `print`) the response headers ([#560](https://github.com/Open-EO/openeo-python-client/issues/560))
 
 ### Changed
 
-### Removed
+- When the bands provided to `Connection.load_stac(..., bands=[...])` do not fully match the bands the client extracted from the STAC metadata, a warning will be triggered, but the provided band names will still be used during the client-side preparation of the process graph. This is a pragmatic approach to bridge the gap between differing interpretations of band detection in STAC. Note that this might produce process graphs that are technically invalid and might not work on other backends or future versions of the backend you currently use. It is recommended to consult with the provider of the STAC metadata and openEO backend on the correct and future-proof band names. ([#752](https://github.com/Open-EO/openeo-python-client/issues/752))
 
 ### Fixed
 

docs/basics.rst

@@ -126,8 +126,8 @@ We load an initial small spatio-temporal slice (a data cube) as follows:
     sentinel2_cube = connection.load_collection(
         "SENTINEL2_L2A",
         spatial_extent={"west": 5.14, "south": 51.17, "east": 5.17, "north": 51.19},
-        temporal_extent = ["2021-02-01", "2021-04-30"],
-        bands=["B02", "B04", "B08"]
+        temporal_extent=["2021-02-01", "2021-04-30"],
+        bands=["B02", "B04", "B08"],
     )
 
 Note how we specify a the region of interest, a time range and a set of bands to load.
@@ -279,8 +279,8 @@ First, we load a new ``SENTINEL2_L2A`` based data cube with this specific ``SCL`
     s2_scl = connection.load_collection(
         "SENTINEL2_L2A",
         spatial_extent={"west": 5.14, "south": 51.17, "east": 5.17, "north": 51.19},
-        temporal_extent = ["2021-02-01", "2021-04-30"],
-        bands=["SCL"]
+        temporal_extent=["2021-02-01", "2021-04-30"],
+        bands=["SCL"],
     )
 
 Now we can use the compact "band math" feature again to build a
@@ -362,7 +362,7 @@ Building on the experience from previous sections, we first build a masked EVI c
     sentinel2_cube = connection.load_collection(
         "SENTINEL2_L2A",
         spatial_extent={"west": 5.14, "south": 51.17, "east": 5.17, "north": 51.19},
-        temporal_extent = ["2020-01-01", "2021-12-31"],
+        temporal_extent=["2020-01-01", "2021-12-31"],
         bands=["B02", "B04", "B08", "SCL"],
     )
 
@@ -413,6 +413,7 @@ which we massage a bit more:
     from openeo.rest.conversions import timeseries_json_to_pandas
 
     import json
+
     with open("evi-aggregation.json") as f:
         data = json.load(f)
 

docs/cookbook/job_manager.rst

@@ -77,9 +77,11 @@ Basic usage example with a remote process definition:
 
     # Initialize job database from a dataframe,
     # with desired parameter values to fill in.
-    df = pd.DataFrame({
-        "start_date": ["2021-01-01", "2021-02-01", "2021-03-01"],
-    })
+    df = pd.DataFrame(
+        {
+            "start_date": ["2021-01-01", "2021-02-01", "2021-03-01"],
+        }
+    )
     job_db = create_job_db("jobs.csv").initialize_from_df(df)
 
     # Create and run job manager,

docs/cookbook/localprocessing.rst

@@ -48,11 +48,12 @@ The following code snippet loads Sentinel-2 L2A data from a public STAC Catalog,
     >>> temporal_extent = ["2019-01-01", "2019-06-15"]
     >>> bands = ["red"]
     >>> properties = {"eo:cloud_cover": dict(lt=50)}
-    >>> s2_cube = local_conn.load_stac(url=url,
-    ...    spatial_extent=spatial_extent,
-    ...    temporal_extent=temporal_extent,
-    ...    bands=bands,
-    ...    properties=properties,
+    >>> s2_cube = local_conn.load_stac(
+    ...     url=url,
+    ...     spatial_extent=spatial_extent,
+    ...     temporal_extent=temporal_extent,
+    ...     bands=bands,
+    ...     properties=properties,
     ... )
     >>> s2_cube.execute()
     <xarray.DataArray 'stackstac-08730b1b5458a4ed34edeee60ac79254' (time: 177,
@@ -94,6 +95,7 @@ With some sample data we can now check the STAC metadata for the local files by
 .. code:: python
 
    from openeo.local import LocalConnection
+
    local_data_folders = [
        "./openeo-localprocessing-data/sample_netcdf",
        "./openeo-localprocessing-data/sample_geotiff",
@@ -162,10 +164,11 @@ We can perform the same example using data provided by STAC Collection:
 .. code:: python
 
     from openeo.local import LocalConnection
+
     local_conn = LocalConnection("./")
 
     url = "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a"
-    spatial_extent =  {"east": 11.40, "north": 46.52, "south": 46.46, "west": 11.25}
+    spatial_extent = {"east": 11.40, "north": 46.52, "south": 46.46, "west": 11.25}
     temporal_extent = ["2022-06-01", "2022-06-30"]
     bands = ["red", "nir"]
     properties = {"eo:cloud_cover": dict(lt=80)}

docs/cookbook/tricks.rst

@@ -69,11 +69,13 @@ For example:
 .. code-block:: python
 
     # `execute` with raw JSON string
-    connection.execute("""
+    connection.execute(
+        """
         {
             "add": {"process_id": "add", "arguments": {"x": 3, "y": 5}, "result": true}
         }
-    """)
+    """
+    )
 
     # `download` with local path to JSON file
     connection.download("path/to/my-process-graph.json")
@@ -99,7 +101,7 @@ where you could pass a *backend-side* path as ``geometries``, e.g.:
 
     cube = cube.aggregate_spatial(
         geometries="/backend/path/to/geometries.json",
-        reducer="mean"
+        reducer="mean",
     )
 
 The client would handle this by automatically adding a ``read_vector`` process
@@ -124,7 +126,7 @@ for example as follows:
 
     cube = cube.aggregate_spatial(
         geometries=process("read_vector", filename="/backend/path/to/geometries.json"),
-        reducer="mean"
+        reducer="mean",
     )
 
 Note that this is also works with older versions of the openEO Python client library.

docs/data_access.rst

@@ -274,7 +274,7 @@ For example, to filter on the relative orbit number of SAR data:
         "SENTINEL1_GRD",
         ...,
         properties={
-            "relativeOrbitNumber": lambda x: x==116
+            "relativeOrbitNumber": lambda x: x == 116,
         },
     )
 

docs/development.rst

@@ -246,7 +246,7 @@ we will use a concrete version ``0.8.0`` in the examples below.
 #.  Create release commit:
 
     A.  **Drop the pre-release suffix** from the version string in ``openeo/_version.py``
-        so that it just a "final" semantic versioning string, e.g. ``0.8.0``
+        so that it is just a "final" semantic versioning string, e.g. ``0.8.0``
 
     B.  **Update CHANGELOG.md**: rename the "Unreleased" section title
         to contain version and date, e.g.::

docs/examples/udf/smooth_savitzky_golay.py

@@ -0,0 +1,16 @@
+import xarray
+from scipy.signal import savgol_filter
+
+from openeo.udf import XarrayDataCube
+
+
+def apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:
+    """
+    Apply Savitzky-Golay smoothing to a timeseries datacube.
+    This UDF preserves dimensionality, and assumes an input
+    datacube with a temporal dimension 't' as input.
+    """
+    array: xarray.DataArray = cube.get_array()
+    filled = array.interpolate_na(dim="t")
+    smoothed_array = savgol_filter(filled.values, 5, 2, axis=0)
+    return XarrayDataCube(array=xarray.DataArray(smoothed_array, dims=array.dims, coords=array.coords))

docs/examples/udf/udf_modify_spatial.py

@@ -0,0 +1,55 @@
+import numpy as np
+import xarray
+
+from openeo.metadata import CollectionMetadata
+from openeo.udf import XarrayDataCube
+from openeo.udf.debug import inspect
+
+
+def apply_metadata(input_metadata: CollectionMetadata, context: dict) -> CollectionMetadata:
+
+    xstep = input_metadata.get("x", "step")
+    ystep = input_metadata.get("y", "step")
+    new_metadata = {
+        "x": {"type": "spatial", "axis": "x", "step": xstep / 2.0, "reference_system": 4326},
+        "y": {"type": "spatial", "axis": "y", "step": ystep / 2.0, "reference_system": 4326},
+        "t": {"type": "temporal"},
+    }
+    return CollectionMetadata(new_metadata)
+
+
+def fancy_upsample_function(array: np.array, factor: int = 2) -> np.array:
+    assert array.ndim == 3
+    return array.repeat(factor, axis=-1).repeat(factor, axis=-2)
+
+
+def apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:
+    array: xarray.DataArray = cube.get_array()
+
+    cubearray: xarray.DataArray = cube.get_array().copy() + 60
+
+    # We make prediction and transform numpy array back to datacube
+
+    # Pixel size of the original image
+    init_pixel_size_x = cubearray.coords["x"][-1] - cubearray.coords["x"][-2]
+    init_pixel_size_y = cubearray.coords["y"][-1] - cubearray.coords["y"][-2]
+
+    if cubearray.data.ndim == 4 and cubearray.data.shape[0] == 1:
+        cubearray = cubearray[0]
+    predicted_array = fancy_upsample_function(cubearray.data, 2)
+    inspect(predicted_array, "test message")
+    coord_x = np.linspace(
+        start=cube.get_array().coords["x"].min(),
+        stop=cube.get_array().coords["x"].max() + init_pixel_size_x,
+        num=predicted_array.shape[-2],
+        endpoint=False,
+    )
+    coord_y = np.linspace(
+        start=cube.get_array().coords["y"].min(),
+        stop=cube.get_array().coords["y"].max() + init_pixel_size_y,
+        num=predicted_array.shape[-1],
+        endpoint=False,
+    )
+    predicted_cube = xarray.DataArray(predicted_array, dims=["bands", "x", "y"], coords=dict(x=coord_x, y=coord_y))
+
+    return XarrayDataCube(predicted_cube)

docs/federation-extension.rst

@@ -37,6 +37,7 @@ using :py:meth:`OpenEoCapabilities.ext_federation_backend_details() <openeo.rest
 .. code-block:: python
 
     import openeo
+
     connection = openeo.connect(url=...)
     capabilities = connection.capabilities()
     print("Federated backends:", capabilities.ext_federation_backend_details())
@@ -57,6 +58,7 @@ and can be inspected as follows:
 .. code-block:: python
 
     import openeo
+
     connection = openeo.connect(url=...)
     collections = connection.list_collections()
     print("Number of collections:", len(collections))