implement lazy rasterstack #62
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@vincentsarago please let me know what you think. |
…rts in implementations fix resample_spatial does not appear in the process list #60
…ry, and io modules
…nd improve median/mean functions
…and RasterStack, and improve error handling for GeoJSON format
…for RasterioIOError
|
@vincentsarago I simplify by enforcing rasterstack as data parameter only |
There was a problem hiding this comment.
Copilot reviewed 23 out of 28 changed files in this pull request and generated 1 comment.
Files not reviewed (5)
- services/earthsearch.json: Language not supported
- titiler/openeo/processes/data/aggregate_spatial.json: Language not supported
- titiler/openeo/processes/data/load_stac.json: Language not supported
- titiler/openeo/processes/data/reduce_dimension.json: Language not supported
- titiler/openeo/processes/data/resample_spatial.json: Language not supported
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Updated the Copernicus service configuration to include temporal extent in the process graph. - Modified the response handling in the EndpointsFactory to return image data directly. - Introduced new mathematical functions `first` and `last` to retrieve the first and last elements of an array, respectively. - Improved the temporal dimension reduction logic to return a RasterStack instead of ImageData. - Enhanced the LoadCollection class to validate input bands and extract resolution information more robustly. - Added JSON definitions for the new `first` and `last` processes, including examples and descriptions.
…ve STAC API reader with dimension estimation and error handling
…imports and formatting in various modules
…ve error handling for data extraction
… redundant 'properties.' prefix in argument definitions
…ve 'properties.' prefix in test cases and enhance pixel limit checks
Co-authored-by: Vincent Sarago <vincent.sarago@gmail.com>
…ndex and label; improve readability in LoadCollection class by formatting asset selection
adjust types and remove pyproj
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To address #57, this PR adds a new class LazyRasterStack implementing RasterStack that will read the data only when the imagedata in the dictionary is accessed. This way, the graph can easily support load_collection and an independent reducer without using load_collection_and_reduce systematically. This should also fix the issue of being aligned with datacube object type in #59. Finally, a good workload and output estimation must be implemented (complete #58) to ensure the graph is not impossible to process synchronously.
I tested sucessfully with some simple graph
[ ] merge branch Update subtype from 'imagedata' to 'datacube' #59