Skip to content

[EPIC] multi-image/version pipelines #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jdries opened this issue Oct 16, 2024 · 3 comments
Open

[EPIC] multi-image/version pipelines #25

jdries opened this issue Oct 16, 2024 · 3 comments
Assignees

Comments

@jdries
Copy link
Contributor

jdries commented Oct 16, 2024

We now support running batch jobs with multiple images, but do not have a way to keep them up to date.
Some design considerations:

  • to enable transitioning UDF's between python 3.8 and 3.11, users will need up to date versions of both images
  • in integration testing, it makes sense to run some batch job tests with both versions, but not necessarily all of them
  • synchronous calls remain on one fixed version
  • unit tests for openeo-geopyspark-driver: run against both versions
  • image building pipelines may need separation from integrationtests: now we always build a full new image for each tests
  • artifactory has issues dealing with too many dev images, we don't want to make this even worse. Images with failed tests should not be retained long, while images with successful tests are often more useful to keep longer. (https://github.com/eu-cdse/openeo-cdse-infra/issues/274)
@soxofaan
Copy link
Member

Related to desire to drop Python 3.8 support in openeo python client (because quite some relevant libraries already stopped caring about python 3.8, e.g. geopandas, pystac, ...):

@jdries
Copy link
Contributor Author

jdries commented Feb 3, 2025

When we continuously build new images, we can continue with this one:
Open-EO/openeo-geopyspark-driver#752

@JeroenVerstraelen JeroenVerstraelen assigned jdries and unassigned tcassaert Apr 1, 2025
jdries added a commit that referenced this issue Apr 23, 2025
jdries added a commit that referenced this issue Apr 23, 2025
@jdries
Copy link
Contributor Author

jdries commented Apr 23, 2025

There's now a separate jenkins pipeline to trigger building and testing for python 3.11.
It uses same git branch, so the jenkinsfile itself is made to be generic.
https://jenkins.vgt.vito.be/job/openEO/job/openeo-integrationtests-python311/

jdries added a commit that referenced this issue Apr 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants