Skip to content

Commit 1fec3d5

Browse files
committed
Try only allowing dask 10Gib in GHA
The memory limit argument is used to tell dask how much memory it is allowed to use (and maybe to work out chuck sizes, I'm not sure). I think GitHub actions runners have 16GB of RAM, so dask needs to use substantially less to not be killed. The log messages from dask in the failed runs indicated it had decided to use about 14GB, so it must use a heuristic that assumes it can use most of the available ram, too much in this case.
1 parent 39ab72a commit 1fec3d5

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

tests/integration_test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ odc-stats --version
1212
echo "Test LS GeoMAD"
1313

1414
odc-stats save-tasks --config https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/709daaee176c04e33de4cc9600462717cca5b34d/dev/services/odc-stats/geomedian/ga_ls8c_nbart_gm_cyear_3.yaml --year=2015 --tiles 49:50,24:25 --overwrite ls-geomad-cyear.db
15-
odc-stats run --memory-limit=15.6GiB --threads=1 --config https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/709daaee176c04e33de4cc9600462717cca5b34d/dev/services/odc-stats/geomedian/ga_ls8c_nbart_gm_cyear_3.yaml --location file:///tmp --overwrite ls-geomad-cyear.db
15+
odc-stats run --memory-limit=10GiB --threads=2 --config https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/709daaee176c04e33de4cc9600462717cca5b34d/dev/services/odc-stats/geomedian/ga_ls8c_nbart_gm_cyear_3.yaml --location file:///tmp --overwrite ls-geomad-cyear.db
1616

1717
./tests/compare_data.sh /tmp/x49/y24/ ga_ls8c_nbart_gm_cyear_3_x49y24_2015--P1Y_final*.tif
1818

0 commit comments

Comments
 (0)