force download = true

[ericemc3]

This isn’t really an issue,
but I observed a noticeable speed improvement (about 2×) with SET force_download = TRUE;.

set zipfs_extension_remove = TRUE ;
set zipfs_extension = '!!' ;

SET force_download = TRUE ;

FROM 'zip://https://api.insee.fr/melodi/file/DS_TOUR_CAP/DS_TOUR_CAP_CSV_FR!!*_data.csv' ;

-> 4 s

SET force_download = FALSE; -- default setting

FROM 'zip://https://api.insee.fr/melodi/file/DS_TOUR_CAP/DS_TOUR_CAP_CSV_FR!!*_data.csv' ;

-> 8 s

I assume it involves a trade-off between bandwidth, the number of HTTP HEAD/GET requests, and performance.

[isaacbrodsky]

This makes sense because zipfs is probably reading the central directory once to resolve the glob and then again to read the files, and perhaps multiple times for each file. If you know the exact path within the zip file it may be the same speed as using force_download.

Ref: https://duckdb.org/docs/stable/configuration/overview.html