Skip to content

Conversation

mackenzie-grimes-noaa
Copy link
Contributor

Linear Issue

IDSSE-1082

Changes

  • Bug fix to always including trailing slash to s3 URL paths when running ls(), adding one when the function caller did not

Explanation

Since base class ProtocolUtils was introduced, ls was being invoked the same way for AWS and HTTP servers. On HTTP servers this works with or without a trailing slash, but on AWS S3 buckets a trailing slash is required.

E.g. this command will return just 1 item--the folder itself--instead of the folder's contents:

% s5cmd --no-sign-request ls \
  s3://noaa-nbm-grib2-pds/blend.20250107/20/core

                                  DIR  core/

But this command will correctly return the list of files inside the S3 folder:

s5cmd --no-sign-request ls \
  s3://noaa-nbm-grib2-pds/blend.20250107/20/core/

2025/01/07 20:54:59          52070831  blend.t20z.core.f001.ak.grib2
2025/01/07 20:55:01              5948  blend.t20z.core.f001.ak.grib2.idx
2025/01/07 20:55:01         160560514  blend.t20z.core.f001.co.grib2
2025/01/07 20:55:05             21866  blend.t20z.core.f001.co.grib2.idx
2025/01/07 20:54:57            662248  blend.t20z.core.f001.gu.grib2
...

@mackenzie-grimes-noaa mackenzie-grimes-noaa merged commit cbb5d18 into main Jan 13, 2025
2 checks passed
@mackenzie-grimes-noaa mackenzie-grimes-noaa deleted the bug/aws-ls-trailing-slash branch January 13, 2025 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant