Missing ETag from S3 UploadPart response

[madsborup]

Bug report

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

Similar to aws/aws-sdk-js-v3#5708

I'm trying to implement MultiPartUpload using the AWS S3 client, but the response object of UploadPart does not include the ETag property, though it's included in the HTTP response. I suspect this may be due to ETag header not allowed in CORS settings (as mentioned in the above similar issue).

To Reproduce

const client = new S3Client({
  forcePathStyle: true,
  region: "eu-central-1",
  endpoint: "https://....supabase.co/storage/v1/s3",
  credentials: {
    accessKeyId: "...",
    secretAccessKey: process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY,
    sessionToken: userSession.access_token,
  },
});

const uploadPartResponse = await client.send(
  new UploadPartCommand({
    Bucket: BUCKET_NAME,
    Key: key,
    PartNumber: partNumber,
    UploadId: uploadId,
    Body: chunk,
  })
);

Expected behavior

ETag property should be present in uploadPartResponse object.

Screenshots

System information

• OS: macOS
• @supabase/supabase-js: ^2.48.1
• @aws-sdk/client-s3: ^3.733.0

[Rudra-Sankha-Sinhamahapatra]

@madsborup are you working on this issue?

[cboycecollab]

Same here! I am experiencing the same thing. Whether using the Upload functionality from @aws-sdk/lib-storage or just using the aws-sdk/client-s3 everything works fine except that the ETag is not being propagated back to the code so the uploads will never complete.

[cboycecollab]

@madsborup Are you currently working on this issue?

[madsborup]

@cboycecollab No, I contacted support but haven't received any updates...

[cboycecollab]

@madsborup Okay no worries, I honestly surprised that nobody else has mentioned this as an issue. It makes the S3 functionality useless from when uploading from the front-end

[fenos]

Hello, guys i'm looking into this

[fenos]

By the first look, it seems that we have a test covering this.
https://github.com/supabase/storage/blob/master/src/test/s3-protocol.test.ts#L475

however, I'm digging deeper

[fenos]

Got it, i believe the issue is at the cloudflare level, where the etag is getting modified / removed.
I'll be looking into adding a page rule for the S3 endpoint to allow cloudflare to pass the correct etag on this operation.

Stay tuned

[cboycecollab]

@fenos you are the goat thank you for looking into this. I am curious though, how did you come to the conclusion that it was an issue at the cloudflare level? Additionally, from what you've been able to uncover, does there seem to be any immediate workarounds?

[fenos]

The test is passing locally regarding the Etag, as I was debugging it locally I couldn't find anything wrong with it.

However, I had a couple of instances where cloudflare was changing the request to storage and I had to patch it with custom page rules.

after reading how they handle etags in headers it looks like they are issue, which can be disabled.

We use cloudflare in front of storage at the moment hence is very likely this is the issue.

I'll send a patch sometimes next week 👍🏻

[cboycecollab]

Really appreciate it dude! Yeah the upload worked fine as long as there was only one chunk in the upload but as soon as there was multiple chunks it started screaming about the ETag.

[cboycecollab]

@fenos please let me know when / if you get a chance to submit a PR for this!

[SouthCentralCode]

hey,

I'm getting the exact same issue of etag not being accessible in the headers of the response within a react js component, however this is occurring for me on the local docker supabase instance i'm testing with.

I am using signedUrl instead of the s3 client as i am working client-side, but getting the signed url via an api call to the server.

The signedUrl functionality is working as expected for the other multipart related calls.

Assuming this is a CORS issue on my local instance, i'm struggling on how i can override the CORS setting anywhere.

[nvkane15]

@fenos Appreciate you looking into this as we are experiencing the issue as well. Any update on when a fix might get into production?

[fenos]

Hey guys, sorry, but just got back from paternity leave
Will prioritise this

[fenos]

I'm unable to reproduce the issue on my end against the supabase.com hosted version.
Do you guys know how to reproduce this reliably?

A simple test I've done:

  async function send() {
    const key = 'test-path-1.jpg'
    const chunk = Buffer.alloc(1025 * 5)

    const multipart = await client.send(
        new CreateMultipartUploadCommand({
            Bucket: BUCKET_NAME,
            Key: key,
        })
    )
    
    const uploadPartResponse = await client.send(
        new UploadPartCommand({
          Bucket: BUCKET_NAME,
          Key: key,
          PartNumber: 1,
          UploadId: multipart.UploadId,
          Body: chunk,
        })
      );

     console.log('etag', uploadPartResponse.ETag)
  }

the etag is printed just fine

[fenos]

are you guys doing this on the server or browser?

[madsborup]

Browser. I assume it works fine server side, as the HTTP response includes the ETag (as in my original post), but it's not present in the response object, which is why I proposed it may be a CORS issue.

[SouthCentralCode]

i had this issue on my local docker supabase instance from browser client.

I dug around in Kong etc... but could'nt figure out how to add the CORS setting..

Access-Control-Expose-Headers: Etag

I had to move over to AWS to get it working.

[fenos]

Gotcha!
Yes, i'm now fully on board with this. It's definitely a CORS issue, I'm working for a solution

[cboycecollab]

Hey @fenos yeah I was doing it from the Browser using the Upload utility authenticating with the ANON service key from the front-end.

const upload = new Upload(s3Client, {
  Bucket: 'bucket-name',
  Key: 'path/to/file',
  ContentType: 'image/jpeg',
  Body: file,
})

[cboycecollab]

Hey @fenos just wanted to follow up in here to ask about any progress on this issue! Cheers

[cboycecollab]

Hey @fenos just wanted to bump this!