custom_resources: AwsCustomResource deployment fails with "Uploaded file must be a non-empty zip" error #34176

jscott-resilient · 2025-04-17T14:27:48Z

Describe the bug

My project is currently on version 2.165.0 and upgrading to anything higher (I've tried 2.166.0) causes the following error to occur:

cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 7a077b2b-1336-4280-b3c1-198d2fbc08c2) (SDK Attempt Count: 1)

The construct code is as follows:

import * as cdk from 'aws-cdk-lib';

import { Construct } from 'constructs';

interface CrossRegionParameterReaderProps {
  region: string;
  parameterName: string;
  resource: string;
}

export class CrossRegionParameterReader extends cdk.custom_resources.AwsCustomResource {
  constructor(scope: Construct, name: string, props: CrossRegionParameterReaderProps) {
    const { parameterName, region, resource } = props;

    const ssmAwsSdkCall: cdk.custom_resources.AwsSdkCall = {
      service: 'SSM',
      action: 'getParameter',
      parameters: {
        Name: parameterName
      },
      region,
      physicalResourceId: { id: Date.now().toString() } // Update physical id to always fetch the latest version
    };

    super(scope, name, {
      onUpdate: ssmAwsSdkCall,
      policy: {
        statements: [
          new cdk.aws_iam.PolicyStatement({
            resources: [resource],
            actions: ['ssm:GetParameter'],
            effect: cdk.aws_iam.Effect.ALLOW
          })
        ]
      }
    });
  }

  public getParameterValue(): string {
    return this.getResponseField('Parameter.Value').toString();
  }
}

Thanks for looking into this! If you need anymore information please let me know!

Regression Issue

Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.165

Expected Behavior

The zipped lambda code within the cdk bootstrap s3 bucket is formatted correctly to allow deployment

Current Behavior

The zipped lambda code is defined within the bootstrap s3 bucket but causes the "Uploaded file must be a non-empty zip", it is a non empty zip file so the issue causing this error is unknown

Reproduction Steps

Upgrading aws-cdk-lib version higher than 2.165

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.1010.0 (build 6b421db)

Framework Version

No response

Node.js Version

v20.13.1

OS

macOS Sequoia Version 15.4

Language

TypeScript

Language Version

Typescript (5.8.3)

Other information

No response

The text was updated successfully, but these errors were encountered:

ykethan · 2025-04-17T15:31:25Z

Hey @jscott-resilient, thank you for reaching out. On diving into the issue and working on reproducing the issue using the code provided, here's what i have tried

Reproductions steps:

installed: "aws-cdk-lib": "2.165.0" and defined the stack as follows

import * as cdk from "aws-cdk-lib";
import "source-map-support/register";
import { CrossRegionParameterReader } from "../lib/custom-empty-zip";

const app = new cdk.App();

// Create a stack to contain our resources
const stack = new cdk.Stack(app, "MyReproStack", {
  crossRegionReferences: true,
  env: {
    region: "us-east-1",
  },
});

// Create the CrossRegionParameterReader within the stack
new CrossRegionParameterReader(stack, "parameterreader", {
  region: "us-west-1",
  parameterName: "/my-app/staging/test",
  resource: "arn:aws:ssm:us-west-1:<account-id>:parameter/my-app/staging/test",
});

import * as cdk from "aws-cdk-lib";

import { Construct } from "constructs";

interface CrossRegionParameterReaderProps {
  region: string;
  parameterName: string;
  resource: string;
}

export class CrossRegionParameterReader extends cdk.custom_resources
  .AwsCustomResource {
  constructor(
    scope: Construct,
    name: string,
    props: CrossRegionParameterReaderProps
  ) {
    const { parameterName, region, resource } = props;

    const ssmAwsSdkCall: cdk.custom_resources.AwsSdkCall = {
      service: "SSM",
      action: "getParameter",
      parameters: {
        Name: parameterName,
      },
      region,
      physicalResourceId: { id: Date.now().toString() }, // Update physical id to always fetch the latest version
    };

    super(scope, name, {
      onUpdate: ssmAwsSdkCall,
      policy: {
        statements: [
          new cdk.aws_iam.PolicyStatement({
            resources: [resource],
            actions: ["ssm:GetParameter"],
            effect: cdk.aws_iam.Effect.ALLOW,
          }),
        ],
      },
    });
  }

  public getParameterValue(): string {
    return this.getResponseField("Parameter.Value").toString();
  }
}

cdk deploy
Then, updated the package.json to "aws-cdk-lib": "2.166.0",
cdk deploy but did not observe an error
made some change such as physicalResourceId and tried deploying. Additionally, changed the resource name to re-deploy the resource. But did not observe a Uploaded file must be a non-empty zip error.

Could let us know if the reproduction steps used does align with your project setup and the steps used?
Do provide us any additional information that could help us reproduce the issue?

github-actions · 2025-04-19T16:05:34Z

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

jscott-resilient · 2025-04-22T10:13:44Z

Ah sorry for missing the reply, easter celebrations got in the way! Thanks for looking into this, so it does look like you have reproduced it exactly. The only difference on our side is that the region parameter passed into CrossRegionParameterReader is us-east-1 and the region parameter for the app is eu-west-2.

So when deploying here are the logs that cdk produces under version 2.165.0

collaborate-api-cf-dev: creating CloudFormation changeset...
collaborate-api-cf-dev | 0/7 | 10:35:54 AM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack               | collaborate-api-cf-dev User Initiated
collaborate-api-cf-dev | 0/7 | 10:35:56 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 1/7 | 10:35:57 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/7 | 10:37:17 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) Requested update required the provider to create a new physical resource
collaborate-api-cf-dev | 2/7 | 10:37:17 AM | UPDATE_COMPLETE      | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 3/7 | 10:37:19 AM | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 
collaborate-api-cf-dev | 3/7 | 10:37:20 AM | DELETE_IN_PROGRESS   | AWS::CloudFormation::CustomResource      | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 2/7 | 10:37:22 AM | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource      | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 3/7 | 10:37:22 AM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 

 ✅  cf (collaborate-api-cf-dev)

✨  Deployment time: 103.47s

and then the logs for 2.166.0

cf (collaborate-api-cf-dev): deploying... [3/4]
collaborate-api-cf-dev: creating CloudFormation changeset...
collaborate-api-cf-dev | 0/8 | 11:07:33 AM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack               | collaborate-api-cf-dev User Initiated
collaborate-api-cf-dev | 0/8 | 11:07:36 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 0/8 | 11:07:36 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 0/8 | 11:07:37 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/8 | 11:07:37 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/8 | 11:07:38 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 7c09f0a5-3340-4d12-b08b-98be30074e2d) (SDK Attempt Count: 1)" (RequestToken: 45c590e9-c541-a3a6-3224-250ffa5f2856, HandlerErrorCode: InvalidRequest)
collaborate-api-cf-dev | 1/8 | 11:07:39 AM | UPDATE_FAILED        | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) Resource update cancelled
collaborate-api-cf-dev | 1/8 | 11:07:39 AM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack               | collaborate-api-cf-dev The following resource(s) failed to update: [GraphQlEndpointCertificateReaderA3F7C336, CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A]. 
collaborate-api-cf-dev | 1/8 | 11:07:41 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 1/8 | 11:07:41 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/8 | 11:07:41 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 0/8 | 11:07:42 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/8 | 11:07:49 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 2/8 | 11:09:31 AM | UPDATE_COMPLETE      | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 3/8 | 11:09:32 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 
collaborate-api-cf-dev | 4/8 | 11:09:33 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 

Failed resources:
collaborate-api-cf-dev | 11:07:38 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 7c09f0a5-3340-4d12-b08b-98be30074e2d) (SDK Attempt Count: 1)" (RequestToken: 45c590e9-c541-a3a6-3224-250ffa5f2856, HandlerErrorCode: InvalidRequest)
❌  collaborate-api-cf-dev failed: _ToolkitError: The stack named collaborate-api-cf-dev failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 7c09f0a5-3340-4d12-b08b-98be30074e2d) (SDK Attempt Count: 1)" (RequestToken: 45c590e9-c541-a3a6-3224-250ffa5f2856, HandlerErrorCode: InvalidRequest)

No other changes have been made to the code, is there something I could look for on my end which could be a likely candidate to cause this error?

ykethan · 2025-04-22T18:42:51Z

@jscott-resilient thank you for the information. Retried the reproduction with different region as suggested but i was able not able to reproduce the issue. But noticed similar issues previously opened #18459 and #32869.

it does appear a fix was released in the newer versions of AWS CDK #33201. Could you try upgrading to latest or atleast aws-cdk-lib to v2.183.0 and retry deploy?
Additionally, from comment latest #18459 (comment) it does appear removing the cdk.out and re-running synth or deploy does mitigate the issue.

Do let me us know if this does not fix the issue.

github-actions · 2025-04-24T20:06:02Z

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

jscott-resilient · 2025-04-25T09:19:46Z

Hi @ykethan,

Thanks for looking into this! So I tried version 2.192.0 and still no luck (even with removing cdk.out and running synth).

I ran synth and retrieved the zip file from the assets s3 bucket using version 2.165.0 and 2.192.0. For 2.165.0 the zip file successfully unzipped and contained a index.js and a entrypoint.js file. But doing the same for 2.192.0 I get the following error when trying to unzip the file

so it looks like the error in the logs is correct!

I have tried deleting the asset from s3 bucket but still the same

Do you have any more suggestions?

ykethan · 2025-04-25T20:24:00Z

Hey @jscott-resilient, interesting. With the latest versions of CDK packages installed could you try running the deploy command with --force flag and let us know if this mitigated the issue? This should try and force publish all assets

cdk deploy --force

pahud · 2025-04-25T21:52:42Z

Hi @jscott-resilient,

Thanks for the detailed report.

To help diagnose why your specific deployment fails with v2.166.0+, could you please provide details about the environment for your CrossRegionParameterReader custom resource?

VPC: Is it configured to run within a VPC?
Internet: If yes, do the subnets have internet egress (e.g., via NAT Gateway), or are they isolated?
Explicit Context/Props: Have you explicitly set installLatestAwsSdk on the resource instance, or set the context flag @aws-cdk/customresources:installLatestAwsSdkDefault in cdk.json or via --context?

jscott-resilient · 2025-04-29T10:12:20Z

Hi @ykethan,

Gave it a try, unfortunately the same result :(

This is the deploy command that was run with the latest version (at the time 2.192.0)

cdk deploy cf --app .build/stacks/stacks.js -o .build/cdk.out/cf --no-staging --require-approval never --progress=events --force

and the output

collaborate-api-cf-dev: creating CloudFormation changeset...
collaborate-api-cf-dev | 0/9 | 10:33:16 AM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack               | collaborate-api-cf-dev User Initiated
collaborate-api-cf-dev | 0/9 | 10:33:19 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 1/9 | 10:33:21 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 87b890c3-49f9-4bc7-8818-56cd24880cd9) (SDK Attempt Count: 1)" (RequestToken: 3c99a8c8-2033-7aea-6d01-0ad3da60b68b, HandlerErrorCode: InvalidRequest)
collaborate-api-cf-dev | 1/9 | 10:33:22 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) Resource update cancelled
collaborate-api-cf-dev | 1/9 | 10:33:22 AM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack               | collaborate-api-cf-dev The following resource(s) failed to update: [AWS679f53fac002430cb0da5b7982bd22872D164C4C, CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A]. 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 0/9 | 10:33:25 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:32 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 2/9 | 10:33:32 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 3/9 | 10:33:33 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 
collaborate-api-cf-dev | 4/9 | 10:33:34 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 

Failed resources:
collaborate-api-cf-dev | 10:33:21 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 87b890c3-49f9-4bc7-8818-56cd24880cd9) (SDK Attempt Count: 1)" (RequestToken: 3c99a8c8-2033-7aea-6d01-0ad3da60b68b, HandlerErrorCode: InvalidRequest)
❌  collaborate-api-cf-dev failed: _ToolkitError: The stack named collaborate-api-cf-dev failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 87b890c3-49f9-4bc7-8818-56cd24880cd9) (SDK Attempt Count: 1)" (RequestToken: 3c99a8c8-2033-7aea-6d01-0ad3da60b68b, HandlerErrorCode: InvalidRequest)

and thanks for joining the fight @pahud! I Really appreciate it!

The lambda function is not configured to run within a VPC and within the CrossRegionParameterReader Construct (which extends AwsCustomResource) the installLatestAwsSdk prop is left undefined, which reading the docs defaults to true.

pahud · 2025-05-02T11:39:26Z

@jscott-resilient

Hi, this is interesting and weird. Are you able to create a new repo with minimal cdk code required for repro or just make it a zip and attach to this issue so we can clone/download and try deploy in our environment? We need to make sure we use the same codebase. If yes, please make sure only include general required code, don't include business-related code. Thank you.

github-actions · 2025-05-05T04:07:34Z

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

jscott-resilient · 2025-05-07T13:07:23Z

So I've created a separate repo to test this in and I'm no longer getting the error (which isn't very helpful), but I have noticed something in the logs.

In the new demo repo the lambda function is referred to by a generated id AWS679f53fac002430cb0da5b7982bd2287 (version 2.194.0)

demo-stack | 0/5 | 11:10:20 | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | demo-stack User Initiated
demo-stack | 0/5 | 11:10:22 | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata         | CDKMetadata/Default (CDKMetadata) 
demo-stack | 1/5 | 11:10:23 | UPDATE_COMPLETE      | AWS::CDK::Metadata         | CDKMetadata/Default (CDKMetadata) 
demo-stack | 1/5 | 11:10:23 | UPDATE_IN_PROGRESS   | AWS::Lambda::Function      | AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
demo-stack | 2/5 | 11:10:31 | UPDATE_COMPLETE      | AWS::Lambda::Function      | AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
demo-stack | 2/5 | 11:10:31 | UPDATE_IN_PROGRESS   | Custom::AWS                | GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
demo-stack | 2/5 | 11:11:44 | UPDATE_IN_PROGRESS   | Custom::AWS                | GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) Requested update required the provider to create a new physical resource
demo-stack | 3/5 | 11:11:44 | UPDATE_COMPLETE      | Custom::AWS                | GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
demo-stack | 4/5 | 11:11:45 | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack | demo-stack 
demo-stack | 4/5 | 11:11:46 | DELETE_IN_PROGRESS   | AWS::CloudFormation::CustomResource | GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
demo-stack | 3/5 | 11:11:48 | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource | GraphQlEndp

but in the original project using version 2.165.0 the logs refer to the lambda function as cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler

collaborate-api-cf-dev: creating CloudFormation changeset...
collaborate-api-cf-dev | 0/7 | 10:35:54 AM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack               | collaborate-api-cf-dev User Initiated
collaborate-api-cf-dev | 0/7 | 10:35:56 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 1/7 | 10:35:57 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/7 | 10:37:17 AM | UPDATE_IN_PROGRESS   | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) Requested update required the provider to create a new physical resource
collaborate-api-cf-dev | 2/7 | 10:37:17 AM | UPDATE_COMPLETE      | Custom::AWS                              | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 3/7 | 10:37:19 AM | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 
collaborate-api-cf-dev | 3/7 | 10:37:20 AM | DELETE_IN_PROGRESS   | AWS::CloudFormation::CustomResource      | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 2/7 | 10:37:22 AM | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource      | cf/GraphQlEndpointCertificateReader/Resource/Default (GraphQlEndpointCertificateReaderA3F7C336) 
collaborate-api-cf-dev | 3/7 | 10:37:22 AM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack               | collaborate-api-cf-dev

and in the same project (using 2.192.0) the logs refer to two lambda functions cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler and AWS679f53fac002430cb0da5b7982bd2287, could that be a potential issue?

collaborate-api-cf-dev | 0/9 | 10:33:16 AM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack               | collaborate-api-cf-dev User Initiated
collaborate-api-cf-dev | 0/9 | 10:33:19 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/9 | 10:33:20 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 1/9 | 10:33:21 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) Resource handler returned message: "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400, Request ID: 87b890c3-49f9-4bc7-8818-56cd24880cd9) (SDK Attempt Count: 1)" (RequestToken: 3c99a8c8-2033-7aea-6d01-0ad3da60b68b, HandlerErrorCode: InvalidRequest)
collaborate-api-cf-dev | 1/9 | 10:33:22 AM | UPDATE_FAILED        | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) Resource update cancelled
collaborate-api-cf-dev | 1/9 | 10:33:22 AM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack               | collaborate-api-cf-dev The following resource(s) failed to update: [AWS679f53fac002430cb0da5b7982bd22872D164C4C, CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A]. 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 1/9 | 10:33:25 AM | UPDATE_IN_PROGRESS   | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 0/9 | 10:33:25 AM | UPDATE_COMPLETE      | AWS::CDK::Metadata                       | cf/CDKMetadata/Default (CDKMetadata) 
collaborate-api-cf-dev | 1/9 | 10:33:32 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/AWS679f53fac002430cb0da5b7982bd2287 (AWS679f53fac002430cb0da5b7982bd22872D164C4C) 
collaborate-api-cf-dev | 2/9 | 10:33:32 AM | UPDATE_COMPLETE      | AWS::Lambda::Function                    | cf/Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler (CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A) 
collaborate-api-cf-dev | 3/9 | 10:33:33 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev 
collaborate-api-cf-dev | 4/9 | 10:33:34 AM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack               | collaborate-api-cf-dev

The next step I can do in dev is tear down the stack and rebuild it from scratch to see if it would work, but this will be impossible to do in prod so is there anything else I could try?

Thanks again for helping out,

Joe

ykethan · 2025-05-07T18:11:36Z

Hey @jscott-resilient , thank you for getting back to us on this. while this may be the unlikely, could you try running the following to verify the versions used?

npm list aws-cdk-lib aws-cdk

Then try removing the node_modules, package-lock.json and cdk.out.
reinstall the packages npm install
redeploying cdk with a --force

this is to rule out if the packages may be the reason for this behavior and verify by rebuilding and deploying the assets.

github-actions · 2025-05-09T20:06:05Z

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

jscott-resilient added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 17, 2025

github-actions bot added potential-regression Marking this issue as a potential regression to be checked by team member @aws-cdk/custom-resources Related to AWS CDK Custom Resources labels Apr 17, 2025

ykethan added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Apr 17, 2025

github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Apr 19, 2025

github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Apr 22, 2025

ykethan added the needs-triage This issue or PR still needs to be triaged. label Apr 22, 2025

ykethan added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Apr 22, 2025

github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Apr 24, 2025

github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Apr 25, 2025

ykethan added the needs-triage This issue or PR still needs to be triaged. label Apr 25, 2025

ykethan added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Apr 25, 2025

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Apr 26, 2025

github-actions bot mentioned this issue May 1, 2025

Monthly issue metrics report #34323

Open

ykethan added the needs-triage This issue or PR still needs to be triaged. label May 1, 2025

pahud added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels May 2, 2025

pahud self-assigned this May 2, 2025

pahud added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label May 5, 2025

github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label May 5, 2025

github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels May 7, 2025

ykethan added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label May 7, 2025

github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label May 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

custom_resources: AwsCustomResource deployment fails with "Uploaded file must be a non-empty zip" error #34176

custom_resources: AwsCustomResource deployment fails with "Uploaded file must be a non-empty zip" error #34176

jscott-resilient commented Apr 17, 2025 •

edited

Loading

ykethan commented Apr 17, 2025

github-actions bot commented Apr 19, 2025

jscott-resilient commented Apr 22, 2025

ykethan commented Apr 22, 2025

github-actions bot commented Apr 24, 2025

jscott-resilient commented Apr 25, 2025

ykethan commented Apr 25, 2025

pahud commented Apr 25, 2025

jscott-resilient commented Apr 29, 2025

pahud commented May 2, 2025

github-actions bot commented May 5, 2025

jscott-resilient commented May 7, 2025

ykethan commented May 7, 2025

github-actions bot commented May 9, 2025

custom_resources: AwsCustomResource deployment fails with "Uploaded file must be a non-empty zip" error #34176

custom_resources: AwsCustomResource deployment fails with "Uploaded file must be a non-empty zip" error #34176

Comments

jscott-resilient commented Apr 17, 2025 • edited Loading

Describe the bug

Regression Issue

Last Known Working CDK Version

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

ykethan commented Apr 17, 2025

github-actions bot commented Apr 19, 2025

jscott-resilient commented Apr 22, 2025

ykethan commented Apr 22, 2025

github-actions bot commented Apr 24, 2025

jscott-resilient commented Apr 25, 2025

ykethan commented Apr 25, 2025

pahud commented Apr 25, 2025

jscott-resilient commented Apr 29, 2025

pahud commented May 2, 2025

github-actions bot commented May 5, 2025

jscott-resilient commented May 7, 2025

ykethan commented May 7, 2025

github-actions bot commented May 9, 2025

jscott-resilient commented Apr 17, 2025 •

edited

Loading