DD_EXTENSION | ERROR | Failed to buffer the request body: length limit exceeded

[ex-nerd]

I recently upgraded from v65 to v87 (and now to v88) and am suddenly seeing this error in some of my lambda logs:

DD_EXTENSION | ERROR | Failed to buffer the request body: length limit exceeded

Additionally, about 5s after this an uncaught exception is logged by the lambda layer:

Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:95:16)
at WriteWrap.callbackTrampoline (node:internal/async_hooks:130:17)

Followed almost immediately by:

LAMBDA_RUNTIME Failed to post handler success response. Http response code: 403. {"errorMessage":"State transition from Ready to InvocationErrorResponse failed for runtime. Error: State transition is not allowed","errorType":"InvalidStateTransition"}

This ultimately results in my API calls receiving 502 errors from the server.

Unfortunately, due to the error itself I don't know much about the request itself other than the response payload is rather large.

Even if DD can't send logs to its own server, that should never prevent a response from being returned to my API clients.

[lym953]

@ex-nerd Thanks for reporting. To help us understand the issue: does your Lambda use a tracing Layer such as datadog-lambda-js like arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Node22-x:129? If so, what's the version?

[ex-nerd]

Yes, I upgraded to 129 at the same time. I have since reverted the lambda extension to 65 but kept the lambda js at 129 and the errors are no longer happening.

[lym953]

Thanks for the info. I reproduced the issue. Will take further look next week.