Client socket changes between aws-sdk v2 and v3

[camhart]

Describe the issue

We recently updated from aws-sdk v2 to v3. We did normal testing, and everything was good. But once we pushed to production, with a production load, aws-sdk v3 started to give the error below:

WARN @smithy/node-http-handler:WARN - socket usage at capacity=50 and 766 additional requests are enqueued.
See https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-maxsockets.html
or increase socketAcquisitionWarningTimeout=(millis) in the NodeHttpHandler config.

This should really be a documented change. I'm not sure what the difference is between v2 and v3, but I'd assume others are going to hit this problem.

I had to work with an AWS support engineer to resolve it, since the change isn't documented anywhere that I could find. See ticket #174544667100759.

In case other's come across this, I had to update my s3 and dynamodbdocumentclient to allow for more sockets. Default is 50. Keep alive should be enabled by default already

const s3 = new S3Client({
    requestHandler: {
        httpsAgent: { maxSockets: 400 }, //this is what fixed it
    },
});

const client = new DynamoDBClient({
    requestHandler: {
        httpsAgent: { maxSockets: 400 }, //this is what fixed it
    },
});

Links

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/dynamodb/
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/

[aBurmeseDev]

Hi @camhart - thanks for reaching out and your feedback on the documentation.

I'm glad you were able to resolve the issue and I couldn't find the internal ticket reference to the case number you shared, however, I can share this doc that explains requestHander and socket exhaustion. Another docs/developer guide you shared also explains the behavior around configuring maxSockets. Hope that helps!

[camhart]

I see the documentation. Perhaps I should have read more, however in my entire time going from v2 -> v3 I never happened across any of those documents. I'd recommend a big warning in the api documentation for each client construction indicating a breaking change for those migrating from v2 -> v3. And consider increasing the default max sockets to a value larger than 50. What was it in v2?