Performance
Size of PDF: 7.3kb
Average Lambda duration went from ~2624ms at the beginning (1 PDF to merge) to ~7826 ms (100 PDFs to merge) to the end. It’s interesting seeing how the lambda warms up; the first run (1 PDF to merge) took 4242ms, but the second run (2 PDFs to merge) took 1158ms.
Various parts of the process were logged in CloudWatch:
- # of files to merge: 1
- save pdfs to tmp directory: 702ms
- get file names of files in tmp directory: 1ms
- merge pdfs: 979ms
- delete files in tmp folder after merging pdfs: 1ms
- upload merged pdf to S3: 1ms
- total lambda runtime: 4242ms
- # of files to merge: 2
- save pdfs to tmp directory: 175ms
- get file names of files in tmp directory**: 1ms
- merge pdfs: 825ms
- delete files in tmp folder after merging pdfs**: 0ms
- **upload merged pdf to S3: 0ms
- total lambda runtime: 1158ms
- # of files to merge: 25
- save pdfs to tmp directory: 1562ms
- get file names of files in tmp directory: 57ms
- merge pdfs: 1780ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 3896ms
- # of files to merge: 50
- save pdfs to tmp directory: 3296ms
- get file names of files in tmp directory: 14ms
- merge pdfs: 3000ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 6488ms
- # of files to merge: 75
- save pdfs to tmp directory: 3986ms
- get file names of files in tmp directory: 1ms
- merge pdfs: 3829ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 8193ms
- # of files to merge: 100
- max memory used: 118 MB
- save pdfs to tmp directory: 4910ms
- get file names of files in tmp directory: 58ms
- merge pdfs: 5179ms
- delete files in tmp folder after merging pdfs: 17ms
- upload merged pdf to S3: 17ms
- total lambda runtime: 10407ms
This run was performed using a larger PDF than in the 1st run. Size of PDF: 18.7kb
Average Lambda duration went from ~4220ms at the beginning (1 set of PDFs to merge) to ~11600ms (100 set of PDFS to merge).
- # of files to merge: 1
- save pdfs to tmp directory: 597ms
- get file names of files in tmp directory: 2ms
- merge pdfs: 979ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 3759ms
- # of files to merge: 2
- save pdfs to tmp directory: 136ms
- get file names of files in tmp directory: 19ms
- merge pdfs: 900ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 1204ms
- # of files to merge: 25
- save pdfs to tmp directory: 1421ms
- get file names of files in tmp directory: 1ms
- merge pdfs: 2617ms
- delete files in tmp folder after merging pdfs: 0ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 4219ms
- # of files to merge: 50
- save pdfs to tmp directory: 2960ms
- get file names of files in tmp directory: 0ms
- merge pdfs: 4180ms
- delete files in tmp folder after merging pdfs: 20ms
- upload merged pdf to S3: 1ms
- total lambda runtime: 7680ms
- # of files to merge: 75
- save pdfs to tmp directory: 3710ms
- get file names of files in tmp directory: 78ms
- merge pdfs: 5881ms
- delete files in tmp folder after merging pdfs: 12ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 10048ms
- # of files to merge: 100
- save pdfs to tmp directory: 5298ms
- get file names of files in tmp directory: 31ms
- merge pdfs: 7320ms
- delete files in tmp folder after merging pdfs: 1ms
- upload merged pdf to S3: 0ms
- total lambda runtime: 12920ms
22:46:37.773Z This run was performed using a larger PDF than in the 1st and 2nd runs. Size of PDF: 96.7kb
start: 17:09
- # of files to merge: 1
- save pdfs to tmp directory: 700ms
<!-- - get file names of files in tmp directory: 2ms -->
- merge pdfs: 1140ms
<!-- - delete files in tmp folder after merging pdfs: 0ms -->
- upload merged pdf to S3: 0ms
- total lambda runtime: 4175ms
- # of files to merge: 2
- save pdfs to tmp directory: 460ms
<!-- - get file names of files in tmp directory: 2ms -->
- merge pdfs: 1180ms
<!-- - delete files in tmp folder after merging pdfs: 0ms -->
- upload merged pdf to S3: 0ms
- total lambda runtime: 2045ms
- # of files to merge: 25
- save pdfs to tmp directory: 1893ms
<!-- - get file names of files in tmp directory: 2ms -->
- merge pdfs: 6100ms
<!-- - delete files in tmp folder after merging pdfs: 0ms -->
- upload merged pdf to S3: 0ms
- total lambda runtime: 8423ms
- # of files to merge: 50
- save pdfs to tmp directory: 3292ms
<!-- - get file names of files in tmp directory: 2ms -->
- merge pdfs: 11881ms
<!-- - delete files in tmp folder after merging pdfs: 0ms -->
- upload merged pdf to S3: 0ms
- total lambda runtime: 15816ms
Around the ~60 PDFs mark (5.8mb), the Lambda function started to time out, as execution time was capped at 20s. This execution time is set by the AWS dev/user. Most likely, if the function execution time wasn't capped, the Lambda function would have started failing because it would start exceeding the memory, which by default is/was set to 128mb. Memory and execution time can both be increased if there is a need to do so, though the cost to run Lambdas may also increase.
aws logs get-log-events --region us-east-1 --log-group-name /aws/lambda/mergePdfs --log-stream-name 2016/10/03/[\$LATEST]e00b77d0d762412dbe7ecdcf3d3da7e3 --output text>lambda.log