Skip to content

Commit 58b9c08

Browse files
Schnoukiclaude
andauthored
feat: Update SQS message size limit from 256 KB to 1 MB (#136)
* Fix typo in documentation * feat: Update SQS message size limit from 256 KB to 1 MB AWS increased the SQS message size limit from 256 KB to 1 MB on Aug 4, 2025. This commit updates our code to reflect this change: - MAX_MESSAGE_LENGTH constant updated from 262144 to 1048576 bytes - Documentation updated to reference 1 MB instead of 256 KB - Tests adjusted to work with the new limit - Added xfail for localstack tests as ElasticMQ still uses the old limit 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 161d0fa commit 58b9c08

File tree

3 files changed

+14
-11
lines changed

3 files changed

+14
-11
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ When batching is enabled:
108108

109109
- tasks are added to SQS by batches of 10, reducing the number of AWS operations
110110
- it is not possible to get the task `MessageId`, as it is not known until the batch is sent
111-
- care has to be taken about message size, as SQS limits both the individual message size and the maximum total payload size to 256 kB.
111+
- care has to be taken about message size, as SQS limits both the individual message size and the maximum total payload size to 1 MB.
112112

113113
## Batch Reads
114114

@@ -507,7 +507,7 @@ Localstack tests should perform faster than testing against AWS, and besides, th
507507
Run [ElasticMQ](https://github.com/softwaremill/elasticmq) and make sure that the SQS endpoint is available by the address localhost:4566:
508508

509509
```bash
510-
uv run -p 4566:9324 --rm -it softwaremill/elasticmq-native
510+
docker run -p 4566:9324 --rm -it softwaremill/elasticmq-native
511511
```
512512

513513
Then run

sqs_workers/queue.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828

2929
DEFAULT_MESSAGE_GROUP_ID = "default"
3030
SEND_BATCH_SIZE = 10
31-
MAX_MESSAGE_LENGTH = 262144 # 256 KiB
31+
MAX_MESSAGE_LENGTH = 1048576 # 1 MiB (since Aug 4, 2025)
3232

3333
if TYPE_CHECKING:
3434
from collections.abc import Generator, Iterable

tests/test_sqs.py

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
import time
55

66
import botocore
7+
import localstack_client.session
78
import pytest
89

910
from sqs_workers import (
@@ -157,17 +158,19 @@ def test_batch_should_keep_messages_until_overflow(sqs, queue_name):
157158
assert len(queue.get_raw_messages(0)) == 1
158159

159160

160-
def test_batch_flush_on_large_messages(sqs, queue_name):
161+
def test_batch_flush_on_large_messages(sqs_session, sqs, queue_name):
162+
if isinstance(sqs_session, localstack_client.session.Session):
163+
pytest.xfail("ElasticMQ still has the old 256 KiB message limit")
164+
161165
queue = sqs.queue(queue_name)
162166
say_hello_task = queue.connect_processor("say_hello", say_hello)
163167

164-
# 256KiB is our message limit
168+
# 1 MiB is our message limit
165169
with say_hello_task.batch():
166170
# no messages after 9 tasks
167171
for n in range(9):
168-
# each message is approx 32427 Bytes
169-
say_hello_task.delay(username=f"Homer {n} 🍩" * 1_000_000)
170-
# first 9 items is ~283651 Bytes so flush is triggered
172+
say_hello_task.delay(username=f"Homer {n} 🍩" * 4_000_000)
173+
# first 9 items is ~1.1 MiB so flush is triggered
171174
# and we walk back 1 item
172175
assert len(queue.get_raw_messages(0)) == 8
173176

@@ -182,11 +185,11 @@ def test_batch_fails_on_a_giant_message(sqs_session, sqs, queue_name):
182185
queue = sqs.queue(queue_name)
183186
say_hello_task = queue.connect_processor("say_hello", say_hello)
184187

185-
# 262144 Bytes is our message limit
188+
# 1 MiB is our message limit
186189
with say_hello_task.batch():
187190
with pytest.raises(botocore.exceptions.ClientError) as excinfo:
188-
# message ~264087 bytes long
189-
say_hello_task.delay(username="Homer 🍩" * 10_150_000)
191+
# message exceeds 1 MiB limit
192+
say_hello_task.delay(username="Homer 🍩" * 40_600_000)
190193
assert "MessageTooLong" in str(excinfo.value)
191194

192195

0 commit comments

Comments
 (0)