Corrupted message body received by consumer using RabbitMQ.Client 7.1.2 under high load (C#/.NET Framework 4.8)

[sintesfabien]

Describe the bug

When consuming messages from RabbitMQ using RabbitMQ.Client 7.1.2 (.NET Framework 4.8, C# 7.3), the consumer occasionally receives a corrupted message body under high load.
The message body (ea.Body.ToArray()) contains unexpected binary data and the JSON payload is truncated or malformed, causing deserialization to fail.
However, when the same message is rejected and routed to a dead-letter queue (DLQ), the message in the DLQ is perfectly valid JSON and matches what was originally published.

Reproduction steps

1. Use RabbitMQ.Client 7.1.2 with .NET Framework 4.8.
2. Publish a large number of JSON messages (UTF-8 encoded) to a queue.
3. Start multiple consumers (each with its own channel and AsyncEventingBasicConsumer) consuming from the same queue.
4. In the consumer, read the message body with ea.Body.ToArray() and decode with Encoding.UTF8.GetString(body).
5. Under high load, observe that some messages cannot be deserialized due to unexpected characters or truncation.
6. Reject the corrupted message so it is routed to a dead-letter queue.
7. Retrieve the message from the DLQ and observe that it is valid JSON and not corrupted.

Expected behavior

The consumer should always receive the exact message body that was published, without any corruption or truncation.
The buffer returned by ea.Body.ToArray() should match the original published message.

Additional context

Additional context
• The producer always publishes valid JSON messages (UTF-8).
• The DLQ message is always correct, confirming the producer is not at fault.
• Each consumer uses its own channel (no channel sharing).
• The issue only appears under high load with multiple concurrent consumers.
• The problem is not systematic but occurs randomly during load tests.
• Example of corrupted message (base64, as received by the consumer):
AQAHAAAATgA8ACgAACRlZGkuZXhlYy5jbHQuc2VuZC1leGNoYW5nZS03MDQ5LW5xNGghZWRpLmV4ZWMuY2x0LnNlbmQtcXVldWUtNzA0OS1ucTRoAc4CAAcAAAAgADwAAAAAAAAAAAFhkAAQYXBwbGljYXRpb24vanNvbgLOAwAHAAABYXsiQ29kZUNsaWVudCI6IjcwNDkiLCJDb2RlQ2hhbm5lbCI6Im5xNGgiLCJQYXRoIjoiQzpcXFVzZXJzXFxzaW50ZXNcXEFwcERhdGFcXExvY2FsXFxUZW1wXFxBbWlUcmFuc21pc3Npb25Vbml0VGVzdFxcTG9hZFRlc3RfNzVkNjMzZmYtMDc2Yy00MzJlLWIyY2ItM2QzNjM4YzJhYTMyXFxmaWxlXzE=
Decoded, this contains binary data and a truncated JSON string.
• Example of the same message in the DLQ (as seen in RabbitMQ admin):
{"CodeClient":"7049","CodeChannel":"nq4h","Path":"C:\Users\sintes\AppData\Local\Temp\AmiTransmissionUnitTest\LoadTest_75d633ff-076c-432e-b2cb-3d3638c2aa32\file_1c6b188e_1560.bin","Dir":"send","Type":"clt","Mode":"sftp","Date":"2025-06-09 17:55:05","Retry":0,"WatcherIdu":"","InteractiveIdu":""}
• The issue persists with the latest version (7.1.2).
• No channel/thread sharing, and the buffer is always copied with ToArray().
Is this a known issue? Are there any recommended workarounds or further diagnostics? Any help would be appreciated.

[sintesfabien]

Heavy load means 5000 messages (publish before consumers starts) and 10 consumers.

[Gsantomaggio]

@sintesfabien Thank you for reporting the issue. Can you please create a GitHub repo to reproduce the problem? It will be easier for us to understand.

[lukebakken]

@sintesfabien - it is unlikely that this library is the source of this error. Think about it - this library is the second most popular AMQP 0.9.1 library for RabbitMQ (after the Java one), and if such a basic issue were the fault of this library, we would have heard about it.

A much more likely explanation is your use of the library. We are willing to assist you, but only if you make it as easy as possible for us to help you. Right now, you are asking us to guess how to reproduce this issue, which is not something we will do for you.

We will need the following:

• Answers to:
  • Is this an application that operated normally for some time, and then started to experience this issue?
  • If so, how long did it operate normally?
  • When this issue started to arise, what changed?
• A complete description of your environment. Are you using a load balancer? What is your RabbitMQ version? Erlang version? What operating system and version is running RabbitMQ?
• What operating system and version is running your producing and consuming applications?
• We need a git repository with code to reproduce the issue. We should not have to spend any time getting your code to compile, run, or demonstrate the issue. Clear instructions are required.

[sintesfabien]

Hi,
The corrupted message issue was a related issue to another upstream issue. I'll open a separate thread for that.

[michaelklishin]

@sintesfabien I don't know what an "upstream" means but we won't investigate anything for which you don't have an executable way to reproduce. We do not guess in this community, and you are neither a paying user nor a contributor, so you should be extra mindful about how much time you expect the maintainers to spend on these text descriptions of an alleged problem.

[michaelklishin]

@sintesfabien are you aware of the Consumer Memory Safety section in the docs?

consumer interface implementations must deserialize or copy delivery payload before delivery handler > method returns. Retaining a reference to the payload is not safe: the memory allocated for it can be > deallocated at any moment after the handler returns.

Back when 7.0.0 has just shipped, it was a very common issue to see, and many users who claimed "message corruption" were not doing what the client expected them to do.

[lukebakken]

This issue is due to incorrect use of this library - #1847 (comment)