Skip to content

Fix AMQP Object message decompression - #2256

Merged
cshannon merged 2 commits into
apache:mainfrom
cshannon:fix-amqp-object-decompression
Jul 28, 2026
Merged

Fix AMQP Object message decompression#2256
cshannon merged 2 commits into
apache:mainfrom
cshannon:fix-amqp-object-decompression

Conversation

@cshannon

Copy link
Copy Markdown
Contributor

This fixes the AmqpMessageSupport utility to correctly decompress the body of Object Message's in all cases.

Previously the decompression could stop early and not decompress the entire stream of data. This was due to the loop incorrectly casting the read int from the inflater stream as a byte before comparing to -1 to look for end of stream.

This is incorrect because the read() method can return a value between -1 and 255. Java uses Two's Complement to represent signed ints, so if the returned value int is 255 and is cast to a byte it becomes -1. This meant that reading 255 would return -1 leading to the code to exit thinking end of stream has been reached.

This was discovered when I as was looking into the "flaky" test error for #2238, JMSInteroperabilityTest.testOpenWireToQpidObjectMessageWithOpenWireCompression:493 » MessageFormat Failed to read object: null

Turns out, the test was not flaky but showed a real error. The test uses a random UUID, so by the test would intermittently fail depending on if the UUID decompression ran into a 255 int or not.

@cshannon cshannon self-assigned this Jul 27, 2026
@cshannon
cshannon requested a review from mattrpav July 27, 2026 22:27
This fixes the AmqpMessageSupport utility to correctly decompress the
body of Object Message's in all cases.

Previously the decompression could stop early and not decompress the entire stream of
data. This was due to the loop incorrectly casting the read int from the inflater
stream as a byte before comparing to -1 to look for end of stream.

This is incorrect because the read() method can return a value between -1 and 255.
Java uses Two's Complement to represent signed ints, so if the returned value int
is 255 and is cast to a byte it becomes -1. This meant that reading 255 would return -1
leading to the code to exit thinking end of stream has been reached.
@cshannon
cshannon requested a review from tabish121 July 27, 2026 22:43

@tabish121 tabish121 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cshannon
cshannon merged commit 2d60a85 into apache:main Jul 28, 2026
28 of 30 checks passed
@cshannon
cshannon deleted the fix-amqp-object-decompression branch July 28, 2026 14:26
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Apache ActiveMQ v6.4.0 Jul 28, 2026
cshannon added a commit that referenced this pull request Jul 28, 2026
This fixes the AmqpMessageSupport utility to correctly decompress the
body of Object Message's in all cases.

Previously the decompression could stop early and not decompress the entire stream of
data. This was due to the loop incorrectly casting the read int from the inflater
stream as a byte before comparing to -1 to look for end of stream.

This is incorrect because the read() method can return a value between -1 and 255.
Java uses Two's Complement to represent signed ints, so if the returned value int
is 255 and is cast to a byte it becomes -1. This meant that reading 255 would return -1
leading to the code to exit thinking end of stream has been reached.

(cherry picked from commit 2d60a85)
cshannon added a commit that referenced this pull request Jul 29, 2026
This fixes the AmqpMessageSupport utility to correctly decompress the
body of Object Message's in all cases.

Previously the decompression could stop early and not decompress the entire stream of
data. This was due to the loop incorrectly casting the read int from the inflater
stream as a byte before comparing to -1 to look for end of stream.

This is incorrect because the read() method can return a value between -1 and 255.
Java uses Two's Complement to represent signed ints, so if the returned value int
is 255 and is cast to a byte it becomes -1. This meant that reading 255 would return -1
leading to the code to exit thinking end of stream has been reached.

(cherry picked from commit 2d60a85)
cshannon added a commit that referenced this pull request Jul 29, 2026
This fixes the AmqpMessageSupport utility to correctly decompress the
body of Object Message's in all cases.

Previously the decompression could stop early and not decompress the entire stream of
data. This was due to the loop incorrectly casting the read int from the inflater
stream as a byte before comparing to -1 to look for end of stream.

This is incorrect because the read() method can return a value between -1 and 255.
Java uses Two's Complement to represent signed ints, so if the returned value int
is 255 and is cast to a byte it becomes -1. This meant that reading 255 would return -1
leading to the code to exit thinking end of stream has been reached.

(cherry picked from commit 2d60a85)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants