THRIFT-6145: Validate Ruby Compact decoder bounds

[kpumuk]

Ruby’s pure and native CompactProtocol readers accept fifth-byte varint payload bits outside the uint32 range, and they produce different values for the same input. Binary length prefixes above the signed int32 size domain are also passed to the transport instead of being rejected by the protocol.

This change gives both readers the same boundary behavior. Invalid fifth-byte payload bits now raise ProtocolException::INVALID_DATA, while binary lengths above INT32_MAX raise ProtocolException::SIZE_LIMIT before read_all is called. Valid uint32 varints, signed i32 values, binary sizes through INT32_MAX, and ordinary Compact messages remain supported.

Benchmarks

I ran the repository’s small Compact read benchmark against current master (c2def39207a73394420088da9b4b105571dd9036) and this change in the same Ruby container:

ruby test/rb/benchmarks/protocol_benchmark.rb --json --small-runs 10000 --scenarios rb-cmp-read-small
THRIFT_BENCHMARK_SKIP_NATIVE=1 ruby test/rb/benchmarks/protocol_benchmark.rb --json --small-runs 10000 --scenarios rb-cmp-read-small

Each result is the median of 11 independent warmed runs.

Mode	Master median (range)	Proposed median (range)	Delta
Native extension	0.051075 s (0.049545–0.053581)	0.052138 s (0.050545–0.054560)	+2.08%
Pure Ruby	0.180099 s (0.173008–0.235398)	0.177465 s (0.174482–0.180574)	-1.46%

The ranges overlap. This focused small-structure read workload does not show a clear performance change.

• Did you create an Apache Jira ticket? (Request account here, not required for trivial changes)
• If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
• Did you squash your changes to a single commit? (not required, but preferred)
• Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
• If your change does not involve any code, include [skip ci] anywhere in the commit message to free up build resources.

[Copilot]

Pull request overview

Aligns Ruby CompactProtocol’s pure-Ruby and native (C extension) decoders on strict varint32 and binary-length bounds, so malformed inputs are rejected consistently and large binary lengths are rejected before any payload read is attempted.

Changes:

• Tightened read_varint32 in both pure Ruby and the C extension to reject 5th-byte payload bits that would overflow uint32 (while still allowing valid 5-byte uint32 varints).
• Added an early INT32_MAX bound check for Compact read_binary, raising ProtocolException::SIZE_LIMIT before calling read_all.
• Expanded Ruby specs to cover the new binary size-limit behavior and the new varint32 overflow condition.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
lib/rb/spec/compact_protocol_spec.rb	Renames shared varint32 size fixtures and adds coverage for binary size upper bounds and varint32 5th-byte overflow.
lib/rb/lib/thrift/protocol/compact_protocol.rb	Pure-Ruby decoder: adds binary size bound check and refines varint32 decoding to reject overflowing 5th-byte payload bits.
lib/rb/ext/compact_protocol.c	Native decoder: mirrors the pure-Ruby varint32 5th-byte overflow validation and adds the same binary size limit check.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.