fix(puffin): return DataInvalid instead of panicking on malformed footer length#2867
fix(puffin): return DataInvalid instead of panicking on malformed footer length#2867arpitjain099 wants to merge 1 commit into
Conversation
…ter length Reading the footer of a Puffin file subtracts lengths from the file size without checking that the file is large enough. Two cases underflow: - read_footer_payload_length subtracts FOOTER_STRUCT_LENGTH from the file size, so any file shorter than 12 bytes underflows. - read_footer_bytes subtracts footer_length, which is derived from the footer_payload_length u32 read out of the file itself, so a declared payload length larger than the file underflows. Both panic with subtract-with-overflow in debug builds and wrap to a bogus read range in release builds. Add a minimum file length check and use checked_sub, returning ErrorKind::DataInvalid like the neighbouring magic and bounds checks already do. Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
viirya
left a comment
There was a problem hiding this comment.
Reproduced both failure modes by reverting the guards locally with the new tests kept: test_file_shorter_than_minimum_length_returns_error and test_footer_payload_length_larger_than_file_returns_error both die with attempt to subtract with overflow at exactly the two subtractions described, and pass with the guards in place. All 39 puffin tests pass on current main with the change applied.
The MIN_FILE_LENGTH arithmetic checks out against the Puffin layout: header magic (4) + footer magic (4) + empty payload (0) + payload-size (4) + flags (4) + trailing magic (4) = 20 = MAGIC_LENGTH * 2 + FOOTER_STRUCT_LENGTH. And the framing is right — this is robustness, matching the existing DataInvalid style of the magic/flags checks in the same file, not a security claim. Leaving read_with_prefetch alone is correct too, since its own guards keep its slicing in range and the fallback lands in the now-checked read.
One tiny behavioral note, fine as-is: the length check now runs before the magic check, so a short file with garbage magic reports "too short" rather than "bad magic" — arguably the more informative of the two errors anyway. LGTM.
What changes are included in this PR?
FileMetadata::readlocates the footer by subtracting from the file size, but never checks the file is big enough first. Two subtractions can underflow:read_footer_payload_lengthdoesinput_file_length - FOOTER_STRUCT_LENGTH, so any file shorter than 12 bytes underflows.read_footer_bytesdoesinput_file_length - footer_length, wherefooter_lengthis derived fromfooter_payload_length, the u32 read out of the FooterPayloadSize field of the file being parsed. A file that declares a payload larger than itself underflows here.In debug builds both panic with "attempt to subtract with overflow". In release the subtraction wraps and you get a bogus read range instead.
Worth saying up front: I'm treating this as robustness, not a security issue. Puffin files are table statistics written by the engines that already write the table, so I'm not claiming a meaningful trust boundary. The point is narrower - a truncated or corrupt file should surface as a
DataInvaliderror, the way the magic checks and the bounds checks indecode_flags/extract_footer_payload_as_stralready do in this same file, rather than panicking on the caller.The fix adds a
MIN_FILE_LENGTHcheck inreadand switches the second subtraction tochecked_sub, both returningErrorKind::DataInvalid.I left
read_with_prefetchalone. Itsprefetch_hint > 16andprefetch_hint <= input_file_lengthguards already keep its own slicing in range, and when the declared footer is larger than the hint it falls back toread, which is now checked.Are these changes tested?
Two unit tests, one per case. Both panic before the fix:
After the fix,
cargo test -p iceberg --lib puffingives 39 passed, 0 failed. Full cratecargo test -p iceberg --libis 1444 passed, 0 failed, andcargo clippy -p iceberg --all-targetsis clean.