Root cause
The Sparse encoding spec (encoding-format/dict-runend-sparse.md) states:
Tags 4-6 (chunk_offsets) are in-memory-only. A conformant reader rejects a node that carries them and requires exactly two children (patch_indices, patch_values).
SparseEncodingDecoder.java:56-59 reads only tags 1-3 (len, offset, indices_ptype) and silently ignores any extra tags. The decoder also never validates that exactly two children are present.
Impact
Hardening gap only — no wrong output from production Rust-written files (no production writer emits chunk_offsets or a 3rd child). A malformed or future-format file carrying these fields would be silently accepted rather than rejected with a VortexException.
Fix
- After decoding
PatchesMetadata, assert ctx.childCount() == 2 — throw VortexException if not.
- Consider validating that
patch_indices are strictly sorted (another spec invariant currently unchecked); unsorted indices would silently corrupt output.
Reference
Spec: https://github.com/vortex-data/vortex/blob/mp/spec/docs/specification/encoding-format/dict-runend-sparse.md
Root cause
The Sparse encoding spec (
encoding-format/dict-runend-sparse.md) states:SparseEncodingDecoder.java:56-59reads only tags 1-3 (len,offset,indices_ptype) and silently ignores any extra tags. The decoder also never validates that exactly two children are present.Impact
Hardening gap only — no wrong output from production Rust-written files (no production writer emits
chunk_offsetsor a 3rd child). A malformed or future-format file carrying these fields would be silently accepted rather than rejected with aVortexException.Fix
PatchesMetadata, assertctx.childCount() == 2— throwVortexExceptionif not.patch_indicesare strictly sorted (another spec invariant currently unchecked); unsorted indices would silently corrupt output.Reference
Spec:
https://github.com/vortex-data/vortex/blob/mp/spec/docs/specification/encoding-format/dict-runend-sparse.md