Skip to content

SparseEncodingDecoder: silently ignores chunk_offsets metadata; does not reject 3rd child #250

Description

@dfa1

Root cause

The Sparse encoding spec (encoding-format/dict-runend-sparse.md) states:

Tags 4-6 (chunk_offsets) are in-memory-only. A conformant reader rejects a node that carries them and requires exactly two children (patch_indices, patch_values).

SparseEncodingDecoder.java:56-59 reads only tags 1-3 (len, offset, indices_ptype) and silently ignores any extra tags. The decoder also never validates that exactly two children are present.

Impact

Hardening gap only — no wrong output from production Rust-written files (no production writer emits chunk_offsets or a 3rd child). A malformed or future-format file carrying these fields would be silently accepted rather than rejected with a VortexException.

Fix

  1. After decoding PatchesMetadata, assert ctx.childCount() == 2 — throw VortexException if not.
  2. Consider validating that patch_indices are strictly sorted (another spec invariant currently unchecked); unsorted indices would silently corrupt output.

Reference

Spec: https://github.com/vortex-data/vortex/blob/mp/spec/docs/specification/encoding-format/dict-runend-sparse.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions