v0.12.1

Null validity and real-world file compatibility are the two themes of this release. The
Raincloud conformance suite — 247 public datasets written
by the Python vortex-data bindings — exposed a systematic family of silent-corruption bugs where
nullable encoding children's validity masks were dropped during decode, causing null rows to appear
as invented values (0.0, the FoR base, empty string). All five affected encodings are patched:
fastlanes.bitpacked validity-child chain, vortex.dict (eager + layout), vortex.runend,
vortex.sparse (primitive + utf8/binary), vortex.datetimeparts, and fastlanes.alprd.
Real-world files also uncovered scan failures: mixed per-column chunk grids, nested struct layout,
FSST-compressed string-dict offsets, RLE double/float pools, narrow-integer dict pools, all-null
columns, and zone-map stats from the current Rust writer format (vortex.zoned). Unsigned integer
silent-corruption in CSV export and filter predicates is also fixed. The conformance suite itself
ships as a gate-running weekly workflow plus four per-PR JNI interop fixtures.

Fixed

• Per-zone stats from current Rust writers (vortex.zoned, vortex-jni 0.76.0) decode again. The 0.76 zoned layout replaced the legacy vortex.stats bit-set metadata with an aggregate-function spec list and dropped the per-stat truncation flags; the reader now reconstructs the stats table from that spec list (min/max/sum/null_count), so columnZoneStats and aggregate push-down work against those files instead of throwing ClassCastException. (#197)
• Scans of files whose columns use different chunk grids no longer fail with mixed per-column chunking beyond 1-vs-N is not supported. The scan planner now splits at the merged boundary grid — the sorted union of every column's chunk boundaries, matching the Rust reference — and decodes each column's covering chunk once, slicing it zero-copy to each window. This handles both nested grids (Raincloud emotions-dataset-for-nlp, where label's coarse chunks nest inside text's finer grid) and disjoint grids (uci-beijing-multi-site-air-quality, where numeric and station boundaries do not nest). Aligned N-vs-N and 1-vs-N scans keep their existing slice-free fast path. (#221)
• Hardened the vortex.zoned metadata decoder against malformed input: an attacker-controlled length varint could overflow its pos + len bounds checks and crash with NegativeArraySizeException/IndexOutOfBoundsException; the checks are now overflow-safe and unparseable metadata falls back to per-chunk stats. Decimal columns — whose Rust zone table keeps a sum field this reader cannot map — now also fall back instead of decoding a misaligned table. (#197)
• CSV export renders nested struct columns as JSON object cells ({"field":value,...}, strings JSON-escaped, null fields as JSON null, nested structs recursed) instead of throwing unsupported array type: StructArray. (#217)
• Scanning a struct whose columns include a nested struct no longer fails chunk planning. A nested vortex.struct layout column (e.g. Raincloud countries-of-the-world's data column) is now treated as a single full-range chunk source and decoded through the layout registry's new struct decoder into a StructArray, matching the Rust reference (a nested struct spans its parent's row range, not an independent chunking). schema/count/inspect already worked; plain scans and select of sibling columns now work too. CSV export of the struct column itself remains unsupported (a separate rendering limitation). (#207)
• CSV export renders unsigned integer columns (U8–U64) with their unsigned values — high-half values previously printed as two's-complement negatives (uci-wine magnesium U8 132 exported as -124), silent corruption found by the Raincloud conformance suite. (#208)
• Unsigned integer columns are handled unsigned in the remaining consumers: the CLI filter predicates (magnesium >= 130 now matches its high-half U8 rows instead of silently dropping them — the worst class of the bug: wrong query results), the TUI grid and inspector views, and the Calcite adapter (U8/U16/U32 map to the next wider signed SQL type so their full range fits; U64 stays BIGINT and fails loud rather than surfacing a negative for values ≥ 2^63, both when read and when summed). (#216)
• fastlanes.rle decodes F64/F32 value pools (LazyRleDoubleArray, LazyRleFloatArray) — files from the Python bindings RLE-encode double columns with long constant runs, which previously failed to scan with unsupported ptype F64. (#209)
• Lazy dict decode now covers I8/U8/I16/U16 value columns (DictByteArray, DictShortArray) — files from the Python bindings dict-encode narrow-integer columns, which previously failed to scan with unsupported ptype for lazy dict. (#206)
• CSV export handles all-null (DType.Null) columns as empty fields instead of throwing unsupported array type: NullArray. (#211)
• String-dict columns whose values are FSST-compressed no longer fail to scan with IndexOutOfBoundsException — the dictionary value offsets are now read at their true ptype width instead of a hardcoded 8-byte stride (uci-magic-gamma-telescope's class column decompressed to 4-byte offsets). (#215)
• Null rows no longer silently decode as values: wrapper decoders now propagate the row validity that files from the Python bindings carry deep in the encoding tree. Three representations were dropped — a trailing validity child on fastlanes.bitpacked (reached through vortex.alp/vortex.zigzag/fastlanes.for, which delegate validity to their encoded child per the Rust ValidityChild contract), dict pools with invalid slots, and dict codes with their own validity — across both the eager vortex.dict decoder and the lazy dict layout path. Found on real data: penguins and kepler exported invented values (32.1, 0.0) for thousands of null cells. (#210)
• vortex.runend propagates nullable run-values' validity: a null run now nulls every row it covers instead of expanding to a filler value (uci-online-retail customerid u16? nulls previously decoded as the FoR base). Row validity is a lazy run-end bool over the same run-ends, matching the Rust ValidityVTable<RunEnd>. (#225)
• vortex.sparse propagates nullability: a fill_value: null array nulls every unpatched position (world-energy-consumption biofuel_cons_change_pct f64? previously decoded them as 0.0), and a null patch value nulls its own position. Row validity is a lazy sparse bool whose fill is fill_value.is_valid() and whose patch bits are the patch values' validity, matching the Rust ValidityVTable<Sparse>. (#226)
• vortex.sparse over utf8/binary values now carries the same row validity as the primitive path: a fill_value: null string column nulls every unpatched position instead of rendering an empty string, and a nullable patch value nulls its own position instead of surfacing its raw bytes. The Rust ValidityVTable<Sparse> is generic over the values encoding, so VarBin reuses the identical sparse-bool validity — completing the #226 fix for string/binary columns. (#232)
• vortex.datetimeparts propagates null component rows instead of throwing DateTimeParts: null cell at index: when any part (days/seconds/subseconds) decodes to a null — as a nullable vortex.runend child now does after #225 — the reassembled timestamp row is null, unblocking scans of bi-yalelanguages and bi-euro2016. (#235)
• fastlanes.alprd now propagates left_parts validity: null float rows no longer decode as 0.0. (#234)

Added

• Integration test for null-fill Sparse and null-run RunEnd round-trip, running on every gate. (#233)
• Real-world conformance suite against the Raincloud corpus: 247 public datasets whose Vortex files are written by the Python bindings, each validated value-for-value against its Parquet sibling. scripts/hydrate-raincloud-corpus.sh hydrates any subset (cache → mirror → local build), RaincloudConformanceIntegrationTest tests whatever is hydrated against the checked-in per-dataset status matrix, and a weekly workflow runs a size-capped sweep. First triage found four reader gaps on real data, including one silent-corruption bug (unsigned integers rendered as signed). (#205)