Skip to content

feat(encoding): decode generic mini-block offsets - #8042

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-3-range-deltafrom
xuanwo/generic-block-stack-4-offset-reader
Draft

feat(encoding): decode generic mini-block offsets#8042
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-3-range-deltafrom
xuanwo/generic-block-stack-4-offset-reader

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 4 of the generic block compression series. Depends on #8041.

Variable-width mini-block decoding now distinguishes the existing interleaved Flat layout from a generic offset descriptor. The generic path consumes the descriptor-derived zero/one payload plus the values buffer and validates cardinality, zero origin, monotonicity, Arrow signed bounds, and the final value length. The legacy path keeps the existing wire shape while converting malformed buffer panics into contextual errors.

This is reader-only. Production writers still emit the legacy layout for every file version.

Validation:

  • cargo fmt --all
  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • focused generic Range/Delta and malformed legacy reader tests
  • cargo test -p lance-encoding compression::tests
  • cargo test -p lance-encoding encodings::physical
  • cargo clippy --all --tests --benches -- -D warnings

Stack navigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant