Skip to content

feat(encoding): select bounded offset codecs - #8044

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-5-block-dictionaryfrom
xuanwo/generic-block-stack-6-offset-selector
Draft

feat(encoding): select bounded offset codecs#8044
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-5-block-dictionaryfrom
xuanwo/generic-block-stack-6-offset-selector

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 6 of the Generic Block Sequence Compression series.

Depends on #8043.

This layer adds the adopter-private bounded offset statistics, candidate planning, and complete wire-cost comparison used to choose a reusable concrete block compressor for independent u32/u64 offset chunks. Candidates cover Constant, Range, Delta, Flat/bitpacking, RLE, Dictionary, and configured general compression. Losing candidates do not materialize payloads.

The selector remains test-only in this layer, so production mini-block writers and persisted bytes are unchanged. The next stack layer enables it only for eligible Lance 2.3 variable-width mini-block offsets.

Validation:

  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • cargo test -p lance-encoding compression::tests
  • cargo test -p lance-encoding encodings::physical
  • cargo clippy --all --tests --benches -- -D warnings
  • cargo fmt --all

Stack navigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant