Skip to content

feat(encoding): encode generic mini-block offsets - #8045

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-6-offset-selectorfrom
xuanwo/generic-block-stack-7-offset-writer
Draft

feat(encoding): encode generic mini-block offsets#8045
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-6-offset-selectorfrom
xuanwo/generic-block-stack-7-offset-writer

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 7 of the Generic Block Sequence Compression series.

Depends on #8044.

This layer adds the generic variable-width mini-block writer capability. It computes complete serialized wire cost before selecting a layout, fixes one reusable concrete offset compressor for the page, and writes metadata-only or one-payload offset codecs without changing chunk-local read granularity. It also promotes the concrete block compressors needed by the production writer out of test-only compilation.

The default compression strategy does not select this path yet, so persisted writer behavior remains unchanged in this layer. The next stack layer is the Lance 2.3 adopter and version gate.

Validation:

  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • cargo test -p lance-encoding encodings::physical::binary::tests
  • cargo test -p lance-encoding encodings::logical::primitive::miniblock::tests
  • cargo clippy --all --tests --benches -- -D warnings
  • cargo fmt --all

Stack navigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant