Skip to content

feat(encoding): adopt generic offsets in Lance 2.3 - #8046

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-7-offset-writerfrom
xuanwo/generic-block-stack-8-v23-offset-adopter
Draft

feat(encoding): adopt generic offsets in Lance 2.3#8046
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-7-offset-writerfrom
xuanwo/generic-block-stack-8-v23-offset-adopter

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 8 of the Generic Block Sequence Compression series.

Depends on #8045.

This is the first production adopter. Ordinary Lance 2.3 variable-width mini-block pages may select the generic offset container when its complete serialized wire size is strictly smaller. Lance 2.0 through 2.2 retain legacy interleaved Flat offsets byte-for-byte; explicit outer general compression retains legacy offsets; SparseLayout remains on its existing wire.

Page-level coverage verifies Range, Delta, Delta(Range), legacy fallback, unchanged chunk-local take request shape, and the SparseLayout exclusion. The format reference documents the two unambiguous mini-block containers and the bounded block-sequence contract.

Validation:

  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • cargo test -p lance-encoding test_v2_3_
  • cargo test -p lance-encoding test_variable_offset_codec_is_version_gated
  • cargo clippy --all --tests --benches -- -D warnings
  • cargo fmt --all
  • make -C docs build

The docs build succeeded with existing missing-external-doc-repository and link warnings; its generated tracked-file side effects were restored before commit.

Stack navigation

@github-actions

Copy link
Copy Markdown
Contributor

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

  • Start a vote following the Lance community voting process.
    Format specification modifications need 3 binding +1 votes (excluding the
    proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
  • Once the vote passes, link the completed vote in this PR. It should not be
    merged until the vote is linked.

@github-actions github-actions Bot added A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request labels Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant