Skip to content

perf(encoding): benchmark generic offsets - #8048

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-8-v23-offset-adopterfrom
xuanwo/generic-block-stack-9-benchmarks
Draft

perf(encoding): benchmark generic offsets#8048
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-8-v23-offset-adopterfrom
xuanwo/generic-block-stack-9-benchmarks

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 9 of the Generic Block Sequence Compression series.

Depends on #8046.

This layer adds representative Range and Delta corpora for encoder, full-scan decoder, and cold/warm random-take benchmarks. Each case compares Lance 2.2 legacy offsets with Lance 2.3 generic offsets and includes encoded bytes in the benchmark identifier.

Local release-with-debug results (10 samples, 1 second measurement) showed:

  • Range bytes: 5,283,840B -> 4,218,880B (-20.2%)
  • Delta bytes: 10,436,608B -> 9,355,264B (-10.4%)
  • Range take median: cold 482us -> 400us; warm 491us -> 394us
  • Delta scan/take remained in the same local range; Range full scan was slower because metadata-only offsets are materialized during decode

These short local measurements are representative smoke benchmarks, not a performance acceptance threshold.

Validation:

  • cargo check -p lance-encoding --benches
  • cargo bench -p lance-encoding --bench encoder --profile release-with-debug -- variable_offsets --sample-size 10 --measurement-time 1 --warm-up-time 1 --noplot
  • cargo bench -p lance-encoding --bench decoder --profile release-with-debug -- variable_offsets_decode --sample-size 10 --measurement-time 1 --warm-up-time 1 --noplot
  • cargo test -p lance-encoding (575 passed, 5 ignored; integration test passed; doctest ignored)
  • cargo clippy --all --tests --benches -- -D warnings
  • cargo fmt --all

Stack navigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant