Skip to content

Validate segment offsets when populating initial segments#8827

Open
robert3005 wants to merge 1 commit into
developfrom
rk/populatevalidate
Open

Validate segment offsets when populating initial segments#8827
robert3005 wants to merge 1 commit into
developfrom
rk/populatevalidate

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

This shouldn't panic and instead can error

Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 added the changelog/fix A bug fix label Jul 17, 2026
@robert3005
robert3005 enabled auto-merge (squash) July 17, 2026 16:02
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 21be72b 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +2.7%
Engines: DataFusion No clear signal (+8.0%, medium confidence) · DuckDB No clear signal (-2.3%, low confidence)
Vortex (geomean): 0.985x ➖
Parquet (geomean): 0.972x ➖
Shifts: Parquet (control) -2.8% · Median polish -1.7%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.024x ➖, 0↑ 0↓)
name PR 21be72b (ns) base 7cf53cb (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 9749998 9632925 1.01
vortex_q01/datafusion:vortex-file-compressed 6265785 6047747 1.04
datafusion / parquet (0.949x ➖, 0↑ 0↓)
name PR 21be72b (ns) base 7cf53cb (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 19875797 20532528 0.97
vortex_q01/datafusion:parquet 4486517 4826794 0.93
duckdb / vortex-file-compressed (0.972x ➖, 0↑ 0↓)
name PR 21be72b (ns) base 7cf53cb (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 9983688 10341086 0.97
vortex_q01/duckdb:vortex-file-compressed 6078197 6212619 0.98
duckdb / parquet (0.995x ➖, 0↑ 0↓)
name PR 21be72b (ns) base 7cf53cb (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23397273 23424912 1.00
vortex_q01/duckdb:parquet 9372319 9452056 0.99

No file size changes detected.

@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.17%

⚡ 1 improved benchmark
✅ 1669 untouched benchmarks
⏩ 53 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation true_count_vortex_buffer[128] 580.6 ns 522.2 ns +11.17%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/populatevalidate (21be72b) with develop (cc6803a)2

Open in CodSpeed

Footnotes

  1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (7cf53cb) during the generation of this report, so cc6803a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant