Skip to content

Remove the old v2 benchmarks website#8805

Merged
connortsui20 merged 1 commit into
developfrom
ct/nuke-v2-bench-website
Jul 16, 2026
Merged

Remove the old v2 benchmarks website#8805
connortsui20 merged 1 commit into
developfrom
ct/nuke-v2-bench-website

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Rationale for this change

The live benchmarks website is now located at https://github.com/vortex-data/benchmarks-website and it has been going strong for the past few weeks, so I think it is time to remove this.

What changes are included in this PR?

Deletes the old website code from here.

What APIs are changed? Are there any user-facing changes?

N/A

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 requested review from a10y and lwwmanning July 16, 2026 20:06
@connortsui20 connortsui20 added the changelog/chore A trivial change label Jul 16, 2026
@connortsui20
connortsui20 requested a review from robert3005 July 16, 2026 20:07
@connortsui20
connortsui20 enabled auto-merge (squash) July 16, 2026 20:08
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 49b2b3f 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +2.2%
Engines: DataFusion Likely regression (+12.5%, medium confidence) · DuckDB No clear signal (-7.1%, low confidence)
Vortex (geomean): 0.946x ➖
Parquet (geomean): 0.935x ➖
Shifts: Parquet (control) -6.5% · Median polish -5.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (0.998x ➖, 0↑ 0↓)
name PR 49b2b3f (ns) base 17f843a (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 9748857 9877069 0.99
vortex_q01/datafusion:vortex-file-compressed 6405207 6345790 1.01
datafusion / parquet (0.887x ✅, 1↑ 0↓)
name PR 49b2b3f (ns) base 17f843a (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 19895297 21920309 0.91
vortex_q01/datafusion:parquet 🚀 4454162 5135545 0.87
duckdb / vortex-file-compressed (0.916x ➖, 1↑ 0↓)
name PR 49b2b3f (ns) base 17f843a (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 9951652 10602435 0.94
vortex_q01/duckdb:vortex-file-compressed 🚀 5863516 6559583 0.89
duckdb / parquet (0.986x ➖, 0↑ 0↓)
name PR 49b2b3f (ns) base 17f843a (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23345295 23596214 0.99
vortex_q01/duckdb:parquet 9376827 9543018 0.98

No file size changes detected.

@codspeed-hq

codspeed-hq Bot commented Jul 16, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.55%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
✅ 1710 untouched benchmarks
⏩ 10 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation compress[(100000, 4)] 1.8 ms 1.6 ms +11.93%
Simulation true_count_vortex_buffer[128] 580.6 ns 522.2 ns +11.17%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/nuke-v2-bench-website (49b2b3f) with develop (17f843a)

Open in CodSpeed

Footnotes

  1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@connortsui20
connortsui20 merged commit 3975895 into develop Jul 16, 2026
84 of 85 checks passed
@connortsui20
connortsui20 deleted the ct/nuke-v2-bench-website branch July 16, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants