Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#### Features

* [#2914](https://github.com/ruby-grape/grape/pull/2914): Bench 4.0.0 in the version throughput benchmark - [@ericproulx](https://github.com/ericproulx).
* Your contribution here.

#### Fixes
Expand Down
3 changes: 2 additions & 1 deletion benchmark/version_throughput/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Cross-version throughput benchmark for Grape. Measures `BenchAPI.call(env)` requ
## Usage

```sh
# default: 3.0.1, 3.1.1, 3.2.1, 3.3.5, master
# default: 3.0.1, 3.1.1, 3.2.1, 3.3.5, 4.0.0, master
ruby benchmark/version_throughput/run.rb

# subset
Expand All @@ -41,6 +41,7 @@ YJIT columns are only emitted if the running Ruby was built with YJIT support (`
## Interpreting results

- **Noise floor is ~5-8%.** A single 5s window on macOS can easily move a few percent under thermal throttling or background load. Rerun before drawing conclusions on small deltas.
- **`master` right after a release benches the same code twice.** When nothing has landed on `master` since the newest benched release, the two rows exercise identical code and their spread is a direct read of that session's noise floor — not a regression or a win.
- **Run on a quiet machine.** Close other apps, plug in the laptop, don't touch the keyboard during the run. Each version takes ~14s of wall-clock measurement plus bundle install on first use.
- **`master` vs released gems is not apples-to-apples for code paths that changed.** If a refactor moved code between files, both numbers still measure the same `app.rb` request — that's the point — but interpret deltas as "end-to-end request cost" rather than per-method.
- **YJIT speedup is `(yjit_ips - no_yjit_ips) / no_yjit_ips`.** Both passes share the same Ruby binary; only the `--yjit` flag differs.
Expand Down
15 changes: 8 additions & 7 deletions benchmark/version_throughput/RESULTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Grape throughput by version

Generated: 2026-09-07 17:13:53 CEST
Generated: 2026-09-07 18:16:50 CEST
Ruby: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin25]
Host: Darwin 25.6.0 arm64
YJIT available: true
Expand All @@ -9,13 +9,14 @@ Single-threaded `Benchmark.ips`, 2s warmup + 5s measure, `BenchAPI.call(env)` ag

| Version | No-YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT speedup |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 3.0.1 | 34,512 | 28.98 | — | — | 52,487 | 19.05 | — | — | +52.1% |
| 3.1.1 | 45,656 | 21.90 | +32.3% | +32.3% | 85,284 | 11.73 | +62.5% | +62.5% | +86.8% |
| 3.2.1 | 47,440 | 21.08 | +3.9% | +37.5% | 90,745 | 11.02 | +6.4% | +72.9% | +91.3% |
| 3.3.5 | 67,834 | 14.74 | +43.0% | +96.6% | 137,675 | 7.26 | +51.7% | +162.3% | +103.0% |
| master | 120,877 | 8.27 | +78.2% | +250.2% | 230,153 | 4.34 | +67.2% | +338.5% | +90.4% |
| 3.0.1 | 34,255 | 29.19 | — | — | 54,997 | 18.18 | — | — | +60.6% |
| 3.1.1 | 43,711 | 22.88 | +27.6% | +27.6% | 84,743 | 11.80 | +54.1% | +54.1% | +93.9% |
| 3.2.1 | 45,470 | 21.99 | +4.0% | +32.7% | 89,697 | 11.15 | +5.8% | +63.1% | +97.3% |
| 3.3.5 | 67,602 | 14.79 | +48.7% | +97.3% | 135,813 | 7.36 | +51.4% | +146.9% | +100.9% |
| 4.0.0 | 123,107 | 8.12 | +82.1% | +259.4% | 233,841 | 4.28 | +72.2% | +325.2% | +89.9% |
| master | 122,134 | 8.19 | -0.8% | +256.5% | 235,134 | 4.25 | +0.6% | +327.5% | +92.5% |

Over time, 3.0.1 → master: **+250.2%** without YJIT, **+338.5%** with YJIT.
Over time, 3.0.1 → master: **+256.5%** without YJIT, **+327.5%** with YJIT.

## Notes
- All versions exercised through the same `BenchAPI` definition (kept stable in `app.rb`).
Expand Down
2 changes: 1 addition & 1 deletion benchmark/version_throughput/run.rb
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
HERE = __dir__
TMP = File.join(ROOT, 'tmp', 'bench-versions')

DEFAULT_VERSIONS = %w[3.0.1 3.1.1 3.2.1 3.3.5 master].freeze
DEFAULT_VERSIONS = %w[3.0.1 3.1.1 3.2.1 3.3.5 4.0.0 master].freeze
versions = (ENV['GRAPE_VERSIONS']&.split(',')&.map(&:strip) || DEFAULT_VERSIONS).freeze

def gemfile_for(version)
Expand Down
Loading