Skip to content

userver: reformat (just for update results)#964

Merged
MDA2AV merged 2 commits into
MDA2AV:mainfrom
botanegg:userver-update-results
Jul 3, 2026
Merged

userver: reformat (just for update results)#964
MDA2AV merged 2 commits into
MDA2AV:mainfrom
botanegg:userver-update-results

Conversation

@botanegg

@botanegg botanegg commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Description

It was previously incorrect (due to overridden by json-tls test)


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@botanegg

botanegg commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f userver --save

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: userver | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,026,922 6555.9% 221MiB +70.4% +11.1%
baseline 4096 1,019,661 6363.5% 441MiB +4.2% -0.9%
pipelined 512 1,443,633 6717.2% 265MiB -0.5% +1.5%
pipelined 4096 1,324,629 6703.8% 899MiB -1.2% +0.1%
limited-conn 512 828,453 6203.2% 253MiB +18.1% +5.4%
limited-conn 4096 872,918 6390.8% 585MiB +2.7% +1.2%
json 4096 659,597 6387.2% 559MiB +214.8% -3.6%
json-comp 512 300,206 6546.7% 257MiB +188.0% +2.4%
json-comp 4096 298,148 6523.9% 604MiB +177.0% +10.6%
json-comp 16384 298,253 6525.1% 1.6GiB +178.7% +14.3%
json-tls 4096 554,073 6557.0% 739MiB ~0% +10.3%
upload 32 701 3272.2% 1.1GiB +0.3% ~0%
upload 256 634 6595.3% 3.6GiB +1.8% -7.7%
api-4 256 37,086 403.3% 206MiB +71.0% +7.3%
api-16 1024 123,403 1781.4% 393MiB +132.3% +24.4%
static 1024 162,732 6596.2% 445MiB -1.6% +1.6%
static 4096 187,129 6577.4% 946MiB -0.7% +1.6%
static 6800 195,167 6558.7% 1.3GiB +0.5% ~0%
async-db 1024 241,632 5700.2% 308MiB +1.2% +2.3%
Full log
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    21.95ms   10.23ms  92.67ms   68.41%
    Req/Sec     2.95k   350.91     9.88k    97.70%
  954396 requests in 5.10s, 56.69GB read
Requests/sec: 187129.79
Transfer/sec:     11.12GB
[info] CPU 6577.4% | Mem 946MiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    22.06ms   10.29ms 108.70ms   68.46%
    Req/Sec     2.93k   371.03    15.17k    97.36%
  950639 requests in 5.10s, 56.47GB read
Requests/sec: 186390.70
Transfer/sec:     11.07GB
[info] CPU 6350.1% | Mem 976MiB

=== Best: 187129 req/s (CPU: 6577.4%, Mem: 946MiB) ===
[info] saved results/static/4096/userver.json
httparena-bench-userver
httparena-bench-userver

==============================================
=== userver / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    35.69ms   19.47ms 226.17ms   72.44%
    Req/Sec     3.05k   316.71     8.36k    96.07%
  989313 requests in 5.10s, 58.77GB read
Requests/sec: 193967.96
Transfer/sec:     11.52GB
[info] CPU 6405.8% | Mem 1.3GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    35.31ms   18.68ms 194.66ms   71.24%
    Req/Sec     3.07k   412.23    13.67k    97.14%
  995496 requests in 5.10s, 59.14GB read
Requests/sec: 195167.21
Transfer/sec:     11.59GB
[info] CPU 6558.7% | Mem 1.3GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    35.54ms   19.08ms 215.76ms   72.38%
    Req/Sec     3.05k   373.14    13.02k    96.75%
  990631 requests in 5.10s, 58.85GB read
Requests/sec: 194220.12
Transfer/sec:     11.54GB
[info] CPU 6418.1% | Mem 1.3GiB

=== Best: 195167 req/s (CPU: 6558.7%, Mem: 1.3GiB) ===
[info] saved results/static/6800/userver.json
httparena-bench-userver
httparena-bench-userver

==============================================
=== userver / async-db / 1024c (tool=gcannon) ===
==============================================
[info] resetting postgres for a clean per-profile baseline
[info] starting postgres sidecar
httparena-postgres
[info] postgres ready (seeded)
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.84ms   3.32ms   6.52ms   11.40ms   17.10ms

  2316576 requests in 10.00s, 2316578 responses
  Throughput: 231.60K req/s
  Bandwidth:  892.81MB/s
  Status codes: 2xx=2316578, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2316576 / 2316578 responses (100.0%)
  Reconnects: 92265
  Per-template: 461977,463450,464676,464410,462063
  Per-template-ok: 461977,463450,464676,464410,462063
[info] CPU 5256.3% | Mem 307MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.61ms   3.11ms   6.15ms   11.00ms   16.50ms

  2413659 requests in 10.00s, 2413661 responses
  Throughput: 241.29K req/s
  Bandwidth:  930.46MB/s
  Status codes: 2xx=2413661, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2413655 / 2413661 responses (100.0%)
  Reconnects: 96647
  Per-template: 481194,482550,484031,484151,481729
  Per-template-ok: 481194,482550,484031,484151,481729
[info] CPU 5696.9% | Mem 304MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.60ms   3.10ms   6.13ms   10.90ms   16.20ms

  2416326 requests in 10.00s, 2416328 responses
  Throughput: 241.57K req/s
  Bandwidth:  931.57MB/s
  Status codes: 2xx=2416328, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2416324 / 2416328 responses (100.0%)
  Reconnects: 96635
  Per-template: 481823,483053,484402,484631,482415
  Per-template-ok: 481823,483053,484402,484631,482415
[info] CPU 5700.2% | Mem 308MiB

=== Best: 241632 req/s (CPU: 5700.2%, Mem: 308MiB) ===
[info] input BW: 16.13MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/userver.json
httparena-bench-userver
httparena-bench-userver
[info] skip: userver does not subscribe to crud
[info] skip: userver does not subscribe to fortunes
[info] skip: userver does not subscribe to baseline-h2
[info] skip: userver does not subscribe to static-h2
[info] skip: userver does not subscribe to baseline-h2c
[info] skip: userver does not subscribe to json-h2c
[info] skip: userver does not subscribe to baseline-h3
[info] skip: userver does not subscribe to static-h3
[info] skip: userver does not subscribe to gateway-64
[info] skip: userver does not subscribe to gateway-h3
[info] skip: userver does not subscribe to production-stack
[info] skip: userver does not subscribe to unary-grpc
[info] skip: userver does not subscribe to unary-grpc-tls
[info] skip: userver does not subscribe to stream-grpc
[info] skip: userver does not subscribe to stream-grpc-tls
[info] skip: userver does not subscribe to echo-ws
[info] skip: userver does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

@MDA2AV MDA2AV merged commit 3986c9e into MDA2AV:main Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants