Skip to content

[python] FastPySGI: update version to v0.6 and enable JSON-TLS#605

Open
remittor wants to merge 3 commits intoMDA2AV:mainfrom
remittor-pr:fastpysgi-0.6
Open

[python] FastPySGI: update version to v0.6 and enable JSON-TLS#605
remittor wants to merge 3 commits intoMDA2AV:mainfrom
remittor-pr:fastpysgi-0.6

Conversation

@remittor
Copy link
Copy Markdown
Contributor

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.

@remittor remittor marked this pull request as draft April 22, 2026 18:10
@remittor remittor force-pushed the fastpysgi-0.6 branch 9 times, most recently from db77c89 to 838c4dc Compare April 23, 2026 13:30
@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented Apr 23, 2026

I think the verbose does not yet work on all tests, so you probably hit them.

I'll look it up later today

@remittor
Copy link
Copy Markdown
Contributor Author

I think the verbose does not yet work on all tests, so you probably hit them.

It looks like my latest changes introduced a bug into the ASGI mode. I don't know how to reproduce it yet. I'll have to think about it...

@remittor
Copy link
Copy Markdown
Contributor Author

/benchmark -f fastpysgi-wsgi --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

🚀 Benchmark run triggered for fastpysgi-wsgi (all tests) with --save. Results will be posted here when done.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: fastpysgi-wsgi | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,412,217 6822.8% 368MiB -0.4% +69.9%
baseline 4096 1,418,387 6451.9% 859MiB -3.6% +18.7%
pipelined 512 3,220,182 6601.2% 348MiB +11.3% +60.3%
pipelined 4096 2,695,695 6401.5% 848MiB -4.2% +18.0%
limited-conn 512 1,052,535 6144.6% 412MiB -1.4% +52.9%
limited-conn 4096 1,156,332 6665.3% 506MiB -3.9% +31.1%
json 4096 742,763 6353.9% 850MiB -3.3% +13.3%
json-comp 512 465,935 5937.5% 544MiB ~0% +18.2%
json-comp 4096 485,024 6285.7% 779MiB -2.4% +18.6%
json-comp 16384 480,463 6208.9% 958MiB -2.1% -28.0%
json-tls 4096 613,700 6446.0% 663MiB NEW NEW
upload 32 1,862 1754.2% 2.1GiB +12.4% +5.0%
upload 256 1,465 4937.7% 4.3GiB +10.7% +7.5%
api-4 256 19,815 315.1% 89MiB +0.8% +8.5%
api-16 1024 55,469 1273.8% 250MiB -0.3% +14.6%
static 1024 1,328,316 6527.7% 436MiB -1.4% +43.6%
static 4096 1,304,924 6238.8% 862MiB -1.9% +18.5%
static 6800 1,296,049 6373.6% 1.2GiB -1.9% +9.1%
async-db 1024 74,289 4486.0% 880MiB -6.2% +9.3%
Full log
[info] CPU 6238.8% | Mem 862MiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.18ms    2.73ms 163.39ms   97.70%
    Req/Sec    20.39k     1.78k   40.76k    88.75%
  6617902 requests in 5.10s, 101.32GB read
Requests/sec: 1297772.12
Transfer/sec:     19.87GB
[info] CPU 6502.0% | Mem 875MiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.24ms    3.37ms 235.10ms   98.57%
    Req/Sec    20.33k     1.81k   36.56k    86.90%
  6593786 requests in 5.10s, 100.95GB read
Requests/sec: 1292886.03
Transfer/sec:     19.79GB
[info] CPU 6159.0% | Mem 874MiB

=== Best: 1304924 req/s (CPU: 6238.8%, Mem: 862MiB) ===
[info] saved results/static/4096/fastpysgi-wsgi.json
httparena-bench-fastpysgi-wsgi
httparena-bench-fastpysgi-wsgi

==============================================
=== fastpysgi-wsgi / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.31ms    4.51ms 236.36ms   98.04%
    Req/Sec    20.39k     1.94k   38.38k    91.53%
  6611584 requests in 5.10s, 101.23GB read
Requests/sec: 1296049.00
Transfer/sec:     19.84GB
[info] CPU 6373.6% | Mem 1.2GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.13ms   12.31ms 446.07ms   98.97%
    Req/Sec    20.13k     2.42k   37.09k    93.43%
  6521023 requests in 5.10s, 99.84GB read
Requests/sec: 1278810.74
Transfer/sec:     19.58GB
[info] CPU 6478.4% | Mem 1.2GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.66ms    7.93ms 338.41ms   99.23%
    Req/Sec    20.26k     2.26k   40.58k    92.53%
  6562955 requests in 5.10s, 100.48GB read
Requests/sec: 1286709.10
Transfer/sec:     19.70GB
[info] CPU 6320.7% | Mem 1.2GiB

=== Best: 1296049 req/s (CPU: 6373.6%, Mem: 1.2GiB) ===
[info] saved results/static/6800/fastpysgi-wsgi.json
httparena-bench-fastpysgi-wsgi
httparena-bench-fastpysgi-wsgi

==============================================
=== fastpysgi-wsgi / async-db / 1024c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   13.53ms   11.40ms   20.40ms   74.60ms   371.90ms

  726996 requests in 10.00s, 726997 responses
  Throughput: 72.68K req/s
  Bandwidth:  280.93MB/s
  Status codes: 2xx=726997, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 726995 / 726997 responses (100.0%)
  Reconnects: 28832
  Per-template: 144844,145044,145785,145832,145490
  Per-template-ok: 144844,145044,145785,145832,145490
[info] CPU 4380.9% | Mem 879MiB

[run 2/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   13.52ms   11.00ms   20.30ms   70.60ms   499.90ms

  729507 requests in 10.00s, 729509 responses
  Throughput: 72.93K req/s
  Bandwidth:  281.83MB/s
  Status codes: 2xx=729509, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 729509 / 729509 responses (100.0%)
  Reconnects: 28975
  Per-template: 145532,145625,146102,146216,146034
  Per-template-ok: 145532,145625,146102,146216,146034
[info] CPU 4431.4% | Mem 883MiB

[run 3/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   13.31ms   10.40ms   19.90ms   69.00ms   470.50ms

  742893 requests in 10.00s, 742896 responses
  Throughput: 74.27K req/s
  Bandwidth:  286.85MB/s
  Status codes: 2xx=742896, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 742896 / 742896 responses (100.0%)
  Reconnects: 29547
  Per-template: 148174,148599,148680,149065,148378
  Per-template-ok: 148174,148599,148680,149065,148378
[info] CPU 4486.0% | Mem 880MiB

=== Best: 74289 req/s (CPU: 4486.0%, Mem: 880MiB) ===
[info] input BW: 4.96MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/fastpysgi-wsgi.json
httparena-bench-fastpysgi-wsgi
httparena-bench-fastpysgi-wsgi
[info] skip: fastpysgi-wsgi does not subscribe to crud
[info] skip: fastpysgi-wsgi does not subscribe to baseline-h2
[info] skip: fastpysgi-wsgi does not subscribe to static-h2
[info] skip: fastpysgi-wsgi does not subscribe to baseline-h2c
[info] skip: fastpysgi-wsgi does not subscribe to json-h2c
[info] skip: fastpysgi-wsgi does not subscribe to baseline-h3
[info] skip: fastpysgi-wsgi does not subscribe to static-h3
[info] skip: fastpysgi-wsgi does not subscribe to gateway-64
[info] skip: fastpysgi-wsgi does not subscribe to gateway-h3
[info] skip: fastpysgi-wsgi does not subscribe to production-stack
[info] skip: fastpysgi-wsgi does not subscribe to unary-grpc
[info] skip: fastpysgi-wsgi does not subscribe to unary-grpc-tls
[info] skip: fastpysgi-wsgi does not subscribe to stream-grpc
[info] skip: fastpysgi-wsgi does not subscribe to stream-grpc-tls
[info] skip: fastpysgi-wsgi does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@remittor remittor marked this pull request as ready for review April 23, 2026 14:24
@remittor
Copy link
Copy Markdown
Contributor Author

/benchmark -f fastpysgi-asgi --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: fastpysgi-asgi | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 782,845 6794.2% 293MiB +0.8% +5.8%
baseline 4096 984,376 6710.9% 801MiB +0.9% -0.9%
pipelined 512 1,261,650 6717.8% 269MiB -5.5% -2.5%
pipelined 4096 1,536,542 6682.6% 799MiB -0.3% ~0%
limited-conn 512 611,804 6201.0% 356MiB +0.5% -1.3%
limited-conn 4096 782,263 6484.5% 1.6GiB -0.9% ~0%
json 4096 583,452 6414.6% 2.2GiB -3.1% +37.5%
json-comp 512 357,223 6028.4% 411MiB +0.4% -2.5%
json-comp 4096 406,077 6393.4% 1.8GiB -1.8% +20.0%
json-comp 16384 394,548 6274.4% 5.3GiB +1.4% -3.6%
upload 32 1,865 1812.2% 2.2GiB +0.1% +10.0%
upload 256 1,572 4455.5% 4.5GiB -0.6% ~0%
api-4 256 29,543 396.8% 303MiB +14.3% -21.2%
api-16 1024 73,478 1586.1% 845MiB +6.7% +17.8%
static 1024 871,759 6667.3% 359MiB -1.6% +55.0%
static 4096 956,406 6588.5% 302MiB +0.3% -62.5%
static 6800 935,770 6363.9% 366MiB +0.5% -3.0%
async-db 1024 142,675 6288.9% 1.1GiB -0.9% ~0%
Full log
Transfer/sec:     14.64GB
[info] CPU 6588.5% | Mem 302MiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.26ms  705.79us  24.35ms   76.21%
    Req/Sec    15.02k   816.77    25.24k    96.81%
  4870703 requests in 5.10s, 74.58GB read
Requests/sec: 955027.63
Transfer/sec:     14.62GB
[info] CPU 6560.8% | Mem 798MiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.27ms  659.62us  22.76ms   77.26%
    Req/Sec    14.99k     1.10k   33.14k    97.42%
  4858806 requests in 5.10s, 74.40GB read
Requests/sec: 952601.15
Transfer/sec:     14.59GB
[info] CPU 6477.4% | Mem 804MiB

=== Best: 956406 req/s (CPU: 6588.5%, Mem: 302MiB) ===
[info] saved results/static/4096/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.23ms    1.16ms  35.92ms   88.99%
    Req/Sec    14.70k     1.25k   38.37k    96.20%
  4772761 requests in 5.10s, 73.08GB read
Requests/sec: 935770.17
Transfer/sec:     14.33GB
[info] CPU 6363.9% | Mem 366MiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.23ms    1.26ms  60.69ms   87.30%
    Req/Sec    14.70k     0.87k   27.97k    96.44%
  4762955 requests in 5.10s, 72.93GB read
Requests/sec: 933912.38
Transfer/sec:     14.30GB
[info] CPU 6541.8% | Mem 1.2GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.24ms    1.12ms  40.36ms   86.56%
    Req/Sec    14.67k     0.91k   32.19k    96.29%
  4754167 requests in 5.10s, 72.79GB read
Requests/sec: 932147.57
Transfer/sec:     14.27GB
[info] CPU 6408.8% | Mem 1.2GiB

=== Best: 935770 req/s (CPU: 6363.9%, Mem: 366MiB) ===
[info] saved results/static/6800/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / async-db / 1024c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.67ms   3.40ms   13.90ms   61.90ms   153.70ms

  1426756 requests in 10.00s, 1426758 responses
  Throughput: 142.64K req/s
  Bandwidth:  550.65MB/s
  Status codes: 2xx=1426758, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1426757 / 1426758 responses (100.0%)
  Reconnects: 56836
  Per-template: 284499,285409,286399,285798,284652
  Per-template-ok: 284499,285409,286399,285798,284652
[info] CPU 6288.9% | Mem 1.1GiB

[run 2/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.72ms   3.42ms   14.40ms   61.70ms   140.80ms

  1423094 requests in 10.00s, 1423097 responses
  Throughput: 142.28K req/s
  Bandwidth:  549.37MB/s
  Status codes: 2xx=1423097, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1423094 / 1423097 responses (100.0%)
  Reconnects: 56780
  Per-template: 283265,284616,286094,285378,283741
  Per-template-ok: 283265,284616,286094,285378,283741
[info] CPU 6322.5% | Mem 1.1GiB

[run 3/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.73ms   3.44ms   14.30ms   60.00ms   144.00ms

  1426333 requests in 10.00s, 1426335 responses
  Throughput: 142.60K req/s
  Bandwidth:  550.85MB/s
  Status codes: 2xx=1426335, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1426332 / 1426335 responses (100.0%)
  Reconnects: 56828
  Per-template: 283811,285148,286350,286436,284587
  Per-template-ok: 283811,285148,286350,286436,284587
[info] CPU 6348.3% | Mem 1.2GiB

=== Best: 142675 req/s (CPU: 6288.9%, Mem: 1.1GiB) ===
[info] input BW: 9.52MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi
[info] skip: fastpysgi-asgi does not subscribe to crud
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2
[info] skip: fastpysgi-asgi does not subscribe to static-h2
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2c
[info] skip: fastpysgi-asgi does not subscribe to json-h2c
[info] skip: fastpysgi-asgi does not subscribe to baseline-h3
[info] skip: fastpysgi-asgi does not subscribe to static-h3
[info] skip: fastpysgi-asgi does not subscribe to gateway-64
[info] skip: fastpysgi-asgi does not subscribe to gateway-h3
[info] skip: fastpysgi-asgi does not subscribe to production-stack
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@remittor remittor force-pushed the fastpysgi-0.6 branch 4 times, most recently from d3d9476 to ec383be Compare April 24, 2026 12:22
@remittor
Copy link
Copy Markdown
Contributor Author

/benchmark -f fastpysgi-asgi --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

🚀 Benchmark run triggered for fastpysgi-asgi (all tests) with --save. Results will be posted here when done.

@remittor
Copy link
Copy Markdown
Contributor Author

@MDA2AV , I still haven't been able to reproduce the issue locally.
I just downgraded the Python version in Docker to 3.9 (that's what I'm testing on), and all the checks passed right away.
So you can commit this PR, and I'll try to fix the issue in v0.7 on other Python versions.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: fastpysgi-asgi | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 589,989 6774.1% 404MiB -24.0% +45.9%
baseline 4096 751,906 6762.3% 1.1GiB -22.9% +39.3%
pipelined 512 530,164 5881.8% 362MiB -60.3% +31.3%
pipelined 4096 1,115,830 6678.8% 956MiB -27.6% +19.6%
limited-conn 512 469,504 6755.0% 490MiB -22.9% +35.9%
limited-conn 4096 616,510 6794.7% 1.9GiB -21.9% +18.7%
json 4096 483,021 6686.7% 2.2GiB -19.8% +37.5%
json-comp 512 249,613 6650.6% 557MiB -15.3% +14.1%
json-comp 4096 293,808 6666.8% 1.8GiB -9.4% ~0%
json-comp 16384 287,234 6696.3% 6.0GiB -9.3% +15.4%
json-tls 4096 442,114 6718.7% 621MiB NEW NEW
upload 32 1,880 6436.5% 2.3GiB +0.9% +15.0%
upload 256 1,600 6474.5% 5.7GiB +1.2% +26.7%
api-4 256 25,497 400.3% 258MiB -1.3% -32.9%
api-16 1024 62,381 1610.0% 765MiB -9.4% +6.6%
static 1024 722,638 6830.4% 398MiB -18.5% +71.8%
static 4096 802,458 6795.9% 838MiB -15.9% +4.1%
static 6800 792,865 6756.7% 2.2GiB -14.8% +496.9%
async-db 1024 111,509 6621.7% 1.4GiB -22.6% +27.3%
Full log
[info] CPU 6795.9% | Mem 838MiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.08ms  636.81us  11.92ms   77.80%
    Req/Sec    12.60k     0.92k   30.73k    97.42%
  4085920 requests in 5.10s, 62.57GB read
Requests/sec: 801128.11
Transfer/sec:     12.27GB
[info] CPU 6797.1% | Mem 1.3GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.11ms    0.89ms  11.07ms   74.51%
    Req/Sec    12.54k     0.91k   30.10k    97.79%
  4067180 requests in 5.10s, 62.28GB read
Requests/sec: 797489.71
Transfer/sec:     12.21GB
[info] CPU 6784.4% | Mem 1.4GiB

=== Best: 802458 req/s (CPU: 6795.9%, Mem: 838MiB) ===
[info] saved results/static/4096/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.58ms    1.28ms  30.22ms   84.25%
    Req/Sec    12.39k     0.89k   30.66k    96.62%
  4017028 requests in 5.10s, 61.52GB read
Requests/sec: 787520.37
Transfer/sec:     12.06GB
[info] CPU 6774.0% | Mem 1.3GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.51ms    1.25ms  35.50ms   85.39%
    Req/Sec    12.48k     1.05k   32.71k    97.18%
  4043887 requests in 5.10s, 61.93GB read
Requests/sec: 792857.46
Transfer/sec:     12.14GB
[info] CPU 6767.6% | Mem 2.2GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.52ms    1.12ms  22.86ms   79.06%
    Req/Sec    12.47k     1.06k   33.21k    97.45%
  4043922 requests in 5.10s, 61.93GB read
Requests/sec: 792865.87
Transfer/sec:     12.14GB
[info] CPU 6756.7% | Mem 2.2GiB

=== Best: 792865 req/s (CPU: 6756.7%, Mem: 2.2GiB) ===
[info] saved results/static/6800/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / async-db / 1024c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.75ms   4.43ms   18.90ms   81.20ms   190.50ms

  1115096 requests in 10.00s, 1115098 responses
  Throughput: 111.48K req/s
  Bandwidth:  430.65MB/s
  Status codes: 2xx=1115098, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1115094 / 1115098 responses (100.0%)
  Reconnects: 44325
  Per-template: 222040,222701,223745,224277,222331
  Per-template-ok: 222040,222701,223745,224277,222331
[info] CPU 6621.7% | Mem 1.4GiB

[run 2/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.62ms   4.41ms   18.60ms   81.20ms   182.20ms

  1111221 requests in 10.00s, 1111222 responses
  Throughput: 111.09K req/s
  Bandwidth:  429.02MB/s
  Status codes: 2xx=1111222, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1111221 / 1111222 responses (100.0%)
  Reconnects: 44206
  Per-template: 221232,222370,222988,222967,221664
  Per-template-ok: 221232,222370,222988,222967,221664
[info] CPU 6660.3% | Mem 1.6GiB

[run 3/3]
gcannon v0.5.2
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.82ms   4.38ms   19.40ms   84.10ms   186.10ms

  1105145 requests in 10.00s, 1105146 responses
  Throughput: 110.49K req/s
  Bandwidth:  426.84MB/s
  Status codes: 2xx=1105146, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1105144 / 1105146 responses (100.0%)
  Reconnects: 43963
  Per-template: 219992,220801,221878,221824,220649
  Per-template-ok: 219992,220801,221878,221824,220649
[info] CPU 6676.0% | Mem 1.7GiB

=== Best: 111509 req/s (CPU: 6621.7%, Mem: 1.4GiB) ===
[info] input BW: 7.44MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi
[info] skip: fastpysgi-asgi does not subscribe to crud
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2
[info] skip: fastpysgi-asgi does not subscribe to static-h2
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2c
[info] skip: fastpysgi-asgi does not subscribe to json-h2c
[info] skip: fastpysgi-asgi does not subscribe to baseline-h3
[info] skip: fastpysgi-asgi does not subscribe to static-h3
[info] skip: fastpysgi-asgi does not subscribe to gateway-64
[info] skip: fastpysgi-asgi does not subscribe to gateway-h3
[info] skip: fastpysgi-asgi does not subscribe to production-stack
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@remittor
Copy link
Copy Markdown
Contributor Author

It's clear to see how much slower Python 3.9 is than 3.13

@remittor remittor marked this pull request as draft April 24, 2026 13:42
@remittor remittor force-pushed the fastpysgi-0.6 branch 2 times, most recently from 0397a2f to 3d2ab48 Compare April 24, 2026 13:54
@remittor
Copy link
Copy Markdown
Contributor Author

I reproduced this bug. It only occurs after test pipeline ;-)
Now I'll be looking for the cause of the bug locally. And this bug doesn't exist on Python 3.9

@remittor remittor force-pushed the fastpysgi-0.6 branch 4 times, most recently from f8486da to 8b9c83e Compare April 25, 2026 11:40
@remittor
Copy link
Copy Markdown
Contributor Author

/benchmark -f fastpysgi-asgi --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

🚀 Benchmark run triggered for fastpysgi-asgi (all tests) with --save. Results will be posted here when done.

@remittor remittor marked this pull request as ready for review April 25, 2026 12:34
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: fastpysgi-asgi | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 819,623 6889.8% 1.6GiB +5.5% +491.7%
baseline 4096 1,027,869 6888.0% 2.4GiB +5.4% +204.0%
limited-conn 512 635,405 6765.2% 1.0GiB +4.4% +184.0%
limited-conn 4096 816,175 6791.8% 2.5GiB +3.4% +56.2%
json 4096 637,681 6803.3% 3.1GiB +5.9% +93.8%
json-comp 512 365,291 6701.0% 810MiB +24.0% +66.0%
json-comp 4096 435,875 6768.9% 2.3GiB +34.4% +27.8%
json-comp 16384 418,617 6743.5% 5.8GiB +32.1% +11.5%
json-tls 4096 561,818 6746.9% 1.4GiB NEW NEW
upload 32 1,923 6436.0% 2.2GiB +3.2% +10.0%
upload 256 1,598 6458.2% 4.9GiB +1.1% +8.9%
api-4 256 32,003 399.8% 424MiB +23.8% +10.3%
api-16 1024 78,198 1606.0% 1.0GiB +13.5% +42.7%
static 1024 918,436 6771.1% 1.2GiB +3.6% +430.6%
static 4096 1,004,902 6695.9% 1.4GiB +5.4% +78.0%
static 6800 978,556 6656.3% 1.4GiB +5.1% +279.9%
async-db 1024 147,280 6702.5% 1.7GiB +2.3% +54.5%
Full log
Requests/sec: 1004902.48
Transfer/sec:     15.39GB
[info] CPU 6695.9% | Mem 1.4GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.15ms    1.62ms  39.96ms   97.77%
    Req/Sec    15.67k     1.16k   35.91k    97.46%
  5089871 requests in 5.10s, 77.95GB read
Requests/sec: 997839.59
Transfer/sec:     15.28GB
[info] CPU 6691.0% | Mem 2.4GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.25ms    2.36ms  48.22ms   98.21%
    Req/Sec    15.61k     1.08k   35.38k    96.41%
  5063372 requests in 5.10s, 77.55GB read
Requests/sec: 992791.95
Transfer/sec:     15.20GB
[info] CPU 6701.0% | Mem 3.2GiB

=== Best: 1004902 req/s (CPU: 6695.9%, Mem: 1.4GiB) ===
[info] saved results/static/4096/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.95ms    1.58ms  34.24ms   93.92%
    Req/Sec    15.36k     1.35k   38.86k    97.58%
  4988081 requests in 5.10s, 76.39GB read
Requests/sec: 978556.85
Transfer/sec:     14.99GB
[info] CPU 6656.3% | Mem 1.4GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.28ms    3.08ms  45.08ms   96.56%
    Req/Sec    15.02k     1.08k   33.44k    96.90%
  4868206 requests in 5.10s, 74.56GB read
Requests/sec: 954558.99
Transfer/sec:     14.62GB
[info] CPU 6680.4% | Mem 2.7GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.49ms    4.11ms  57.83ms   96.73%
    Req/Sec    14.88k     1.16k   32.79k    96.75%
  4824437 requests in 5.10s, 73.89GB read
Requests/sec: 945856.02
Transfer/sec:     14.49GB
[info] CPU 6690.4% | Mem 3.5GiB

=== Best: 978556 req/s (CPU: 6656.3%, Mem: 1.4GiB) ===
[info] saved results/static/6800/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi

==============================================
=== fastpysgi-asgi / async-db / 1024c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.60ms   3.25ms   14.30ms   63.70ms   144.80ms

  1444138 requests in 10.00s, 1444139 responses
  Throughput: 144.38K req/s
  Bandwidth:  557.63MB/s
  Status codes: 2xx=1444139, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1444134 / 1444139 responses (100.0%)
  Reconnects: 57516
  Per-template: 287212,288817,290282,289802,288021
  Per-template-ok: 287212,288817,290282,289802,288021
[info] CPU 6567.8% | Mem 1.4GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.39ms   3.30ms   13.90ms   54.50ms   129.10ms

  1472805 requests in 10.00s, 1472808 responses
  Throughput: 147.24K req/s
  Bandwidth:  568.96MB/s
  Status codes: 2xx=1472808, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1472807 / 1472808 responses (100.0%)
  Reconnects: 58746
  Per-template: 292908,294475,295566,295668,294190
  Per-template-ok: 292908,294475,295566,295668,294190
[info] CPU 6702.5% | Mem 1.7GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.54ms   3.26ms   14.10ms   63.00ms   139.70ms

  1440610 requests in 10.00s, 1440612 responses
  Throughput: 144.03K req/s
  Bandwidth:  556.52MB/s
  Status codes: 2xx=1440612, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1440610 / 1440612 responses (100.0%)
  Reconnects: 57500
  Per-template: 286276,287995,289440,289379,287520
  Per-template-ok: 286276,287995,289440,289379,287520
[info] CPU 6754.2% | Mem 1.9GiB

=== Best: 147280 req/s (CPU: 6702.5%, Mem: 1.7GiB) ===
[info] input BW: 9.83MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/fastpysgi-asgi.json
httparena-bench-fastpysgi-asgi
httparena-bench-fastpysgi-asgi
[info] skip: fastpysgi-asgi does not subscribe to crud
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2
[info] skip: fastpysgi-asgi does not subscribe to static-h2
[info] skip: fastpysgi-asgi does not subscribe to baseline-h2c
[info] skip: fastpysgi-asgi does not subscribe to json-h2c
[info] skip: fastpysgi-asgi does not subscribe to baseline-h3
[info] skip: fastpysgi-asgi does not subscribe to static-h3
[info] skip: fastpysgi-asgi does not subscribe to gateway-64
[info] skip: fastpysgi-asgi does not subscribe to gateway-h3
[info] skip: fastpysgi-asgi does not subscribe to production-stack
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc
[info] skip: fastpysgi-asgi does not subscribe to unary-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc
[info] skip: fastpysgi-asgi does not subscribe to stream-grpc-tls
[info] skip: fastpysgi-asgi does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants