Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0931898
ioxide: subscribe to async, with a delay that stays on the reactor
MDA2AV Aug 26, 2026
0799d64
ioxide: enable the entry
MDA2AV Aug 26, 2026
dcc8a56
Benchmark results: ioxide [skip ci]
github-actions[bot] Aug 26, 2026
50e8999
ioxide: take the delay registration lock once per thread, not once pe…
MDA2AV Aug 26, 2026
0d7ca3b
ioxide: drain delays on the reactor that owns them, and stop spinning…
MDA2AV Aug 26, 2026
810e14a
ioxide: give each reactor 1024 recv slots instead of 256
MDA2AV Aug 26, 2026
afc2925
ioxide: add a ring-native delay on a timerfd, off by default
MDA2AV Aug 26, 2026
0f411ca
Revert "ioxide: give each reactor 1024 recv slots instead of 256"
MDA2AV Aug 26, 2026
f8416e4
ioxide: default the delay to the ring timer, to read it on the bench …
MDA2AV Aug 26, 2026
279cd26
Revert "ioxide: default the delay to the ring timer, to read it on th…
MDA2AV Aug 27, 2026
a1adedd
ioxide: default the delay to the ring timer
MDA2AV Aug 27, 2026
d54ef1b
ioxide: do not resume a ring delay before its deadline
MDA2AV Aug 27, 2026
85dbc7c
async: 16,000 connections at 5ms, down from 64,000 at 15ms
MDA2AV Aug 27, 2026
6d182b7
async: regenerate board data for the 16k/5ms conn set
MDA2AV Aug 27, 2026
b68e4d0
async: 32,000 connections at 10ms, holding the ceiling at 3.20M
MDA2AV Aug 27, 2026
3eb815e
async: 64,000 connections at 20ms, third point on the same 3.20M ceiling
MDA2AV Aug 27, 2026
349e09a
async: settle at 32,000 connections and 10ms
MDA2AV Aug 27, 2026
89287ea
async: drop every stored result from a connection count the profile n…
MDA2AV Aug 27, 2026
906a049
Benchmark results: 7 frameworks (async) [skip ci]
github-actions[bot] Aug 27, 2026
62922ec
ioxide: hold one timer per reactor instead of one per request
MDA2AV Aug 27, 2026
890d2a7
bun: answer GET /delay/{ms} and subscribe to async
MDA2AV Aug 27, 2026
9fd416b
ioxide: complete due waits from the reactor's hot path, not only from…
MDA2AV Aug 27, 2026
89a2009
async: let the generator's thread count be set on its own, and try 128
MDA2AV Aug 27, 2026
b310fd2
Revert the hot-path drain and put the generator back to 64 threads
MDA2AV Aug 27, 2026
889e2b2
async: halve the server's cores, to tell "out of work" from "contended"
MDA2AV Aug 27, 2026
08b8646
ioxide: put the per-request ring timer back as the default, on 16 cores
MDA2AV Aug 27, 2026
f8c1bd7
ioxide: wait on IORING_OP_TIMEOUT, with the library build vendored to…
MDA2AV Aug 27, 2026
d00433b
async: back to the profile's own 32 cores, with the ring timeout in p…
MDA2AV Aug 27, 2026
d28c7a9
ioxide: move onto the released 0.7.211 and drop the vendored package
MDA2AV Aug 28, 2026
172499a
ioxide: take the wait from ioxide.timer instead of a copy of it
MDA2AV Aug 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The reply states which baseline it used, and profiles the other framework does n
| Category | Profiles | Description |
|----------|----------|-------------|
| Connection | `baseline`, `pipelined` *, `limited-conn` | Mixed GET/POST with query parsing (512/4K conns), 16× batched pipelining (reference-only, shown faded, excluded from the composite score), short-lived connections that close after 10 requests |
| Concurrency | `async` * | `GET /delay/{ms}` with the wait named in the route, 64K held connections. Isolates what a framework does while a request is pending, with no database or I/O in the way. Reference-only for now: measured and shown, but not yet part of the composite score |
| Concurrency | `async` * | `GET /delay/{ms}` with the wait named in the route, 32K held connections. Isolates what a framework does while a request is pending, with no database or I/O in the way. Reference-only for now: measured and shown, but not yet part of the composite score |
| Efficiency | `latency-1m` | One million req/s offered at a fixed rate with zrk; the metric is CPU spent per request, read exactly from the container's cgroup rather than sampled. Everyone who finishes serves the same million, so what separates them is the bill. Scored on its own 0-100 formula rather than on rps, since the rate is pinned |
| Workload | `json`, `json-comp`, `json-tls`, `upload`, `static`, `static-tls` | JSON serialization, gzip/brotli compression, HTTP/1.1 over TLS, 20 MB body ingestion, 20-file static asset serving (plaintext and TLS) |
| Database | `async-db` *, `crud` * | Async Postgres sequential scan; realistic REST API with cached reads, list, upsert, update, and optional Redis cache. Both reference-only: the database and its driver dominate these far more than the framework does, so they are measured and shown but no longer decide the ranking |
Expand Down
4 changes: 3 additions & 1 deletion frameworks/bun/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@
"repo": "https://github.com/oven-sh/bun",
"enabled": true,
"tests": [
"baseline", "latency-1m",
"baseline",
"async",
"latency-1m",
"pipelined",
"limited-conn",
"json",
Expand Down
14 changes: 14 additions & 0 deletions frameworks/bun/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,20 @@ const routes = {
// one segment, so a path with a slash in it never reaches the handler
"/static/:name": (req: any) => serveStatic(req.params.name, req),

// GET /delay/{ms} - answered once ms milliseconds have gone by. Bun.sleep returns a
// promise the scheduler resolves on a timer, so awaiting it parks this request and
// hands the thread back: a wait costs a timer entry rather than a blocked worker,
// which is what lets one process hold tens of thousands of them at once.
//
// The value is parsed per request, out of the path Bun.serve matched. Keeping it in
// this scope rather than anywhere shared is what makes overlapping requests with
// different delays each get their own, which validation checks with 32 at a time.
"/delay/:ms": async (req: any) => {
const ms = parseInt(req.params.ms, 10) || 0;
if (ms > 0) await Bun.sleep(ms);
return new Response(String(ms), { headers: TEXT });
},

"/async-db": (req: Request) => asyncDb(qs(req)),

"/fortunes": () => fortunes(),
Expand Down
15 changes: 15 additions & 0 deletions frameworks/ioxide/Handler.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
using System.Runtime.InteropServices;

using ioxide;
using ioxide.timer;
using ioxide.file;
using ioxide.pg;
using ioxide.tls;
Expand Down Expand Up @@ -81,6 +82,20 @@ public static async Task HandleAsync(Reactor reactor, TcpConnection conn)
// /async-db parks the parser: run the query (inline on this reactor's
// ring via ioxide.pg), stream rows into Out, then resume the carry -
// pipelined requests behind it are served in order.
// The async profile's wait, on ioxide.timer: the deadline goes to the
// kernel with the submission, so it completes on this reactor with the
// connection's state still warm and the reactor free for everything else
// it owns meanwhile, which is the whole measurement. One timer per
// connection, re-armed per request - a connection only ever waits once
// at a time, because the handler awaits before it reads again.
while (httpSession.PendingDelay)
{
httpSession.PendingDelay = false;
await (httpSession.Timer ??= new RingTimer(reactor)).DelayAsync(httpSession.PendingDelayMs);
httpSession.CompleteDelay();
httpSession.ResumeFeed();
}

while (httpSession.PendingDb)
{
httpSession.PendingDb = false;
Expand Down
42 changes: 41 additions & 1 deletion frameworks/ioxide/HttpSession.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System.Text.Json;
using ioxide.file;
using ioxide.pg;
using ioxide.timer;

namespace IoxideArena;

Expand Down Expand Up @@ -61,6 +62,17 @@ public HttpSession(Dataset ds, StaticAssets? assets, Precompressed? precompresse
private bool _dbFirstRow;
private int _dbRows;

// /delay/{ms} parks the parser the way /async-db does: the wait happens in
// the handler, on this reactor, and the carry resumes behind it so pipelined
// requests are still answered in order.
public bool PendingDelay;
public int PendingDelayMs;
private bool _delayClose;
// One per connection, reused for every delay it ever asks for. Built on first
// use rather than at construction, so a connection that never waits never makes
// one - which is every connection on every profile but this one.
public RingTimer? Timer;

// /upload streams its body: bytes are counted as they arrive, never buffered whole.
public long PendingUploadRemaining;
private long _uploadTotal;
Expand Down Expand Up @@ -90,6 +102,18 @@ public void Feed(ReadOnlySpan<byte> data)
Pump();
}

/// <summary>The wait is over: answer with the number of milliseconds asked for.</summary>
public void CompleteDelay()
{
Span<byte> num = stackalloc byte[16];
Utf8Formatter.TryFormat(PendingDelayMs, num, out int n);
WriteResp(num[..n], _delayClose);
if (_delayClose)
{
WantClose = true;
}
}

// The streamed upload's body is fully counted; emit the 200 with the total byte count.
private void FinishUpload()
{
Expand All @@ -106,7 +130,7 @@ private void FinishUpload()
private void Pump()
{
int pos = 0;
while (!PendingDb && PendingUploadRemaining == 0
while (!PendingDb && !PendingDelay && PendingUploadRemaining == 0
&& TryOne(_carry.AsSpan(pos, _carryLen - pos), out int consumed, out bool close))
{
pos += consumed;
Expand Down Expand Up @@ -302,6 +326,22 @@ private void Respond(ReadOnlySpan<byte> method, ReadOnlySpan<byte> target, ReadO
PendingDb = true;
PendingDbClose = close;
}
else if (path.StartsWith("/delay/"u8))
{
// The value is echoed back, so it has to be the number that was
// parsed rather than the text: "007" answers 7.
ReadOnlySpan<byte> tail = path[7..];
if (Utf8Parser.TryParse(tail, out int ms, out int used) && used == tail.Length && ms >= 0)
{
PendingDelayMs = ms;
_delayClose = close;
PendingDelay = true;
}
else
{
Write404(close);
}
}
else if (path.SequenceEqual("/upload"u8))
{
Span<byte> num = stackalloc byte[16];
Expand Down
39 changes: 32 additions & 7 deletions frameworks/ioxide/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,36 @@ private static int Main()
// userspace, the fastest backend on loopback.
using var quic = tls ? new QuicEngine(certPath, keyPath, cidLength: 8, alpn: ["h3"]) : null;

// The knobs below are left at the library default unless the environment names
// one. The async profile holds 64,000 connections open at once, which is ~1,000
// per reactor, and a default sized for ordinary keep-alive traffic throttles
// submission long before the CPU is busy - so they have to be reachable.
// These are init-only, so an unset variable has to fall back to the library's
// own default rather than to a number picked here - hence the throwaway
// instance to read those defaults off.
var tcpDefault = new TcpOptions();
var srvDefault = new ServerConfig();
Console.WriteLine($"[cfg] library defaults: ringEntries={srvDefault.RingEntries} recvSlots={srvDefault.RecvSlots} "
+ $"backlog={tcpDefault.ListenBacklog} poolMax={tcpDefault.PoolMax} recvQueue={tcpDefault.RecvQueueEntries} zeroCopy={tcpDefault.ZeroCopySend}");

var tcp = new TcpOptions
{
Port = port,
ExtraPorts = tls ? [(ushort)8081, (ushort)8443] : [],
// 128 KB so a static response fits one slab and the handler sends it without chunk-flushing.
WriteSlabSize = 128 * 1024,
ListenBacklog = EnvInt("IOXIDE_BACKLOG") ?? tcpDefault.ListenBacklog,
PoolMax = EnvInt("IOXIDE_POOL_MAX") ?? tcpDefault.PoolMax,
RecvQueueEntries = EnvInt("IOXIDE_RECV_QUEUE") ?? tcpDefault.RecvQueueEntries,
};

var config = new ServerConfig
{
ReactorCount = reactors,
RecvBufferSize = recvKb * 1024,
RecvSlots = ringEntries,
Tcp = new TcpOptions
{
Port = port,
ExtraPorts = tls ? [(ushort)8081, (ushort)8443] : [],
// 128 KB so a static response fits one slab and the handler sends it without chunk-flushing.
WriteSlabSize = 128 * 1024,
},
RingEntries = (uint)(EnvInt("IOXIDE_SQ_ENTRIES") ?? (int)srvDefault.RingEntries),
Tcp = tcp,
Quic = quic == null ? null : new QuicOptions
{
Port = 8443,
Expand All @@ -88,6 +106,9 @@ private static int Main()
},
};

Console.WriteLine($"[cfg] effective: reactors={config.ReactorCount} ringEntries={config.RingEntries} recvSlots={config.RecvSlots} "
+ $"recvBuf={config.RecvBufferSize} backlog={config.Tcp.ListenBacklog} poolMax={config.Tcp.PoolMax} recvQueue={config.Tcp.RecvQueueEntries}");

var dsPath = Environment.GetEnvironmentVariable("IOXIDE_DATASET") ?? "/data/dataset.json";
var dataset = Dataset.Load(dsPath);

Expand Down Expand Up @@ -227,4 +248,8 @@ private static string ResolveIPv4(string host)

return "127.0.0.1";
}

/// <summary>An int from the environment, or null when it is unset or unparseable.</summary>
private static int? EnvInt(string name)
=> int.TryParse(Environment.GetEnvironmentVariable(name), out int v) && v > 0 ? v : null;
}
15 changes: 8 additions & 7 deletions frameworks/ioxide/ioxide-arena.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="ioxide" Version="0.7.210" />
<PackageReference Include="ioxide.pg" Version="0.7.210" />
<PackageReference Include="ioxide.file" Version="0.7.210" />
<PackageReference Include="ioxide.redis" Version="0.7.210" />
<PackageReference Include="ioxide.http2" Version="0.7.210" />
<PackageReference Include="ioxide.ngtcp2" Version="0.7.210" />
<PackageReference Include="ioxide.nghttp3" Version="0.7.210" />
<PackageReference Include="ioxide" Version="0.7.211" />
<PackageReference Include="ioxide.pg" Version="0.7.211" />
<PackageReference Include="ioxide.timer" Version="0.7.211" />
<PackageReference Include="ioxide.file" Version="0.7.211" />
<PackageReference Include="ioxide.redis" Version="0.7.211" />
<PackageReference Include="ioxide.http2" Version="0.7.211" />
<PackageReference Include="ioxide.ngtcp2" Version="0.7.211" />
<PackageReference Include="ioxide.nghttp3" Version="0.7.211" />
<PackageReference Include="StackExchange.Redis" Version="2.8.16" />
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.0" />
</ItemGroup>
Expand Down
4 changes: 2 additions & 2 deletions frameworks/ioxide/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
"engine": "io_uring",
"description": "ioxide \u2014 a shared-nothing io_uring runtime for .NET, consumed as its published NuGet packages (ioxide, ioxide.pg, ioxide.file, ioxide.http2, ioxide.ngtcp2, ioxide.nghttp3). One ring per reactor thread: multishot accept/recv into provided buffer rings, inline IValueTaskSource continuations, raw-syscall io_uring (no liburing). The HTTP/1.1 handler (request line, headers, Content-Length + chunked bodies, keep-alive, pipelining, fragmented reads) is hand-written on the raw recv/send API; TLS terminates ring-natively (kTLS transmit offload, OpenSSL receive) on :8081, h2 over TLS (pure-C# ioxide.http2) on :8443, and HTTP/3 over QUIC (ngtcp2 + nghttp3) on udp:8443; async-db rides ioxide.pg's ring-native pooled Postgres driver (SCRAM-SHA-256), static on h1 reads the file descriptor straight into the connection's write slab (ioxide.file hands out descriptors, not bytes), so header and body leave in one flush with no intermediate buffer; h2/h3 serve the same assets from memory.",
"repo": "https://github.com/MDA2AV/ioxide",
"enabled": false,
"enabled": true,
"tests": [
"baseline", "latency-1m",
"baseline", "async", "latency-1m",
"pipelined",
"limited-conn",
"json",
Expand Down
2 changes: 1 addition & 1 deletion requests/async-delay.raw
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
GET /delay/15 HTTP/1.1
GET /delay/10 HTTP/1.1
Host: localhost:8080

4 changes: 2 additions & 2 deletions scripts/gen_leaderboard_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@
# tuned (#1310). The other two flags are set for the day it flips:
# engines are measured on it, infrastructure is not — a reverse proxy
# has no application handler to await in.
("async", "Async Delay", "A 15ms wait named in the route, at 64K held connections.",
[64000], [64000], False,True,False),
("async", "Async Delay", "A 10ms wait named in the route, at 32K held connections.",
[32000], [32000], False,True,False),
]),
("Efficiency", [
# Scored. The composite cannot rank this on rps the way it ranks every
Expand Down
7 changes: 7 additions & 0 deletions scripts/lib/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ H2C_PORT=8082 # h2c prior-knowledge (baseline-h2c, json-h2c profiles)
DURATION="${DURATION:-5s}"
RUNS="${RUNS:-3}"
THREADS="${THREADS:-64}"
# The async profile's generator thread count, separately settable. It gets its own
# knob because the profile is closed-loop - every connection holds one request for
# the length of its wait - so what it reports is latency divided into connections,
# and the generator contributes to that latency. Holding the server fixed on a
# 32-core box and changing only this moved the result from 1.42M at 8 threads to
# 2.16M at 16, so the number is not the server's alone.
ASYNC_THREADS="${ASYNC_THREADS:-64}"
H2THREADS="${H2THREADS:-64}"
H3THREADS="${H3THREADS:-64}"

Expand Down
2 changes: 1 addition & 1 deletion scripts/lib/profiles.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ declare -A PROFILES=(
# localhost:8080 cannot be established at all — the whole 16-bit port space
# is smaller than that. 64000 fits with ~500 ports to spare; if the ramp
# ever shows up as connect errors, 61440 is the next stop down.
[async]="1|0|0-31,64-95|64000|async"
[async]="1|0|0-31,64-95|32000|async"
# Latency-1M: the offered rate is pinned at 1M req/s (see ZRK_FIXED_RATE
# in tools/zrk.sh) and the measurement is what the server spent to serve
# it, read exactly out of the container's cgroup rather than sampled.
Expand Down
23 changes: 15 additions & 8 deletions scripts/lib/tools/gcannon.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,21 +41,28 @@ gcannon_build_args() {
-c "$conns" -t "$THREADS" -d 10s -p "$pipeline" -r 25)
;;
async)
# One template, a flat 15ms wait. The value still travels in the
# path, so the handler has to parse it out of the route -- what a
# constant delay gives up is the on-the-wire variation, which is why
# One template, a flat wait. The value still travels in the path, so
# the handler has to parse it out of the route -- what a constant
# delay gives up is the on-the-wire variation, which is why
# validate.sh draws its own values and asserts the differential.
#
# No -r: connections are held for the whole run. That is the point.
# Every held connection is one request the server is currently
# sleeping on, so 64K connections means 64K live timers.
# sleeping on, so 32K connections means 32K live timers.
#
# 10s rather than the 5s default: long enough that establishing
# 64000 connections is a small share of the window, short enough to
# keep three runs plus their cool-downs reasonable.
# 10s rather than the 5s default: long enough that establishing the
# connections is a small share of the window, short enough to keep
# three runs plus their cool-downs reasonable.
#
# ASYNC_THREADS overrides the generator's thread count for this
# profile alone. It exists because the result moves with it: on a
# 32-core box, holding the server fixed and changing only this took
# throughput from 1.42M at 8 threads to 2.16M at 16, so some of what
# the profile reports is the generator rather than the server. The
# default stays at THREADS so nothing changes unless it is set.
args=("http://localhost:$PORT"
--raw "$REQUESTS_DIR/async-delay.raw"
-c "$conns" -t "$THREADS" -d 10s -p "$pipeline")
-c "$conns" -t "${ASYNC_THREADS:-$THREADS}" -d 10s -p "$pipeline")
;;
fortunes)
# Single endpoint, fixed 12-row seed + 1 runtime-injected row.
Expand Down
Loading
Loading