ioxide: subscribe to async, and retune the profile to 16k/5ms - #1341
Merged
Conversation
Owner
Author
|
/benchmark --save |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFramework:
Full log |
MDA2AV
force-pushed
the
feat/ioxide-async-delay
branch
from
August 26, 2026 22:29
cf7ef53 to
66bd8c3
Compare
Owner
Author
|
/benchmark -f ioxide -t async |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
1 similar comment
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFramework:
Full log |
Owner
Author
|
/benchmark -f ioxide -t async Measuring |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFramework:
Full log |
Owner
Author
|
/benchmark -f ioxide -t async Recv slots 256 -> 1024 per reactor. The timer instrumentation showed it completes only 3.2ms late on average against a 37.8ms end-to-end latency, so ~20ms is the read/write path, not the wait. This tests whether slot starvation is that 20ms. Not saving. Watch CPU: it sat at 3000% of an available 6400%, so if this is the limiter, utilisation should rise. Memory will go up (~1GB of recv buffers vs 256MB), which is the trade being measured. |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFramework:
Full log |
Owner
Author
|
/benchmark -f ioxide -t async Delay now defaults to the ring timer: a timerfd armed per request and submitted to the reactor's own ring via On my 32-reactor box the tick beat it (1.58M @ 1765% vs 1.50M @ 2007%), because the ring costs a Not saving. Last run was 1,568,519 @ 2802%, with 6400% available. |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFramework:
Full log |
MDA2AV
added a commit
that referenced
this pull request
Aug 27, 2026
Experiment on the experiment branch; #1341 stays on the tick. The ring was measured once, at 64,000 connections and 15ms, where it bought 5% throughput for 51% more CPU and lost. The profile has changed underneath that result: 16,000 connections at 5ms is a third of the connections and three times the request rate, and the two mechanisms scale with different things. The ring costs a timerfd_settime syscall plus an SQE and a CQE per request, so its cost tracks the request rate and this profile raises that. The tick costs one post per reactor per tick regardless of the rate, so its cost is unchanged at 64 reactors while it now amortises over fewer connections per reactor: 250 here against 1,000 before, which is 250 timers per drain becoming about 60. Both effects point the same way, against the ring, so this is a check rather than a hope. /delay/5 answers in 5.22-5.28ms on this path.
MDA2AV
changed the title
Owner
Author
|
/benchmark-multiple -f ioxide,tokio -t async Consolidated branch: profile at 16,000 conns / 5ms, ioxide on the ring timer with the deadline guard. The previous 2,462,698 came from the unguarded build, where ~0.1% of waits resumed early (worst case 14.9ms into a 15ms delay). This re-reads it with those requests actually waiting, so the number should be a touch lower and honest. tokio unchanged as the control: 2,054,680 @ 3319.6% on this profile. Not saving. |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 16000 | 2,359,158 | 5558.4% | 1.0GiB | NEW | NEW |
✅ tokio
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 16000 | 2,055,165 | 3319.4% | 609MiB | NEW | NEW |
Owner
Author
|
/benchmark-multiple -f ioxide,tokio -t async Profile now 32,000 connections at 10ms. This is a control, not a retune: the ceiling is connections over delay, so this and 16k/5ms are both 3.20M. Connection count is the only variable.
The ordering reverses between the first two, but they differ in ceiling as well as connections so nothing is isolated. Here a drop against the 16k/5ms numbers is connection scaling and cannot be the arithmetic. ioxide on the ring with the deadline guard. Not saving. |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 32000 | 2,390,257 | 6066.3% | 1.5GiB | NEW | NEW |
✅ tokio
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 32000 | 2,359,835 | 4078.8% | 1.2GiB | NEW | NEW |
Owner
Author
|
/benchmark-multiple -f ioxide,tokio -t async 64,000 connections at 20ms — third point with the ceiling pinned at 3.20M, so connection count stays the only variable.
The first two points already say tokio scales on this axis and ioxide does not: doubling connections moved tokio 2.05M -> 2.36M while ioxide went 2.36M -> 2.39M, flat, needing 6066% CPU against tokio 4079% to stay level. This point tests something else. The original 64,000 run gave 1,568,519 against a 4.27M ceiling (37%), which we read as connections being the problem. Here the connections are identical and only the arithmetic differs — if this lands near 2.4M, 64,000 connections were never the difficulty. Caveat: both are already at ~75% of 3.20M and queueing grows fast that close to a ceiling, so some convergence may be the arithmetic rather than the frameworks. Not saving. |
The rest of this entry never waits in userspace: ioxide.pg parks on the Postgres socket through the reactor's ring and the completion arrives on the reactor thread. The delay was the one wait that did not, so this makes it possible to do the same and measures what it costs. Reactor.SubmitRead takes a raw fd, and a timerfd is pollable, so io_uring arms a poll on it internally rather than handing the read to a worker thread. That gets to the same place as IORING_OP_TIMEOUT, which ioxide does not expose (MDA2AV/ioxide#212). One timerfd per connection, created on its first delay and re-armed after, released with the connection. It is correct and it is more precise than the tick: /delay/1 answers in 1.06ms, /delay/200 in 200.23ms, and 30 re-arms on one connection overshoot by 0.18ms on average against the tick's 0.25-0.82ms. It is off by default because it measured worse, and the reason is worth keeping in the file. A socket read is I/O the connection has to do anyway, so the ring costs nothing extra for it. A timer is not: this is a timerfd_settime syscall plus an SQE and a CQE per request, roughly 1.5M syscalls a second at this load, where the tick completes about 13 timers per drain and makes no syscall at all. Four interleaved repetitions at 64,000 connections: ring 1.50M rps 2007% cpu p99 104.2ms p99.9 429.6ms tick 1.58M rps 1765% cpu p99 68.4ms p99.9 373.4ms 14% more CPU for 5% less throughput. The tick's batching is what wins, so the precision is not worth buying here. IORING_OP_TIMEOUT would remove the syscall but not the SQE and CQE, so it is worth re-measuring if that ever lands. IOXIDE_DELAY_MODE=ring selects it. Both paths verified after the default flip.
This reverts commit 7a2727d.
…hardware The two mechanisms cost differently and the difference scales with reactor count, so the 32-reactor result does not settle it. The ring timer costs a timerfd_settime syscall plus an SQE and a CQE per request. That is fixed per request whatever the box. The tick costs one cross-thread post per reactor per tick, so 128,000 posts a second across 32 reactors and 256,000 across the bench box's 64, against the same per-request cost on the other side. Locally the tick wins, over four interleaved repetitions at 64,000 connections: ring 1.50M rps 2007% cpu p99 104.2ms overshoot 0.18ms tick 1.58M rps 1765% cpu p99 68.4ms overshoot 0.25-0.82ms Twice the reactors doubles one side of that and not the other, so this defaults to the ring to get a reading from the hardware the profile is scored on. If the ordering holds there, this flips back. IOXIDE_DELAY_MODE=tick selects the other path. Both verified after the flip: ring answers /delay/15 in 15.17ms and /delay/200 in 200.32ms, tick in 16.17ms.
…e bench hardware" This reverts commit f18dad0.
Measured on the bench hardware at 16,000 connections and 5ms: tick 1,842,424 rps 3435% cpu ring 2,462,698 rps 5831% cpu avg 6.48ms p99 9.91ms p99.9 16.50ms 34% more throughput than the tick and 20% more than tokio's 2,054,680, with 1.48ms of overhead on a 5ms wait. The earlier reading that said the opposite was taken on a 32-reactor box sitting at 0.8% idle with the load generator on it. The ring costs a timerfd_settime syscall plus an SQE and a CQE per request, and on a machine with no CPU to spare that can only come out of throughput, so it lost there. The bench box has 6400% available and the tick was using 3435%, so the same trade buys something instead of costing something. Nothing about the mechanism changed between those two measurements, only whether there was CPU free to spend on it. That also explains the shape of the tick's result rather than just its size. The tick completes a timer only when a drain runs, so work arrives in batches and the reactors idle between them - which is why CPU sat at half the box while throughput flattened. Handing each deadline to the kernel keeps it flowing. The trade needs spare CPU to be worth making, so the comment records both readings and says to re-read it if the profile or the hardware changes. IOXIDE_DELAY_MODE=tick still selects the other path.
The ring completion was treated as proof the wait was over. It is not, and under load it demonstrably is not: instrumenting the completion against its own deadline showed about 0.1% of waits resuming early, the worst of them 14.9ms into a 15ms delay, which is the entire wait skipped. The cause is not the one the code was written to expect. The fd is non-blocking so a read the kernel does not arm a poll for returns -EAGAIN, but that never appeared in the measurement: every early completion carried a valid 8-byte expiration count. The fd is reused for every request on a connection, so a read can pick up an expiration left behind by an earlier one and report a timer that really did fire, only not this request's. So the deadline decides rather than the completion. An early one goes back on the ring instead of resuming, which costs one comparison on a path that is already making a syscall. Measured: earlyCompletions equalled resubmitted on every sample, and completions still tracked armings, so it converges rather than spinning. The effect on the published 2,462,698 is negligible at 0.1%, but a request that answers before it waited is wrong regardless of what it does to the number, and a sampled one would fail validation outright. 120 sequential delays on one connection: none early, none wrong-bodied, overshoot 0.05-0.46ms.
Experiment. At 64,000 connections the profile has been reporting a gap between entries that does not survive inspection: instrumenting the wait shows the timer completing 3.2ms late against 23ms of end-to-end overhead, so the delay is about 13% of what is being measured and the rest is the read and write path. Fewer connections and a shorter wait puts the load somewhere different, and whether the ordering between entries holds is the thing worth knowing. The ceiling is connections over delay, so this moves it from 4.27M to 3.20M. The best entry so far is 2.24M, which leaves headroom but less than before; the docs now say so rather than claiming the ceiling does not bind. Existing results are all at async-64000 and are orphaned by this, so the column is empty until the profile is re-run. That is accepted for now. Generator runs clean and badge parity holds at 579 ranks.
The async column drops out until the profile is re-run, since every existing result is at async-64000.
This is a control rather than a retune. The ceiling is connections over delay, so 32,000 at 10ms and 16,000 at 5ms both come to 3.20M. Doubling both leaves connection count as the only thing that changed, which is the question the profile has been circling. What is known so far, all on the bench hardware: 64,000 / 15ms ceiling 4.27M ioxide 1,568,519 tokio 2,263,962 16,000 / 5ms ceiling 3.20M ioxide 2,360,000 tokio 2,054,680 The ordering reverses between those two, but they differ in ceiling as well as in connections, so the comparison does not isolate anything. This one does: a drop against the 16k/5ms numbers is connection scaling and cannot be the arithmetic, because the arithmetic is identical. Docs carry the new figures - a 64-thread blocking server tops out near 6,400 rps here, and 1-2ms of timer overshoot now costs a tenth to a fifth of the wait rather than a fifth to a third. The best-entry reference moves to 2.36M. Generator runs clean and badge parity holds at 579 ranks.
Continues the controlled series. Connections and delay double together so the ceiling does not move, which leaves connection count as the only variable: 16,000 / 5ms ioxide 2,360,000 (74% of ceiling) tokio 2,054,680 (64%) 32,000 / 10ms ioxide 2,390,257 (75%) tokio 2,359,835 (74%) 64,000 / 20ms ceiling 3.20M What the first two points already show is that tokio scales on this axis and ioxide does not. Doubling the connections took tokio from 2,054,680 to 2,359,835 while ioxide went 2,360,000 to 2,390,257, flat, and it needed 6066% CPU against tokio's 4079% to stay level - 39,402 requests per CPU percent against 57,856. This point matters for a different reason. The original 64,000 result was 1,568,519 against a 4.27M ceiling, 37% of it, and that was read as connections being the problem. Here the connections are the same and only the arithmetic differs. If this lands near 2.4M then 64,000 connections were never the difficulty and the earlier collapse belongs to something else in that configuration. Read the numbers with the ceiling in mind: both entries are already at about 75% of 3.20M, and queueing grows quickly that close to the limit, so some of the convergence between them may be the ceiling rather than the frameworks. Generator runs clean and badge parity holds at 579 ranks.
The profile was walked along a line where connections and delay double together, so the ceiling stays 3.20M and connection count is the only thing that moves: 16,000 / 5ms ioxide 2,360,000 (74%) tokio 2,054,680 (64%) 32,000 / 10ms ioxide 2,390,257 (75%) tokio 2,359,835 (74%) 64,000 / 20ms ioxide 1,641,196 (51%) tokio 1,887,483 (59%) 64,000 costs both entries about a third of their throughput against the same ceiling, and that is the reason to stop at 32,000. Both losing together points at something they share rather than at either server: the load generator holding 64,000 sockets, and the kernel's own cost for that many. A profile whose headroom is being spent by its own harness is not measuring what it says it is. The servers do not argue for stopping here. Measured locally against this build: one reactor serves 1000 connections at 97% of its own ceiling on a fifth of a core, and pinning 1000 per reactor while going from 1 to 16 reactors holds 94-97% throughout with latency flat at 20.1ms. Neither connections per reactor nor reactor count is what bends, so the limit is in the shape of the test. 32,000 is also where the two entries agree, 2,390,257 against 2,359,835, which is what a profile measuring concurrency rather than harness capacity should look like. The gap that does remain is in CPU, 6066% against 4079%, and that is a real difference worth reporting rather than one manufactured by the load. The docs now record why the connection count is what it is, so the next person to reach for a bigger number has the measurement instead of the intuition.
…o longer runs 13 rows across 5 entries, none of which the board can show. async-64000 is from the shape the profile had before it settled at 32,000, and async-32768 and async-49152 are older still, left behind while the connection count was being tuned in the first place. A stale row is not inert here. rebuild_site_data.py keys results by profile and connection count and never prunes, so anything the CATALOG stops asking for simply accumulates, and the next person reading these files cannot tell which numbers the profile actually produces. aspnet-minimal, express, fastapi, tokio async-32768, async-49152, async-64000 ioxide async-64000 async-db is a different profile and is untouched. The async column is empty until the profile is re-run for its subscribers.
The delay now keeps deadlines in the reactor's own queue and arms a single timerfd at the front of it. This is what an event loop normally does - Node and Bun put the next deadline into the poll they were making anyway - and it is the only shape that neither syscalls per request nor leaves the reactor asleep past a deadline that has already passed. The two it replaces each gave up one of those. The per-request timerfd woke the reactor exactly on time but cost a timerfd_settime, an SQE and a CQE every single wait. The tick cost nothing per wait but only completed a timer when something else happened to wake the reactor, which is what left it idle with work already due. Arming only when a wait is due before whatever the timer already holds means most waits cost nothing at all: with waits coming due in batches of about thirteen, that is one arming per thirteen requests. Measured on this box, 16,000 connections at 2ms, three interleaved repetitions: queue 1.99M rps 1852% cpu p99 19.93ms p99.9 34.23ms tick 1.98M rps 1861% cpu p99 18.47ms p99.9 31.07ms ring 1.81M rps 1959% cpu p99 27.80ms p99.9 56.03ms The box is saturated with the load generator on it, so the three compress together here; the bench box is where ring's per-request syscall started to pay for itself by keeping reactors fed, and this gets the same wake-up behaviour without buying it. IORING_OP_TIMEOUT was tried too, with the deadline handed to the kernel on the ring. It needs a Reactor.SubmitTimeout that no published ioxide has, and it measured no better than this: 1.98M and a worse tail, because it is still one SQE and one CQE per request where this is one per batch. Left out until the library exposes it. IOXIDE_DELAY_MODE=ring or =tick still select the other two. Verified on the default path: 80 sequential delays with none early, mean overshoot 0.18ms, and 32 concurrent distinct delays served in 69.6ms against 1088ms serialised.
Bun.sleep returns a promise the scheduler resolves on a timer, so awaiting it parks the request and hands the thread back rather than holding it. Measured on this box: the wait costs bun 22%, 2.09M on the baseline mix against 1.63M on the delay path. That is worth having next to the other JS entry, where the same wait is free - fulmine goes 1.59M to 1.68M - so the profile has two runtimes with the same concurrency model and visibly different timer costs. Verified before subscribing: /delay/10 answers in 10.16ms and /delay/50 in 50.37ms, and 32 overlapping requests carrying delays from 3ms to 65ms complete in 69.6ms against the 1088ms they would take one at a time, none of them early and every body its own value.
… the timer The per-reactor timer was the only thing completing a wait, and deadlines do not arrive in groups: with one wait per request they are spread continuously, so each got its own arming and its own read. That is a syscall per request, which is precisely what holding one timer per reactor was supposed to avoid, and the published run shows it - 2,179,118 rps at 4251% CPU with 2,150% of the box sitting idle. The reactor already passes through the handler loop tens of thousands of times a second under load. Draining there costs a thread-static read and a peek, so most waits now finish for nothing at all and the timer is left covering the only case that needs it: the reactor about to sleep with work already due. Both drains take the same re-entrancy guard. Completing a wait runs its continuation inline and that continuation goes straight back round the loop, which drains again, so without it one connection's resume nests inside another's for as deep as the queue happens to be. Locally this is inside the noise, 2.00M at 1877% against 1.99M at 1852%, because the box is saturated with the load generator on it and has no idle CPU to recover. The bench box has 2,150% of it. Accuracy is the best of any mode so far: /delay/10 answers in 10.02ms and /delay/200 in 200.09ms, 100 sequential waits on one connection are none early with a mean overshoot of 0.15ms, and 32 concurrent distinct delays complete in 69.8ms against 1088ms serialised.
The profile is closed-loop: every connection holds exactly one request for the length of its wait, so what comes out is connections divided by latency, and the generator is part of that latency. It follows that the number is not the server's alone, and measurement says so - holding the server fixed on a 32-core box and changing only the generator's threads took the result from 1.42M at 8 to 2.16M at 16, a 52% move with nothing on the server side touched. Both entries also sit well under the hardware they are given, 4251% and 4068% of 6400%, and they sit there together despite being unrelated implementations. Two different servers agreeing on how much of the box to leave idle points at something they share. ASYNC_THREADS defaults to 128 here, against the 64 the profile has been running, to find out which side of the curve the bench hardware is on. If throughput does not move, the generator was not the constraint and the remaining 3.5ms of queueing above the 10ms wait belongs elsewhere. If it does move, the profile has been reporting the generator as much as the server and the number needs revising before anyone reads a ranking off it.
Both were tested on the bench hardware and both were wrong. The generator was not the constraint. tokio is the control - unchanged code, only the thread count moved - and it went 2,366,578 to 2,364,326 across 64 to 128 threads. A tenth of a percent. So the ~3.5ms of queueing above the 10ms wait does not belong to the harness, and the earlier 52% swing on a 32-core box was that box being oversubscribed rather than anything about the profile. That control also isolates the other change, and it cost throughput: ioxide went 2,179,118 to 2,089,696, down 4.1%, with the thread count accounted for by tokio sitting still. The reason is a hole in the idea rather than a tuning matter. Draining on the hot path never cancelled the armed timerfd, so the timer still fired and still cost its arm and its read; every pass of the handler loop simply added a peek on top. It could only ever be more work. Cancelling the timer to make the drain worth having would cost a syscall of its own, which is the thing being avoided, so there is nothing to salvage here. Back to the per-reactor timer alone, which measured 2,179,118 at 4251% CPU - 31% better throughput per unit CPU than the per-request timerfd it replaced, and level with tokio on CPU where that was 40% above it. ASYNC_THREADS stays as a knob, documented and defaulted to THREADS' 64, since knowing the generator is not the constraint is worth being able to re-check.
Both entries leave a third of the box unused and they leave it together, which
has looked like a cap all along. The arithmetic says otherwise: the profile is
closed-loop, so work is capped at connections over delay, and at the ~19.5us of
CPU a request costs, running the 3.20M ceiling would take about 6240% of the
6400% available. ioxide sits at 2.18M, 68% of the ceiling, using 4251% - which
is 68% of that. On those numbers nothing is blocked; there is simply no more
work to be had.
That is a prediction rather than a proof, and halving the cores tests it. 16
cores with their SMT siblings is 32 logical CPUs and a 3200% budget, which at
19.5us per request buys about 1.64M.
drops to ~1.6M and pins near 3200% the server was never blocked, only idle
holds ~2.18M at ~3200% per-request cost halved, so the wider
set was losing CPU to contention
The second outcome is the interesting one and would mean the extra cores were
costing more than they returned - cross-reactor traffic, memory bandwidth or
SMT siblings fighting over a core. The first says the remaining throughput is
bought by cutting the latency tail rather than by adding CPU, which is where the
zrk numbers already point: p50 sits at 2.21ms against a 2ms wait while p99 is
152ms, so the mean that divides into the connection count is made almost
entirely of tail.
Experiment on the branch, not a proposal for the profile.
Halving the server's cores answered the question the idle CPU had been posing, and not the way the arithmetic predicted: tokio pinned its 3200% while ioxide did not. Two entries on identical hardware and an identical ceiling, one able to spend everything it is given and one not. So ioxide is blocked rather than out of work, and the model that said otherwise - work capped at connections over delay, everyone equally short of it - is wrong about ioxide specifically. The per-request timerfd is the mode that has ever driven its CPU up: 5988% at the full cpuset against the per-reactor timer's 4251%, for 2,336,243 against 2,179,118. That looked like waste at the time and was reverted on those grounds, since it bought 7% more throughput for 41% more CPU. On 16 cores the reading is different: a mode that can occupy the machine is worth more than a mode that cannot, and whether it can is exactly what is being asked. What separates them is where the wake-up comes from. The per-request timer hands every deadline to its own submission, so each one wakes the reactor on its own account. The per-reactor timer holds one deadline at a time, so a reactor with work due behind the front of its queue stays asleep until the front fires. That is the shape that would show up as a reactor unable to fill its core. IOXIDE_DELAY_MODE=queue and =tick still select the other two.
… measure it Sixteen cores separated the two existing modes cleanly, and neither is good enough: tokio 1,769,456 3236% of 3200% 18.3us/req ioxide queue 1,335,794 2678% of 3200% 20.0us/req ioxide ring 1,211,556 3192% of 3200% 26.3us/req Per request, the per-reactor timer is nearly level with tokio. It loses on occupancy: it holds one deadline at a time, so wake-ups arrive in bursts and the reactor sleeps between them with work already due, leaving 522% of the box unused. The per-request timerfd has the opposite problem - every deadline wakes the reactor on its own account, which fills the box, but a timerfd_settime per wait means the CPU goes into syscalls rather than into serving, and it delivers less while using more. Neither can have both, and IORING_OP_TIMEOUT is the shape that can: the deadline rides in the SQE, submitted with the batch the reactor was sending anyway. Every wait wakes the reactor on its own account, and nothing is spent to arrange it. ioxide has submitted IORING_OP_TIMEOUT for its own ticker since long before this, but never exposed it; Reactor.SubmitTimeout does, in 68 lines that reuse the existing slot allocation and completion routing. That is committed separately on the library's own branch as MDA2AV/ioxide#212. The package is vendored rather than published because this is a measurement, not a release. localfeed/ holds one 140 KB build and NuGet.config adds it beside nuget.org, so the runner restores it out of the Docker build context with no feed access; every other package resolves normally. Both go away with the answer - either the API ships and this becomes an ordinary version bump, or the mode loses and all of it comes out. Verified on the default path: /delay/10 answers in 10.11ms and /delay/200 in 200.20ms, 100 sequential waits on one connection are none early with a mean overshoot of 0.28ms, and 32 concurrent distinct delays complete in 69.3ms against 1088ms serialised. IOXIDE_DELAY_MODE=ring, =queue and =tick still select the other three.
…lace Sixteen cores were a diagnostic and they answered it. Every mode, CPU per request and how much of the 3200% each could actually occupy: tokio 1,751,948 3228% 18.4us ioxide native 1,422,729 3205% 22.5us ioxide queue 1,335,794 2678% 20.0us ioxide ring 1,211,556 3192% 26.3us The per-reactor timer is the cheapest of the three and still loses, because cheap per request is worth nothing when 522% of the box goes unused. The per-request timerfd occupies the machine and loses harder, because what it puts the CPU to is timerfd_settime. IORING_OP_TIMEOUT is the only one that occupies the machine and spends it on serving, and it wins ioxide's side by 6.5%. Restoring the cpuset because the constrained box has now said what it had to say, and because the mode that was chosen on it has more room to show on the one the profile actually uses. At 32 cores the per-reactor timer was never CPU limited - 4251% of 6400% - so its throughput was bounded by the idle rather than by the hardware, and that is the bound this lifts. At 22.5us a request, 6400% is 2.84M against tokio's 2,366,578, under a ceiling of 3.20M. That is arithmetic, not a result. The per-request cost may not hold once there are twice the reactors to keep fed.
All seven references go to 0.7.211, the 140 KB build in localfeed/ and the NuGet.config that pointed at it are deleted, and the Dockerfile restores the way it always did. That scaffolding existed to measure IORING_OP_TIMEOUT before the API shipped, the measurement is done and the API has shipped, so it goes. ReactorDelay is gone with it. It had grown four mechanisms - a userspace tick posting drains across threads, a per-reactor timerfd with a priority queue, a per-request timerfd, and the ring timeout - which existed to find out which one belonged in the entry. The bench hardware answered that, so the other three are history rather than code, and what is left is 60 lines against one submission. What replaces it is the RingTimer from MDA2AV/ioxide#213, kept local for now: ioxide.timer missed the 0.7.211 release even though the SubmitTimeout it needs did ship, so the class lives here and becomes a package reference when it is published. It is the same code either way. The wait is per connection and built on first use, so a connection that never waits never makes one - which is every connection on every profile but this one. One op in flight is all a connection needs, because the handler awaits before it reads again. Verified against the released packages: /delay/10 answers in 10.02ms and /delay/200 in 200.09ms, 120 sequential waits on one connection are none early with a mean overshoot of 0.05ms, 32 concurrent distinct delays complete in 69.4ms against 1088ms serialised, and baseline is 3.75M.
ioxide.timer 0.7.211 is on nuget now, so the RingTimer that was carried here
while the package was missing from the release becomes a package reference and
the file goes. Same code, one fewer thing to keep in step.
ReactorDelay.cs is deleted rather than shrunk. It was 552 lines at its largest,
holding four mechanisms - a userspace tick posting drains across threads, a
per-reactor timerfd with a priority queue, a per-request timerfd, and the ring
timeout - which were there to find out which belonged in the entry. The bench
hardware answered that, so the answer is a package and the other three are
history. The entry now says what it wants and nothing about how:
await (httpSession.Timer ??= new RingTimer(reactor)).DelayAsync(ms);
One timer per connection, built on first use, so a connection that never waits
never makes one - which is every connection on every profile except this one.
Verified against the published packages: /delay/10 answers in 10.17ms and
/delay/200 in 200.28ms, 120 sequential waits on one connection are none early
with a mean overshoot of 0.11ms, 32 concurrent distinct delays complete in
68.9ms against 1088ms serialised, /json/1 and a 1 MB upload are unaffected, and
baseline is 3.85M.
MDA2AV
force-pushed
the
feat/ioxide-async-delay
branch
from
August 28, 2026 15:15
d01b071 to
172499a
Compare
Owner
Author
|
/benchmark-multiple -f ioxide,tokio -t async |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFrameworks: 2 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 32000 | 2,504,680 | 5519.0% | 1.4GiB | +7.2% | +7.7% |
✅ tokio
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| async | 32000 | 2,365,701 | 4106.7% | 1.4GiB | NEW | NEW |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Subscribes the ioxide entry to the
asyncprofile, and retunes the profile itself. Both are here because #1342 was folded in on request.The entry
ReactorDelaygives the profile's wait a home that does not drag the connection off the reactor that owns it. Two mechanisms, selectable withIOXIDE_DELAY_MODE:timerfdarmed per request and submitted to the reactor's own ring viaSubmitRead, so the kernel holds the deadline and the completion arrives on the reactor thread. This is howioxide.pgalready waits on the Postgres socket.Measured on the bench hardware at 16,000 connections and 5ms:
The bugs found along the way, each measured rather than guessed:
Registerran on every call. Worth +48% on the bench box (1,059,674 to 1,568,519), and it scales with reactor count, which is why a 32-reactor box only showed +13%.TaskCompletionSourceand itsTaskwas 354 MB/s of garbage, driving ~12.5ms gen0 pauses against a 15ms delay. Pooled onto the session; p99.9 halved.SpinOnce(-1)never yields, so it burned 87% of a core whether it posted anything or not - the busiest thread in the process. Now below 1%.EAGAINas expected: the fd is reused per connection, so a read could pick up an earlier request's expiration and return a valid 8 bytes. ~0.1% of waits, worst case 14.9ms into a 15ms delay. The deadline now decides, and an early completion goes back on the ring.Things tried and reverted because they did not pay:
RecvSlots256 to 1024, andRingEntries/ListenBacklog/PoolMax/RecvQueueEntries, none of which moved anything beyond the run-to-run spread.The profile
64,000 connections at 15ms becomes 16,000 at 5ms. Instrumenting the timer against its own deadline showed it completing 3.2ms late against 23ms of end-to-end overhead, so at 64k the profile was reporting mostly the read/write path rather than the wait. The gap between entries narrowed from 44% to 11.5% on the smaller shape.
Ceiling moves 4.27M to 3.20M. Against a best-so-far of 2.24M that is real headroom but less than before, and the docs now say so instead of claiming the ceiling does not bind.
Touches
profiles.sh, the gcannon raw template, the CATALOG row, both profile docs and the README table. Generator runs clean, badge parity holds at 579 ranks.Existing
async-64000results are orphaned by the retune, so the column is empty until the profile is re-run for all five subscribers. The results committed here predate the retune.