Skip to content

Temporarily disable entries that fail TCP fragmentation, and drop their results - #1289

Merged
MDA2AV merged 1 commit into
mainfrom
chore/disable-frag-failures
Aug 24, 2026
Merged

Temporarily disable entries that fail TCP fragmentation, and drop their results#1289
MDA2AV merged 1 commit into
mainfrom
chore/disable-frag-failures

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Aug 24, 2026

Copy link
Copy Markdown
Owner

From the full validation sweep in #1285. Nineteen entries fail the exhaustive fragmentation check — nine request shapes split at every byte offset, ~1,000 per entry — disabled, with 18 result files removed.

Severity is not uniform

worst shape entries
100% of offsets fishcake, helidon-production, helidon-tuned, ioxide, libreactorng, may-minihttp, minima, nestjs, ringzero, rust-epoll, vinyl-cache, web-framework-csharp
88–91% slim (91%), dart-io (88%)
1–9% hical (9%), effinitive (3%), bjoern (1%), dart-zig (1%), blackbull (1 offset of 140)

The first fourteen cannot parse a request that arrives in more than one read — a real HTTP correctness defect.

The last five are a different kind of defect: an edge case at one split point rather than a parser that cannot reassemble. bjoern and dart-zig fail 9 of 1047 offsets — exactly one per shape, so a single consistent boundary condition. blackbull fails one offset out of 140 and loses 26 result keys for it.

From the full validation sweep in #1285. Nineteen entries fail the
exhaustive fragmentation check, which splits nine request shapes at every
byte offset and requires the server to answer correctly regardless of where
the request is cut across TCP segments.

Twelve fail 100% of offsets in the shapes they fail -- fishcake,
helidon-production, helidon-tuned, ioxide, libreactorng, may-minihttp,
minima, nestjs, ringzero, rust-epoll, vinyl-cache, web-framework-csharp --
and slim and dart-io fail 91% and 88%. Those cannot parse a request that
arrives in more than one read.

The remaining five fail a narrow slice rather than the whole shape: hical
9%, effinitive 3%, and bjoern, dart-zig and blackbull about 1% (blackbull is
a single offset out of 140). They are included because the instruction was
every entry that fails the check, but they are a different kind of defect
from the twelve above -- an edge case at one split point rather than a
parser that cannot reassemble at all -- and are the ones to revisit first if
this looks too broad.
@github-actions

Copy link
Copy Markdown
Contributor

👋 Heads up! This PR modifies the following frameworks:

@MDA2AV MDA2AV changed the title disable entries that fail TCP fragmentation, and drop their results Temporarily disable entries that fail TCP fragmentation, and drop their results Aug 24, 2026
@MDA2AV
MDA2AV merged commit 1fa5354 into main Aug 24, 2026
3 of 22 checks passed
MDA2AV added a commit that referenced this pull request Aug 24, 2026
)

The last group from the sweep in #1285: twelve entries that fail checks
other than the static probe, so neither #1288 (build), #1289 (fragmentation)
nor #1290 (static reference-only) covers them.

  deno-websocket, fleck        reject non-upgrade GET /ws
  node-websocket               text echo, multi-message, dies during the run
  dogrider                     binary echo
  sark                         upload chunked, upload truncated body,
                               StreamSum over h2+TLS
  swerver                      HTTP/2 negotiation and h2c prior-knowledge
  wtx-grpc, wtx-grpc-tls       GetSum over h2c / h2+TLS
  iris, typev                  upload chunked, plus static
  zio-http, zix                upload chunked

Each fails something the entry claims to implement, so the numbers it
publishes for those profiles do not describe a working server.

zio-http is the only flagship here and the only one whose failure is a
single profile -- upload chunked. It is included because the instruction was
the remaining failures, but it is the one worth a second look before this
merges: it loses 23 result keys over one check.
MDA2AV added a commit that referenced this pull request Aug 24, 2026
all_entries() ends on a jq that exits non-zero for a disabled entry, so the
function inherited that status and `changed=$(all_entries)` aborted the whole
step under `bash -e` -- before a single line was echoed, which is why the
failure looked like nothing at all had run.

Whether it triggers depends on which entry sorts last and whether it happens
to be enabled, so it passed on the run that introduced it and failed on the
next one, after #1288, #1289 and #1291 disabled a batch of entries.

An explicit `return 0` on the function.
MDA2AV added a commit that referenced this pull request Aug 24, 2026
all_entries() ends on a jq that exits non-zero for a disabled entry, so the
function inherited that status and `changed=$(all_entries)` aborted the whole
step under `bash -e` -- before a single line was echoed, which is why the
failure looked like nothing at all had run.

Whether it triggers depends on which entry sorts last and whether it happens
to be enabled, so it passed on the run that introduced it and failed on the
next one, after #1288, #1289 and #1291 disabled a batch of entries.

An explicit `return 0` on the function.
MDA2AV added a commit that referenced this pull request Aug 24, 2026
* validate: check TLS quality and TLS fairness, not just ALPN

Until now the only thing checked about TLS was which protocol ALPN settled
on. Two questions went unasked.

What did this connection negotiate -- the fairness half, tls_posture_probe:

  * the certificate must be the one mounted at /certs. Every handshake costs
    the server one signature with it, and on this box RSA-2048 signs 3,052/s
    against 77,124/s for ECDSA P-256. An entry that quietly generates its own
    EC certificate gets 25x cheaper handshakes and nothing would have
    noticed. Verified against a synthetic entry serving its own EC pair.
  * TLS 1.3, against a client offering it. The 1.2 handshake costs an extra
    round trip.
  * one of the three TLS 1.3 AEAD suites, which rules out NULL, anon, export
    and RC4 making "TLS" free.
  * ALPN must never name a protocol the client did not offer. Selecting
    nothing is fine and common -- bun does it, and the client falls back.

The cipher is reported rather than required. The field is already split:
some entries pick AES-128-GCM, some AES-256-GCM, and those measure ~17%
apart on bulk encryption at static-file block sizes (11.1 vs 9.5 GB/s at
16 KB). Not every framework exposes cipher preference, so this is on the
record instead of invisible.

What will the server negotiate at all -- the quality half, tls_quality_scan.
A server can hand a modern client TLS 1.3 and still accept TLS 1.0 or a NULL
cipher from anything that asks. Fails on SSLv2, SSLv3, TLS 1.0, TLS 1.1 and
on NULL, anonymous, export, 64-bit/DES/RC4/MD5 and 3DES cipher lists; notes
obsolete CBC and AEAD-without-forward-secrecy, which are weak but shipped as
defaults by much of the field.

That half is testssl.sh rather than a hand-rolled sweep: it already knows
every suite, it is the reference tool, and -p -s answers in ~5s. Its full run
also produces an SSL Labs grade, but costs ~48s per port and caps every entry
at B on the self-signed certificate the harness mounts, so the graded run is
left for audits. Skippable with HTTPARENA_SKIP_TLS_SCAN=1, and it skips
itself rather than failing an entry when the scanner image is unavailable.

validate: bun 70 -> 79 passed 0 failed, caddy 71/0. bun draws the note --
it accepts CBC and non-FS suites; caddy accepts neither.

* docs: document the TLS checks on the json-tls validation page

The checks added alongside this were not written down anywhere. The json-tls
page now carries them, since that is the profile whose whole point is TLS:

  * the certificate must be the one mounted at /certs, with the numbers that
    make it matter -- RSA-2048 signs 3,052/s against ECDSA P-256's 77,124/s,
    and every handshake is one signature, so a self-generated EC certificate
    is a 25x discount on that step
  * TLS 1.3, because the 1.2 handshake costs an extra round trip
  * one of the three TLS 1.3 AEAD suites, and why the choice among them is
    reported rather than required: the server picks, the field is split, and
    AES-128 against AES-256 is ~17% on bulk encryption
  * ALPN may select nothing, but never something the client did not offer
  * the testssl.sh scan: what fails, what is only noted, how to skip it, and
    why the graded run is left for audits rather than the gate

static-tls, baseline-h2 and static-h2 get a short pointer instead of a copy
-- the checks run there too, and one description is easier to keep true than
four.

* validate: ask about obsolete protocols with openssl rather than a scanner

Same question, asked directly: offer SSLv3, TLS 1.0 and TLS 1.1, and the
NULL, anonymous, export, RC4, DES and 3DES cipher families, and fail when a
handshake actually completes. An alert, a reset or a timeout is a refusal.

OpenSSL will not make those offers at its default security level, so the
probes carry @SECLEVEL=0; a protocol this openssl cannot offer at all is
reported as unprobed rather than counted as refused.

testssl.sh agrees with these results and is what found the first real failure
here -- humming-bird completes a TLS 1.0 handshake with
ECDHE-RSA-AES256-SHA. What decides it for the gate is cost: ~70ms for the
whole set against ~5s, and nothing outside the base image to pull or hang.
testssl.sh stays the better tool for an audit, where its wider suite coverage
and SSL Labs style grade earn the time, and the docs say so.

Verified on a clean host: humming-bird accepts tls1 and tls1_1, bun and caddy
accept neither.

* board: a TLS shield beside entries whose TLS was checked and clean

A small green shield after the framework name, on the table rows and in the
detail modal, for an entry that passed the TLS checks: serves the certificate
the harness mounts, negotiates TLS 1.3 with an AEAD cipher, and refuses
SSLv3, TLS 1.0/1.1 and the NULL, anonymous, export, RC4, DES and 3DES cipher
families.

The verdict is earned rather than declared. validate.sh writes
site/data/tls/<entry>.json when it actually runs those probes, the generator
folds it into meta, and the board reads meta[fw].tls. It is deliberately not
a meta.json field: that would let an entry claim the shield without being
checked, which is the opposite of the point.

Three states, and only one of them shows anything. "pass" gets the shield;
"fail" and absent both get nothing, because most entries have no TLS profile
at all and a red mark against them would say something untrue.

Seeded with the eight entries verified on a clean host this session -- bun,
caddy, elysia, genhttp-11-ioxide, h2o-mruby, hono-bun and slimeweb pass,
humming-bird fails on accepting TLS 1.0 and 1.1. Every other entry stays
unmarked until it is checked.

Hidden below 640px, where the name column is tightest.

check_badge_parity.js: 533 ranks match.

* ci: keep the TLS verdicts a validation run produces

validate.sh writes site/data/tls/<entry>.json when it runs the TLS probes,
and the board turns that into the shield -- but nothing collected the files,
so a full sweep computed 177 verdicts and threw all of them away.

One artifact per entry rather than a commit from inside the job: 177 matrix
jobs writing one directory would race, and a validation run committing to the
PR it is validating is a loop nobody wants. Collected after the run and
committed deliberately.

Uploaded with if: always(), because an entry that fails its TLS checks is
exactly the verdict worth keeping.

* ci: stop the detect step aborting on a disabled entry

all_entries() ends on a jq that exits non-zero for a disabled entry, so the
function inherited that status and `changed=$(all_entries)` aborted the whole
step under `bash -e` -- before a single line was echoed, which is why the
failure looked like nothing at all had run.

Whether it triggers depends on which entry sorts last and whether it happens
to be enabled, so it passed on the run that introduced it and failed on the
next one, after #1288, #1289 and #1291 disabled a batch of entries.

An explicit `return 0` on the function.

* validate: always mount the static directory, and raise the watchdog to 800s

Two things the full sweep on this PR exposed.

/data/static was mounted only when an entry subscribed to a static profile.
benchmark.sh mounts it unconditionally (scripts/lib/framework.sh), and the
entries that read the directory at startup cannot boot without it: rage and
rails copy it with FileUtils.cp_r, userver builds an fs-cache from it. So
when #1290 dropped the static profiles from those three, "does not serve
static" quietly became "does not start" -- all three failed the sweep with
the container dead before the first check.

It is a read-only bind of a small directory and there is nothing to be gained
by leaving it out, so validate.sh now matches benchmark.sh and always mounts
it. rage goes from a dead container to 44 passed, 0 failed.

The watchdog goes 300s -> 800s. The entries with the widest profile sets --
humming-bird and the three web-framework-* entries -- were killed mid-run
rather than failing anything, so their results said nothing at all.

* board: drop the TLS shield; the tls_check badge replaces it

Two badges for one subject was one too many, and the green shield was the
weaker of the two: it marked TLS basics an entry got right by default, while
the gold badge marks a hardening bar an entry opted into and wrote code for.

Removes the renderer, its CSS, the three render sites, the meta field the
board read and the loader that filled it. validate.sh still writes
site/data/tls/<entry>.json -- the verdict is what the opt-in badge is built
on -- but the files themselves are no longer committed, since nothing here
consumes them any more.

It also repairs a real break. The modal insertion had produced

    '</h3>'+tlsShield(fw)<span class="badge b-'+t+'"

-- the +' was lost, which is a syntax error in the board's only script block,
so the page would not have rendered at all. The earlier check evaluated
data.js and the shield function in isolation and never parsed index.html,
which is exactly the gap that let it through. Both are parsed now:

    all script blocks parse
    badge parity ok — 533 ranks match
@tomas-langer

Copy link
Copy Markdown
Contributor

The explanation here is not exactly correct for Helidon.
There is a bug with Connection: close, not with fragmentation of requests.
It still fails, but because the connection does not close (as expected by the probe). It returns 200 correctly.

@Hical61

Hical61 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Confirmed on hical this one's a real bug, The chunked decoder was tracking the wrong offset across fragmented reads (phr_decode_chunked decodes in place, so the byte count it returns is decoded bytes, not consumed bytes, we were treating it as consumed). Splits that landed inside the chunk framing went down the wrong path and returned 400.
Fixed, and added a regression test that walks every split point on the chunked body. All chunked/SSE tests green. Thanks for catching this.

TOKUJI added a commit to TOKUJI/BlackBull that referenced this pull request Aug 26, 2026
`validate_frag.py` sweeps every offset once.  That is the right shape for
finding a defect and the wrong one for confirming a fix: the failure it
looks for is a silently wrong body, so a flaky pass and a correct one are
the same green line.

This hammers a single offset at 0/5/50/200 ms split delays.  The delay
sweep is the point — at 0 ms the kernel often coalesces the two writes, so
no fragmentation happens and a broken server looks fine.  That is exactly
what it showed on the offset from MDA2AV/HttpArena#1289: 0.33.1 wrong
120/120 once a real split occurs, but only 22/40 at 0 ms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TOKUJI added a commit to TOKUJI/BlackBull that referenced this pull request Aug 26, 2026
* bench(httparena): absorb upstream's changes into our entry

Preparing the upstream re-enable (MDA2AV/HttpArena#1324) surfaced three
things our copy had drifted on.

**The pin was 22 minors stale.**  `requirements.txt` installed
`blackbull[compression]==0.56.0` while the tree being benchmarked was
0.78.0, so the container and the wheel under test could disagree about
what "BlackBull" meant.  Now 0.78.0.

**`millionaire` was missing.**  Upstream's blackbull entry subscribes to
it and we did not, for no reason: it is a rate-pinned GET on
`/baseline11` — upstream's `zrk.sh` says "Same GET the baseline profile is
validated on, so nothing new has to be implemented to subscribe".  Added;
no route change.

**`validate-frag.py` is worth carrying.**  It splits nine request shapes
at every byte offset, which is how upstream found the defect that got the
entry disabled: 0.33.1 answered 200 with a body of `57` where `75` was
correct, having passed the first byte of a two-byte chunk to the handler
and dropped the second.  Wrong answer, not an error — the kind a
status-only check cannot see.  Verified against our own entry: 9 shapes,
1047 offsets, 0 failures.

Not absorbed: the upstream entry's narrower profile set (16 vs our 22) and
its `.dockerignore`.  Ours is deliberately wider — it carries crud and the
four gRPC profiles — and our `.dockerignore` already allow-lists `db.py`
and `grpc_bench.py`, which is the bug the upstream one still has.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* bench(httparena): a repeat probe for one fragmentation offset

`validate_frag.py` sweeps every offset once.  That is the right shape for
finding a defect and the wrong one for confirming a fix: the failure it
looks for is a silently wrong body, so a flaky pass and a correct one are
the same green line.

This hammers a single offset at 0/5/50/200 ms split delays.  The delay
sweep is the point — at 0 ms the kernel often coalesces the two writes, so
no fragmentation happens and a broken server looks fine.  That is exactly
what it showed on the offset from MDA2AV/HttpArena#1289: 0.33.1 wrong
120/120 once a real split occurs, but only 22/40 at 0 ms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
TOKUJI added a commit to TOKUJI/HttpArena that referenced this pull request Aug 26, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
MDA2AV pushed a commit to TOKUJI/HttpArena that referenced this pull request Aug 26, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
TOKUJI added a commit to TOKUJI/HttpArena that referenced this pull request Aug 26, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
MDA2AV pushed a commit to TOKUJI/HttpArena that referenced this pull request Aug 26, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
MDA2AV added a commit that referenced this pull request Aug 26, 2026
)

* ioxide: send the FIN when the request asked for Connection: close

#1289 disabled this entry as one of twelve that "fail 100% of offsets" in the
exhaustive fragmentation check. The diagnosis was wrong. Its parser
reassembles fragmented requests correctly at every split offset, with and
without pauses between the writes, and at 384 concurrent partial connections.

What it never did was close the connection. Not even when it put
"Connection: close" in its own response header: the socket was still open
twenty seconds later, and the process leaked a descriptor per request.

validate-frag.py reads to EOF and its shapes send `connection: close`, so it
blocked until its own socket timeout and reported

  timeout (server never answered or never closed)

which the sweep recorded as a fragmentation failure across all nine shapes.
Worth remembering when reading any 100%-of-offsets result: an entry that
answers correctly and never closes looks identical to one that cannot parse.

## Where the fix belongs

Here, not in ioxide. Connection: close is an HTTP decision and ioxide is a
transport that knows nothing about HTTP. The entry already made the decision
correctly -- HttpSession sets WantClose from a case-insensitive match, and
HandleAsync returns on it, which I confirmed: a second request on the same
socket is ignored. What it lacked was a way to act on it.

TcpConnection exposes no Close or Shutdown, on the pinned 0.4.169 or on the
current 0.7.210 -- conn.Close() does not compile against either, and
Dispose() compiles but does not close. It does expose the accepted
descriptor, which is enough.

SHUT_WR rather than close(2) on purpose: the descriptor's lifetime belongs to
ioxide's refcount, and closing it here would free a number the reactor still
holds, so the next accept could hand the same integer to somebody else.

GenHTTP.Engine.Ioxide solves it the same way -- its assembly references libc
and shutdown -- so this is the idiomatic answer for an HTTP layer on ioxide
rather than a workaround. An upstream TcpConnection.ShutdownWrite() would
still be better than every consumer reaching for P/Invoke.

## Result

  frag before:  0 passed, 9 failed, every offset a timeout
  frag after:   9 passed, 0 failed

Connection: close is honoured in every casing, whole or split; keep-alive
still reuses the socket across requests.

## Still disabled

52 other checks fail, in three groups, none related to this:

  * TLS is dead. The handshake passes -- mounted certificate, TLS 1.3, ALPN
    -- but no HTTP comes back over 8081 or 8443, just "Empty reply from
    server". A pristine build of the entry fails identically, so this is not
    the shutdown change. It takes json-tls, static-tls, baseline-h2,
    static-h2 and both h3 profiles with it.
  * POST /upload with Transfer-Encoding: chunked answers 0 rather than
    counting the body.
  * Static files do not follow the disk: replaced files keep serving the old
    bytes, so the entry is caching at startup rather than reading the mount.

Re-enabling needs those three fixed. This lands the close fix and corrects
the record on why it was disabled.

* ioxide: update to 0.7.210, follow the disk, count chunked bodies (#1339)

Continues the close fix. Everything except TLS now passes.

  before   frag 0/9, and 52 failures across four areas
  after    frag 9/9, 56 passed, 31 failed - every one of them TLS

## ioxide 0.4.169 -> 0.7.210

All seven ioxide.* packages, thirteen releases forward. Builds clean and
changes none of the behaviour below on its own; the fixes are the entry's.

## Static files now follow the disk

Both static paths were built once at startup and never looked at the
directory again. ioxide.file opens a descriptor per asset and reads
positionally off it, so replacing a file left the old inode pinned;
Precompressed went further and baked whole responses into memory with
File.ReadAllBytes. That is what makes them fast, and it is also why a
replaced file kept serving the bytes the process opened at boot.

The rules require the opposite - replace a file and the next response carries
the new bytes - so StaticRefresh stamps the tree and checks the stamp before
a static request is answered. Count plus newest mtime, not content, and
explicitly not size: the check replaces a file with one of exactly the same
length, so anything keyed on size would miss it.

Throttled to 250ms and guarded so one reactor does the walk while the others
keep serving the snapshot they hold. Twenty stats a few times a second is
nothing next to reopening per request, and it is well inside the 2s the rules
allow. StaticAssets.Reload() does the identity side; Precompressed rebuilds
into a fresh snapshot that is swapped as one object, so a reader mid-request
never sees a half-replaced dictionary.

  PASS [static file follows the disk]    (1 file replaced, served in 1s)
  PASS [static variant follows the disk] (3 files replaced, served in 1s)

## POST /upload with chunked encoding

Answered 0 for any chunked body. DecodeChunked copies the first 256 bytes
into a stack buffer so /baseline11 can parse an integer body, and reported
that buffer's fill as the length - so a body larger than the peek, which is
every real upload, measured as whatever fit, and the terminating chunk reset
it to nothing. It now counts every chunk as it decodes, separately from the
peek, and /upload answers with that.

  130,000-byte chunked upload -> "130000"

## What is left, and why it is not verified here

All 31 remaining failures are TLS: json-tls, static-tls, and both 8443
profiles. The handshake itself passes - mounted certificate, TLS 1.3, ALPN,
and the whole TLS quality suite - but no HTTP comes back over the socket.

The entry does the handshake in userspace and then relies on kTLS for TX:
outbound writes go out as plaintext and the kernel is expected to produce
the records. The tls kernel module is not loaded on this machine
(/lib/modules/6.14.0-37-generic/kernel/net/tls/tls.ko.zst exists but is not
loaded, and containers here see no tcp_available_ulp), so those writes leave
as plaintext on a socket the client believes is encrypted and it reports an
empty reply. Loading the module needs root I do not have here.

So TLS is untestable on this box rather than proven broken. It needs either
the tls module present on the bench host or a userspace TX path in
ioxide.tls. Confirmed not caused by these changes: a pristine build of the
entry fails TLS identically.

Left disabled. It subscribes to json-tls, static-tls, baseline-h2, static-h2
and both h3 profiles, so re-enabling needs TLS resolved or those profiles
dropped from its tests.
TOKUJI added a commit to TOKUJI/HttpArena that referenced this pull request Aug 27, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
MDA2AV pushed a commit to TOKUJI/HttpArena that referenced this pull request Aug 28, 2026
Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from MDA2AV#1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.
MDA2AV pushed a commit that referenced this pull request Aug 29, 2026
* blackbull: re-enable on 0.78.0

Bumps 0.33.1 to 0.78.0, which fixes the chunked fragmentation failure
from #1289, and re-enables the entry.

app.py is replaced: the previous one targets 0.33.1 and does not start on
0.78.0.  db.py is added for async-db, and the Dockerfile and .dockerignore
now carry it into the image.

The subscribed profile set is unchanged.

maintainers was empty; set to TOKUJI.

* Benchmark results: blackbull  [skip ci]

* blackbull: silence a startup warning the benchmark logs did not need

Every container start printed a seven-line UserWarning from BlackBull's DI,
once per process, and the entry starts four (cleartext, h2c, TLS H1, TLS H2)
— 28 lines per run, before a single request.

The warning is wrong. BlackBull flags cleanup written after a bare `yield`,
because an exception in the handler is re-raised *at* the yield and the
trailing statements never run. `get_db_conn` does not have that shape: the
bare yields were early exits for DB-less mode, and the yield that actually
holds a connection was already wrapped in try/finally. The check counted
statements later in the *file* rather than statements reachable after the
yield, so the wrapped yield below charged the early-exit branch above.
Reported upstream and fixed there; this entry pins 0.78.0, which still has it.

Rather than wait on a release, the provider is now a single yield inside one
try/finally, which is what it should have been: acquire into `conn` (leaving
it None in DB-less mode or on an acquire failure), yield once, release in
`finally` if there is something to release.

Verified against the released 0.78.0 rather than a local checkout: startup
output goes 7 lines -> 0, and all four paths behave as before — DB-less
yields None, an acquire failure yields None and releases nothing, success
yields the connection and releases it once, and a handler that raises still
releases exactly once.

* blackbull: cut the entry's comments to the density its peers use

Two files, one measurement.  Excluding linter directives (# noqa /
# pragma / # type:), the Python entries here sit at a 5.3% prose median
and a 2.6-13.2% range.  app.py had drifted to 31.2%.

Most of that drift was mine: resolving the last rebase pulled in main's
28-line module docstring on the reasoning that both sides' intent should
survive, which took the file from 19.8% to 31.2%.  main's own copy is
32.6%, so it was never the density to match.  The docstring's endpoint
table restated the routes directly below it; what is left says which
profiles are not subscribed and which listener runs on which port, which
the code does not say anywhere.

Also gone: a 14-line note on how BB_ACCESS_LOG wiring works (two lines
now say the part that affects this file), a 5-line rationale for
streaming the upload body, and the /baseline11-vs-/baseline2 docstring.

db.py is now only what async-db needs.  The crud helpers and the whole
Redis path went with it - crud is not subscribed, nothing called any of
it, and it was carrying most of the file's prose.  279 non-blank lines
to 88.

Pool sharing is now in the names rather than in a paragraph about it:
_SHARED_POOL / shared_pool() for the per-worker pool every client
connection on that worker draws from, lease_connection() for the
Depends provider that borrows one for a request and returns it, and
POOL_MAX_PER_WORKER for the per-worker share of the database's budget.

app.py 9.8%, db.py 13.6%, 10.3% combined - against starlette's 7.8% and
aiohttp's 7.6%, which keep their database code in app.py too.

Verified on the released 0.78.0 rather than a local checkout: startup
output stays at zero lines, and the provider's four paths are unchanged
(DB-less yields None, an acquire failure yields None and releases
nothing, success releases once, and a handler that raises still releases
exactly once).

* blackbull: record why the query is not prepared explicitly

The async-db implementation guide asks to "prepare the query once per
connection, reuse across requests".  That is already what this code does,
and the explicit spellings are both worse.

Measured against the seeded Postgres:

  conn.fetch(SQL, ...)                 0.555 ms/cycle, 1 server-side stmt
  await conn.prepare(SQL) per checkout 0.856 ms/cycle, 1 server-side stmt
  prepare in create_pool(init=...)     InterfaceError on second use

A statement prepared in `init` and stored on the connection is invalid the
moment the pool takes it back, so it raises "the underlying connection has
been released back to the pool" on the second checkout.  A first test held
one connection for fifty requests and passed, which proved nothing; the
failure needs an acquire/release between uses.

Plain conn.fetch(SQL, ...) already leaves exactly one prepared statement
on the connection after 200 acquire/release cycles - asyncpg's
per-connection statement cache is the mechanism the guide describes.
Re-preparing per checkout returns the same cached statement and only adds
the call cost.

Comment only; no behaviour change.

* blackbull: pin 0.78.1

0.78.0 wrote most of its container log: writes that kept going after the
peer was gone, and a half-close claim TLS never honoured.  Both are fixed
in 0.78.1.

* Benchmark results: blackbull  [skip ci]

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@MDA2AV
MDA2AV deleted the chore/disable-frag-failures branch August 30, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants