Skip to content

Run the serialisation test suite - #29

Merged
mnot merged 1 commit into
mainfrom
claude/structured-field-tests-pr-114-1aad00
Aug 17, 2026
Merged

Run the serialisation test suite#29
mnot merged 1 commit into
mainfrom
claude/structured-field-tests-pr-114-1aad00

Conversation

@mnot

@mnot mnot commented Aug 17, 2026

Copy link
Copy Markdown
Owner

test/tests/serialisation-tests/ has never been run. Two things kept it out:
the Makefile's test/tests/*.json glob doesn't descend into the subdirectory,
and both harnesses assumed every case has a raw element, so aiming them at
those files died with KeyError: 'raw'.

What changed

Makefile — added SER_TESTS, passed it to test.py and
test_compat_full.py, and added it to the submodule-init rule so a fresh
checkout still bootstraps.

Both harnesses — the phase gating was keyed on must_fail, which conflates
two meanings: on a parse case it means parsing must fail, on a serialisation
case it means serialising must fail. It's now keyed on what the suite
supplies, as the suite README defines it — parse iff the case has raw,
serialise iff it has expected — and test_serialise understands that
must_fail without raw means ser() is required to raise. I checked the
shapes before relying on that: no parse case is missing raw, and none carries
both must_fail and expected, so the gating is equivalent for everything
that ran before.

One latent bug turned up on the way. test.get("canonical", test["raw"])
evaluates its default eagerly, so it blew up on the five non-must_fail
serialisation cases even though canonical was present. Now an explicit
conditional.

Verification

make test exits 0 with 2861 of 2861 in each harness — 544 cases added apiece
(378 key-generated, 124 token-generated, 33 string-generated, 9 number).

Tests that can't fail aren't tests, so I checked both directions by breaking
the library on purpose and confirming the new cases catch it:

  • rounding decimals through a float drops serialisation-tests/number.json to
    6 of 9, flagging 0.00250.003 and 9.99959.999
  • dropping the disallowed-character check in ser_key drops
    key-generated.json to 202 of 378

Both reverted; http_sf/ is untouched by this PR. make lint and make typecheck exit 0.

Note that five assertions in the parse suites fail but count as passes, because
the suite marks them can_fail — base64 bad padding and the syntactic max
and min dates at ±999,999,999,999,999. That's pre-existing and unrelated;
nothing in serialisation-tests/ is can_fail.

Context

This came out of reading httpwg/structured-field-tests#114, which adds decimal
rounding cases that distinguish decimal from binary rounding. If that PR lands,
serialisation-tests/number.json goes from 9 cases to 13 and http-sf passes
all of them, since the harness reads the JSON with parse_float=Decimal. It
would have gone on passing silently either way until now.

Separately, and not addressed here: ser_decimal converts a float argument
with Decimal(input_decimal), which captures the exact binary value. That
fails all four of the proposed cases and two of the five existing ones. The
test suite doesn't exercise it because the harness feeds Decimal, not
float.


Written by Claude Code (Opus 5) in an interactive session, at my direction and
with me reading each step's output. It started as a question about
httpwg/structured-field-tests#114 and turned up the fact that these tests were
never being run.

🤖 Generated with Claude Code

test/tests/serialisation-tests/ has never been exercised. The Makefile's
test/tests/*.json glob doesn't descend into it, and both harnesses assumed
every case has a "raw" element, so pointing them at those files died with
KeyError: 'raw'.

Gate each phase on what the suite supplies, as its README defines it --
parse iff the case has "raw", serialise iff it has "expected" -- rather
than on must_fail, which means "parsing must fail" on a parse case and
"serialising must fail" on a serialisation case. No parse case lacks "raw"
and none carries both must_fail and "expected", so the gating is equivalent
for everything that ran before.

Also corrects an eager default: test.get("canonical", test["raw"]) evaluated
test["raw"] even when canonical was present, which stayed harmless only
while every case had raw.

544 cases added to each harness. 2861 of 2861 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mnot
mnot merged commit 76f3116 into main Aug 17, 2026
5 checks passed
@mnot
mnot deleted the claude/structured-field-tests-pr-114-1aad00 branch August 17, 2026 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant