Complete the #757 deferrals: the exclusion pass, the measured U8 lever, the release-cut certification, and the re-armed thresholds

[juaristi22]

Completes the deferred pieces of #757 per the follow-up contract (issue comment 5427936411); originally stacked on #787, now rebased onto its merge commit and targeting main. Nine commits, each independently verified (ruff + ci_test_groups.py --verify + uk + shared-spec groups; US groups not run — no US file touched).

What lands

1. The uk_target_fit exclusion pass (adjudicated 2026-08-26)

42 windowed schema-2 exclusions on the calibration measure-exclusion register (approver + 2026-08-26 → 2026-11-26 window, apply-time enforced): the 11 £1m+ top-income channel cells, 3 SLC channels, 16 UC payment-distribution cells, the OBR welfare-cap pair, 3 ONS composition cells pending a relationship-to-head frame column (#791, filed with this PR), 6 sparse HMRC band cells, and obr.fuel_duties (universe scope; the ledger retarget stays chronicle-side, your queue). The register census test pins the class counts and the deliberate non-exclusions.

Measured effect (seam on the same spine, 1500 epochs / family_equal): loss 0.1362 → 0.0309, ESS 5022 → 5916, uk_weight_ratio fails → passes (1546.7 → inside the reviewed 1151.3), couple_no_children +34.2% → +0.03%, unrelated_adult's absorber multi_family unbound. Five of the six seam gates green.

2. The would_claim_uc lever run — measured, and re-frozen at 0.55

The pre-registered U8 lever ran as adjudicated: contract raised to 0.85, full spine rebuild (spine-h, 14/14 signed RC battery), strict parity signed (one new register entry + 15 bounds re-minted), seam re-run — then reverted on the receipts, which now live in the contract entry itself:

	0.55 + exclusions	0.85 + exclusions
dwp.uc.households	−44.9%	−44.9%
obr.housing_benefit	−0.0%	−52.4%
UC support (design-weight initial)	3.13m	3.81m (+22%, capital-test capped)

The raise moves claimants off legacy benefits (HB initial £2.08bn → £0.75bn — unreachable under the 10× weight cap), pollutes the QRF predictor surface through engine-computed household_net_income (the flip is payload-non-local: WAS/LCFS/ETB draws moved on ~1,700 households), and buys the UC cells nothing. The binding constraint on UC caseload is capital-test support (uk-data#452 mechanism 2 → microcosm#750), not take-up support. U8 stays frozen at 0.55 with the run recorded in the contract's source block.

3. B5 — the seam refuses shippability claims

--release-candidate and canonical release ids are refused on the calibration driver (the 5413502559 audit): the seam's 6-entry scoped battery can never sign a shippability claim. The release_candidate parameter leaves the seam runtime; the hand-written build-record shippable literal is replaced by a pointer to the certification artifact.

4. B5 — the release-cut certification producer

The 16 declared national preflight/terminal gates get their executable home back: tools/certify_uk_release_cut.py over uk_runtime/release_certification.py, a scoped release-candidate-strict battery over the calibrated candidate, evidence reconstructed from persisted artifacts — the spine sidecar (which now persists each fitting stage's FitWeightRecords: 38 records across 5 stages on the licensed build), the seam's diagnostics/build record, the recompiled register, and the per-run licensed input-mass reference.

The multi-part certification composes over the spine, seam, and release-cut reports: union to the full 35-entry declared set, no gap, no overlap beyond uk_aggregate_admin, per-part signatures and committed-spec scoped digests, full phase coverage, a closed identity join (spine report → sidecar → build record → diagnostics → candidate bytes), the doctrine and its receipted overrides recorded verbatim, and the rule-1 score receipt cross-pinned (the audit's third carried defect). Shippability exists only here. 14 hermetic tests: a green compose plus one refusal per audit point.

5. B5 — the contract side

microcosm-data verifies release_certification.json: exact field set, mirrored part scopes and scoped-manifest digest pins, full-manifest spec pins, the union partition, recomputed shippability (never read off the flag), the diagnostics join, and the release-key signature. The nine refusal points move from per-report to per-certification; test_gate_battery_contract_pins holds every new mirror in lockstep with the live producer.

6. B4 — uk_qrf_tail_concentration re-armed from the #686 L3 baselines

top_k 100 (grid anchor), max_top_share 0.9994670564654868 (the exact measured checked-surface maximum: hmrc_spi_other_social_security_income, 104 carriers on spine-a), min_nonzero_records 104 (the thinnest measured column above the anchor). The three saturated sub-anchor columns (12/22/24 carriers, top-100 share 1.0) go thin-visibly on every run. The gate notes record the measuring run — baselines file digest, measured artifact, defining column per threshold. Gate policy/manifest/fingerprint digests re-cut from the live producer payload. The 12 household-surface grids are measured but not armed (declared follow-up).

Phase 3 — where the first certified cut stands

The candidate-of-record moves to spine-i (rebuilt at this tip, payload-identical to spine-g — the entire increment is payload-inert at 0.55): 14/14 battery at RC strictness with a signed report, fit-weight sidecar, identity ladder e4–e8 green, strict parity signed_parity / 0 unsigned. The committed acceptance receipt + binder now describe it.

The Phase-3 seam attempt (uk-757-first-certified-cut) binds 361 targets at loss 0.0309 and blocks, correctly, at uk_target_fit on 13 characterized cells:

class	cells	disposition
UC caseload / two-child-limit	8 (−28.6%…−62.0%)	the U8 lever is measured and exhausted; the real lever is capital-test support (microcosm#750) — adjudication needed
exclusion-set-dependent measure artifacts	4 (self_employment 40–50k +128%, count 300–500k +40%, state_pension 40–50k +118%, private_pension_count 70–100k +66%)	#792 (defect, filed with this PR — see below)
sparse-band sibling	1 (dividend_income_count 500k–1m −36.7%; its amount sibling is excluded)	candidate 7th sparse-class exclusion — your call

So the certification, the uk_input_mass_parity flip to the cut line, the descriptor retirement (B4/C4 completion), and the #731 scorecard probe wait on those dispositions — exactly the fail-closed behaviour the machinery is for. Everything is staged: once the UC family has a disposition, the cut is a seam run + certify_uk_release_cut.py invocation away.

#792 — measure materialization is exclusion-set-dependent (found by this pass, controlled, filed)

Three seam runs on the same spine, same doctrine, differing only in the register: with the old 5-entry register all 403 initial estimates reproduce the rebind exactly (also proving this PR's seam changes measure-inert); with the 47-entry register 12 of 361 shared measures move, up to +858% (7 HMRC band cells, 5 UC payment-dist cells; the other 349 are bit-stable). Engine-side stochastic variables are materialized in register-dependent order. This must be fenced before any certified line binds on the surface — details and fix directions in #792.

Adjudication queue

1. The 8 UC caseload cells: exclusion, retarget, or wait for microcosm#750's capital-test split? (The lever disposition is exhausted by measurement.)
2. dividend_income_count_500k–1m: 7th sparse-class exclusion, or ride Band edges derive from the exclusion-pruned register, silently widening surviving banded targets #792's fix?
3. Band edges derive from the exclusion-pruned register, silently widening surviving banded targets #792 fix direction: per-variable RNG isolation vs materialize-full-register-always.
4. Doctrine posture for the cut: the certified run carries epochs 1500 / family_equal as receipted overrides (your 2026-08-24 ruling keeps uniform the default); the certification records them verbatim. Promote to doctrine, or certify with receipted overrides?
5. fuel-duties ledger retarget (household-incidence fact vs total receipts) — chronicle-side, carried from the follow-up contract.
6. Household-surface QRF tail gate (12 measured grids, not armed) — follow-up scope.

Deferred, by construction — tracked in #796

Every remaining spine deferral now lives on the WS-E spine-deferrals tracker #796 (sub-issue of #145, the spine counterpart of #736): the B4 completion / C4 retirement (uk_input_mass_parity flip to the first certified line + efrs-post-calibration descriptor retirement — cannot precede the cut, US #327 doctrine), the W8 certified-identity registry entry, the #731 scorecard probe and rule-1 scorer seal, the household-surface QRF tail gate, macro_realism, and the carried dates (owned_land exclusion expires 2026-09-26; measure-exclusion windows 2026-11-25/26). The calibration-side remainder — the 8 UC cells, the sparse sibling, #792's fix direction, the doctrine posture, the fuel-duties retarget — is checklisted on #736.

Verification

• Hermetic: ruff, ci_test_groups.py --verify, full uk + shared-spec groups green at the tip (US groups not run — no US file touched).
• Licensed (evidence in data/ukds/acceptance/757-swap/): spine-h + spine-i builds under the armed battery (signed, RC-strict, 14/14); payload receipts (g↔h isolates the lever, g↔i proves the increment payload-inert); identity ladders; strict parity receipts; the four seam runs (lever, counterfactual, RNG control, Phase-3) with signed gate reports; logbook rows for every attempt.

Closes #623, closes #686, closes #757.

With #787 merged, this PR completes what those three issues still owned: #623's seam gains its full gate story and doctrine-governed exclusion mechanics; #686's L3 baselines arm the thresholds they were measured for; #757's B4/B5 items land and its A-lane gaps carry adjudicated dispositions. The first certified cut — the epic bar — is staged and blocked only on the adjudications above; it is tracked as the capstone of #796 rather than holding these issues open. #791 and #792 were filed as successors with this PR.

🤖 Generated with Claude Code

[juaristi22]

Pre-merge sweep (2026-08-27): exercised the release-cut preflight's two ledger compile-parity gates against the pinned chronicle feed — the one live path this PR's runner had not yet executed — and both failed on register rot accumulated since the June fixtures froze: 13 stale entries (live compilation now matches the fixture: the scotgov council-tax stock bands, SCP spending, three SLC recipient rows) and 13 SLC entries whose kind moved fixture_only → calibration_drift when the SLC chronicle waves completed. Regenerated both signed registers from the feed with the packaged tool at 771a6c27 — zero entries added, so the live diff carried no unsigned differences and the regeneration is exactly the correction the gate's anti-rot refusals demanded. Re-verified: both gates pass against the pinned feed, and the digest-pin suites are green. The producer's first real preflight will not open on a known-stale register.

🤖 Posted with Claude Code

[vahid-ahmadi]

Automated review pass (Claude Code, high effort, diff only — no build or test execution). Four findings, and all four are the same class this lane has been closing all week: a check that reports success without having tested what it names. Given the PR's own framing — "a candidate's shippability verdict comes only from the certification" — that class matters more here than anywhere.

1. packages/microcosm-data/src/microcosm/data/contract.py:4359 — a release with no certification validates clean

The certification is validated only if certification_path.is_file(), and release_certification.json was not added to required_release_files(). So a release directory that simply omits the certification passes validate_release_dir without complaint.

That is the exact green-by-absence this PR sets out to close, one level up: the verdict is meant to come only from the certification, but a release with no certification at all is indistinguishable from a validated one. Adding it to required_release_files() makes the omission loud.

2. uk_runtime/release_certification.py::uk_release_parity_evidence — the two parity sides are keyed at different grains

candidate_targets = set(target_relative_errors) is keyed on bare str(row["name"]) from the diagnostics rows, while reference_targets is keyed f"{spec.name}@{spec.period}". Unless the diagnostics names already carry an @period suffix, the two sets are disjoint: every candidate target reports as missing and every reference target as extra — or, if the gate checks only one direction, it passes vacuously with nothing compared.

Worth confirming which of the two it is, because the failure mode differs sharply. If it fails loudly, it is a bug caught on first run; if it passes, the parity evidence in the certification is empty.

3. uk_runtime/release_certification.py::rehydrate_uk_fit_weight_records — an empty rehydration is not an absent one

A malformed or empty per-stage block returns (), and run_uk_release_cut_battery supplies the artifact whenever it is not None. So uk_weights_audit runs over zero records rather than hitting the evidence_absent gap that would have flagged the missing evidence. A non-Mapping record degrades silently to () instead of raising UKReleaseCertificationError.

The docstring's "never a vacuous pass" holds only if the audit binding itself refuses an empty tuple. Refusing at the rehydration boundary — an unparseable block is an error, not an empty result — seems the more direct fix, since it keeps the distinction between "no weights" and "weights we could not read".

4. uk_runtime/release_certification.py::_verify_score_receipt — the rule-1 cross-pin is a substring test

candidate_sha256 not in json.dumps(receipt) passes as long as the candidate digest appears anywhere in the serialised receipt — an inputs list, a provenance block, a nested pin. A receipt scored against a different artifact therefore satisfies the cross-pin, because the check never reads the field that names what was actually scored.

This is the strongest form of the class in the batch: the check looks like an authentication and reads like one, but a passing result carries almost no information. Comparing the specific field that records the scored artifact would make it mean what its name says.

Related dead code in compose_uk_release_certification: parts_raw[part_name][2] if len(parts_raw[part_name]) > 2 can never be taken, since _load_part always returns a 2-tuple.

---

1 and 4 are the ones I would resolve before merge — both are gates on the shippability verdict itself, and both currently pass in situations they exist to reject. 2 is worth a quick confirmation either way, since which failure mode it has determines whether it is urgent or already visible.

[juaristi22]

Review dispositions (all four findings confirmed in substance; fixed at 58829b62, together with the duplication consolidation the round invited):

1. Certification not required — confirmed, fixed content-keyed. A release directory shipping any national-line gate artifact (release_cut_gates.json, or a terminal_gates.json whose posture is calibration_seam) without release_certification.json now refuses with the omission named — the green-by-absence cannot validate clean. The literal required_release_files() entry you proposed needs a release-id family to key on, and the canonical national release-id form does not exist yet (the seam refuses the exact-k form; the Phase-3 run used an ad-hoc dev id) — that naming ruling plus the publication integration are queued on #796, and the id-keyed rule lands there. Until then the content-keyed refusal covers every directory that carries any part of the story.

2. Parity grain — confirmed working as built, now self-evident. The diagnostics writer labels rows at name@period grain ("name": "dwp.uc.households@2025", with bare target_name as a separate field — every packet in the 757-swap acceptance evidence shows it), which is exactly why the retired June runner's _stage_parity_evidence used str(row["name"]) against f"{spec.name}@{spec.period}" with a comment documenting the shared grain. So the sets intersect correctly and the gate compared for real on the live runs. But the convention was implicit, so it is now a loud refusal: uk_release_parity_evidence rejects any diagnostics row whose name lacks the period label, with a test on both branches.

3. Rehydration — half-confirmed, the real half fixed. The empty-tuple path was deliberate and is not vacuous: the UK weights-audit binding refuses an empty record set (_missing_fit_weight_evidence_gate, the legacy guard the shared binding lacks), so () is a failed audit — a fitting stage that recorded nothing must fail, not go evidence_absent. What you caught that was genuinely wrong is the conflation: a malformed block (non-list stage value, record missing its fields) degraded to that same (), erasing the difference between "no weights" and "weights we could not read". Malformation now raises UKReleaseCertificationError as corruption; only a genuinely empty per-stage list keeps the failed-audit semantics. Tests cover all three shapes.

4. Score cross-pin — confirmed, fixed exactly as proposed. The check now reads artifacts.candidate.sha256 — the field _verify_artifact fills from measured bytes, i.e. the receipt's own record of what was scored — and refuses anything else, including your exact scenario (the digest appearing in an inputs list or on the incumbent's pin), which has a dedicated test. The dead parts_raw[part_name][2] branch is gone.

The duplication sweep (María's ask, same commit): one scope-filtering implementation — uk_scoped_gate_manifest, with a source parameter so the spine driver passes the spec it already loaded (also the hermetic tests' stub point) — replaces the driver's local copy and the dead zero-caller _spine_gate_manifest; one finalize_uk_scoped_gate_report grafts posture/scope-exclusions/admin-receipt and re-signs for both the seam and the release cut, so the certification's parts cannot drift apart in shape; the certification's three private cross-module imports become public names; and GateBatteryRun now exposes gates_manifest_sha256 / policy_sha256 / spec_fingerprint as properties, so digest derivation stops round-tripping through a signed payload. All digest-identical by construction (same entries, phases, policy suffixes); the pin suites and both hermetic groups verify it.

🤖 Posted with Claude Code

[juaristi22]

Naming ruled and landed (cebc9f50): the certified national line's release id is microcosm-uk-2024-25-national — one constant name across cuts, since the Logbook chain and artifact versioning carry run identity and ordering; the 2024-25 segment follows the uk-data survey-vintage convention, and -national keeps the id disjoint from the exact-k -k<N> shape (and may retire as the migration moves on). With the id ruled, finding 1's id-keyed layer lands here rather than waiting on #796: required_release_files() demands release_certification.json for the national id, the seam refuses the shippable name outright, and a lockstep test holds the data-shard mirror to the build-shard constant. The content-keyed refusal from the disposition round stays as the belt for part-carrying directories under any other id.

🤖 Posted with Claude Code

[vahid-ahmadi]

Approving. All four findings fixed at 58829b62 (with the naming ruling at cebc9f50), 23/23 checks green. I checked each fix against the diff rather than the dispositions.

4 — score cross-pin. _verify_score_receipt now reads artifacts.candidate.sha256 and refuses anything else, with the failure message naming both the scored digest and the candidate under certification. This is the one I cared most about: the substring form would have accepted a receipt scored on a different artifact whenever the candidate digest appeared anywhere in the payload, and the check now reads the field that records what was actually measured. The fixture change in test_uk_release_certification.py is the tell — the old green fixture was {"candidate": {"sha256": ...}, "verdict": "scored"}, which the new check correctly rejects, so the test had been passing on a shape the gate could not really authenticate.

1 — certification required. The content-keyed refusal is the right call over the required_release_files() entry I proposed, and for the reason you gave: that rule needs a release-id family to key on, and inventing one mid-PR to satisfy a review comment would have been the worse trade. Refusing any directory that ships release_cut_gates.json, or a terminal_gates.json whose posture is calibration_seam, without the certification covers every directory that carries part of the story today, and the comment records why the id-keyed form is deferred rather than leaving a reader to wonder.

2 — parity grain. My finding was wrong; the fix is still worth having. The diagnostics writer does label rows at name@period, so the sets intersected and the gate compared for real — I read the two key expressions and inferred a mismatch without checking what the writer emits. What you have added is the better outcome anyway: an implicit convention shared across two modules is now a loud refusal, with the message explaining why a bare-name row would fall out of the comparison. That is a real invariant that was previously only a comment on a retired runner.

3 — rehydration. Splitting corruption from emptiness is exactly the right distinction, and I had the two conflated. A non-list stage value or a record missing fit_name / weight_kind now raises UKReleaseCertificationError as corruption, while a genuinely empty per-stage list still returns () so _missing_fit_weight_evidence_gate can fail the audit. Worth restating your framing since it is the part I got wrong: a fitting stage that recorded nothing must fail, not go evidence_absent, and the empty tuple is how that refusal is reached rather than a way around it.

On the duplication sweep. Promoting _scoped_gate_manifest / _resign_gate_report / _aggregate_admin_totals to public names and having one finalize_uk_scoped_gate_report serve both the seam and the release cut is the structural version of the property the old comment only asserted — that the two reports cannot drift apart in shape. Same for GateBatteryRun exposing the digests as properties instead of derivation round-tripping through a signed payload.

Scope of the approval, as before: I reviewed the diff and confirmed CI is green. I did not run the battery, the release cut, or the certification compose, so the digest-identical-by-construction claim is one I have read the reasoning for rather than reproduced.