HMT distributional analysis (mode 2, distributional): Budget 2025 external side — chart-only scope, honestly

[vahid-ahmadi]

First slice of the HMT DA lane. The headline finding shapes the whole PR: HMT publishes no machine-readable decile values. The Budget 2025 "Impact on households" document is a 20-page PDF whose Figures 1.A–1.C are unlabeled chart bars; the supporting-documents page carries XLSX only for the costings tables (4.1/4.2), and no chart-data file exists. So this PR deliberately ships the external side without value claims rather than digitizing charts by eye.

What's here

• sources/hmt-distributional/raw/ — the real publication, fetched 2026-08-17 from the gov.uk assets URL, SHA-256 9f7e68f3…480ea pinned in README, source.json, the registry and the tests.
• data/uk/hmt_da_packages.yaml — the Budget 2025 package entry: all 30 in-scope measures verbatim from pp. 7–9, each with channel (16 tax / 9 welfare / 5 benefits-in-kind) and computability triage (3 expressible / 9 partial / 18 not_expressible, each partial naming what's missing and each not_expressible why), the published exclusions (employer NICs incidence, NLW, devolved decisions, behavioural effects…), the counterfactual (no policy changes since AB2024), horizon (2028-29) and income concept (equivalised net BHC). No invented reform dicts: expressible components reference measure families for the OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54/OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56 registry, so one measure keeps one executable home.
• sources/hmt-distributional/adapter.py — verifies rather than parses: SHA gate, page count, figure inventory, and every registry component title verbatim-anchored in the document text (normalized for PDF line breaks and typographic quotes), so the registry cannot drift from the publication. Emits a meta artifact recording value_claims_emitted: 0 and the value-availability rule. Runs clean against the vendored PDF.
• tests/test_hmt_da_adapter.py — raw-file identity, provenance files, the no-value-emission contract, meta honesty; registry schema checks via importorskip("yaml") (CI's bare-pytest env skips those five assertions; the adapter enforces the same contract on every execution).

What this defers, and to what

• Per-decile external values: enter only if HMT releases data tables or a documented chart digitization lands with per-bar provenance (the registry's value_availability_rule states this as a contract).
• PE-side decile counterparts: blocked on the Adjudicate the UK ingest architecture: #48 vs #32, then re-gate the blocked UK lanes #57 ingest adjudication and the OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54 measure registry for package composition — the components list here is the composition spec they'll consume.

Suite: 167 passed / 5 skipped; ruff format clean.

Builds #61.

🤖 Generated with Claude Code

[DTrim99]

Review — HMT distributional analysis (mode 2, chart-only)

The central call here — shipping the external side without value claims because HMT publishes no machine-readable decile data — is the right one, and it holds across every file: no fabricated or eyeballed decile value appears anywhere. Every number in hmt_da_packages.yaml sits inside a verbatim HMT measure title (rates, dates, "450,000 children" — HMT's own text), never a per-decile £/% chart read, and value_claims_emitted: 0 is asserted in tests with no emit path for value rows by construction.

Verified:

• SHA-256 pin is consistent — one 64-hex hash across meta.json:10, yaml:63, README:394, test:465 (and source.json deliberately omits it, which test_lane_provenance asserts). It's actually checked: adapter.py:319-321 recomputes-and-compares and test:470 recomputes the raw file hash.
• Counts sum both ways — channel 16 tax / 9 welfare / 5 BIK = 30; triage 3 expressible / 9 partial / 18 not_expressible = 30 (both match meta.json).

Should address

• The strongest honesty guarantees aren't in CI. The verbatim-title, figure-presence, and "20-page" checks live only in adapter.py (needs pypdf) and run outside the default pytest path — so tests/test_hmt_da_adapter.py doesn't enforce that titles are actually verbatim or figures actually present; only a manual uv run of the adapter does. A pypdf-gated test (importorskip) mirroring the registry-schema test at :499 would catch drift in CI rather than locally.

Minor

• meta.json:32 has no trailing newline (cosmetic). The measure_family strings referencing OBR policy costings (mode 2): Policy Measures Database + EFO 3.17 → PE-UK static counterparts #54/Baseline as a first-class attribute of every score #13 are unverified pointers — expected for a first slice.

Reviewed with Claude Code assistance.

[vahid-ahmadi]

Addressed in the latest commit:

• Honesty guarantees now in pytest — test_document_anchoring_in_ci_when_pypdf_available loads the adapter and enforces the verbatim-title anchoring (all 30 components), figure anchors, and the 20-page identity against the real committed PDF, gated with importorskip("pypdf")/importorskip("yaml") exactly like the schema test. Verified passing locally under uv run --with pypdf --with pyyaml (6/6 in the file); it skips in CI's bare env — same limitation as the yaml-gated test, and the reason the adapter's run() remains the local backstop. If CI ever grows pypdf/pyyaml, the guarantee turns on with no further change.
• meta.json trailing newline added.

Full suite: 167 passed / 6 skipped; ruff clean. The unverified measure_family pointers stay as-is per your note — they resolve when the #54 registry merges.

[DTrim99]

Re-review — addressed ✅

The pypdf-gated CI test I asked for is in: test_document_anchoring_in_ci_when_pypdf_available importorskips pypdf+yaml and enforces all three guarantees in CI — the 20-page identity, every FIGURE_TITLES anchor present, and verbatim titles (empty unanchored set). Skips gracefully in a bare env with the adapter run as local backstop.

SHA-256 pin, value_claims_emitted: 0, and the 16/9/5 = 30 / 3/9/18 = 30 counts all still hold. Nothing left from my side.

[DTrim99]

Re-reviewed — addressed: the pypdf-gated CI test now enforces the verbatim/figure/20-page guarantees, and the SHA pin + counts still hold. Approving.

[MaxGhenis]

Gate round 1 — the chart-only honesty holds; six findings before rows can land

Verified clean: extraction is text-layer anchoring only — no OCR, no digitization, no numeric values at claimed precision — so there is not a single fabricated-precision row today; the vendored 20-page PDF's SHA-256 matches all four pins; and the current GOV.UK attachment list corroborates that the distributional analysis is PDF-only.

1. Blocking — the lane vanishes from post-The database leaves git: derived artifact, built in CI, published to Supabase storage #74 builds. The adapter writes standalone metadata, not ExternalScore rows; nothing registers in build_db.py; no lane entry exists. A fresh CI-built DB contains no trace of HMT — not even a zero-claim lane.

2. High — chart omissions aren't exactly tallied. The PDF plots 3 figures × 11 groups × 4 series = 132 values; the registry lists figures/series but not groups or expected cells. The accounting to enforce: 132 source marks = 0 emitted + 132 chart_not_digitized, by figure/group/series. (The 30 policy-component accounting does reconcile: 3+9+18 = 16+9+5 = 30.)

3. High — decile identity is prose. "Equivalised net household income, before housing costs" needs to become closed data identity: enumerated deciles in the income_group/quantile vocabulary via a deliberate uk_aliases extension, with housing_costs=bhc and equivalisation pinned as conditions (the HBAI/UKMOD pattern) — and no aliasing of HMT deciles to UKMOD quintiles.

4. High — baseline identity isn't machine-enforced. One prose counterfactual covers all charts and is omitted from emitted metadata — but 1.A/1.B are changes vs the no-policy package world (public services vs Spring Budget 2024 plans) while 1.C is a post-policy level. Without a registered world + ReformRef.baseline + mirrored baseline_policy, future HMT rows silently default to current_law.

5. High — the advertised PDF-anchoring gate is optional in CI. The document checks importorskip pypdf/pyyaml, and CI installs only pytest — so the anchoring guarantee never actually runs there. Install the deps and remove the skip path. Also: the adapter omits the trailing newline the committed metadata has, so wiring into the no-drift build would immediately dirty the tree.

6. Medium — add the deliberate held-out relationship entry (with evidence) before numeric rows land — the resolver currently fails closed on the unknown source, which is correct but temporary; and the "engine defect may be inferred after decomposition" guidance should point at the citable-known-issue gate instead.

🤖 Generated with Claude Code

[vahid-ahmadi]

All six addressed in d376a7f (rebased onto post-#74 main).

1. Blocking — the lane. scorecard_db/ingest_hmt_distributional.py is the missing step: it emits no external_scores rows (deliberately, and it raises if the metadata ever claims otherwise), writes the lane row with the chart-cell accounting as its detail, and mirrors it into the feed under UK. Registered in build_db after uk_deductions.

2. The omission is tallied. You were right that "0 emitted" had no denominator — the registry listed figures and series but not the groups. The eleven income groups are now registry data and the adapter enumerates and checks every cell: 132 source marks = 0 emitted + 132 chart_not_digitized, by figure / income group / series, with the arithmetic raising if it does not close. The 30 policy components still reconcile 3+9+18 = 16+9+5.

3. Decile identity. The ten deciles plus the all-households bar are a closed uk_aliases vocabulary with housing_costs=bhc and equivalisation=modified_oecd pinned beside them, and HMT deciles are recorded as DISTINCT from UKMOD quintiles so nothing can alias them later.

4. Baseline identity. Per figure now: 1.A/1.B are changes against the no-policy-change world, registered in baselines.py with its paragraph citation; 1.C is a post-policy LEVEL and keys current law. The assignment lives in the registry, is emitted into the metadata artifact, and an unregistered chart baseline raises.

5. The CI gate. Correct — it never ran. CI now installs pypdf and pyyaml and the importorskip path is gone. The adapter also writes the trailing newline, so wiring it into the no-drift build no longer dirties the tree.

6. Relationship + diagnosis prose. The held-out entry is registered with its evidence before any numeric row can land, and the guidance now names the citable-known-issue gate (class + rationale + action_link) instead of saying an engine defect "may be inferred".

Suite 264 passed, two builds agree on content_hash, no-drift clean, ruff format clean.