Skip to content

Split private_pension_wealth out of corporate_wealth on the UK WAS stage (#750) - #804

Closed
juaristi22 wants to merge 5 commits into
mainfrom
750-pension-wealth-split
Closed

Split private_pension_wealth out of corporate_wealth on the UK WAS stage (#750)#804
juaristi22 wants to merge 5 commits into
mainfrom
750-pension-wealth-split

Conversation

@juaristi22

Copy link
Copy Markdown
Collaborator

Implements the microcosm side of #750 (uk-data#452 mechanism M2) — the measured lever for the 8 UC caseload/two-child cells blocking the first certified cut (#796). Stacked on #803 (the #792 band-edge fix; its two commits lead this branch and drop out of the diff when #803 merges). Model side already released: policyengine-uk 2.92.0 (policyengine-uk#1824).

The split

The WAS stage folded WAS private pension wealth less current-employment DB (totalpenr8_aggr − dvvaldbt_scaper8_aggr) into corporate_wealth, which policyengine-uk counts as capital in every means test — on the incumbent at 2025, 46% of UC-reporting benefit-unit records fail the £16,000 test, 96% of them on corporate_wealth alone. The stage now emits private_pension_wealth as its own household output: drawn first in chain segment 2, the share-like holdings conditional on it, segment 3 conditional on both. corporate_wealth keeps employee shares/options, UK shares, unit/investment trusts, and the stocks-and-shares-ISA fold. Row identity old corporate_wealth == corporate_wealth + private_pension_wealth holds on the donor by construction and is pinned in tests, with a segment-1 containment test documenting why the first segment's draws cannot move.

Commits:

  1. Engine pin bumppolicyengine-uk>=2.92 in the three uk extras (locked upgrade, 2.89.0 → 2.92.0, zero transitive churn). Required first: an older engine's loader silently drops the new column, narrowing total_wealth and the exposure keys. 2.92's corporate_sector_wealth keeps the shareholding / corporate-land-value / employer-NI keys whole, so in classification terms only the capital tests move.
  2. The split — stage + lockstep spec pair + support bounds (operator-measured on the pinned tab, sha 18b3eb98…: donor pensions − db_pensions min 0.00 over 15,128 rows, zero negative rows → nonnegative_outputs; outward bound [0, 8000000], byte-verified via the tool's --check; the standing Port the WAS wealth imputation and regional property uprating as declarative source stages #714 SDC adjudication extends to the one new column) + export allow-list + stage-support gate columns/zero-allowances + coverage-manifest family outputs + the moved digests (battery policy/manifest/fingerprint, spine and release_cut certification part mirrors, UK spec bundle sha). The QRF household-tail note records the 13th unarmed household column; arming stays the WS-E spine deferrals: known bugs and signed inconsistencies awaiting their increment #796 follow-up.
  3. Evidence re-record under 2.92 — the bump alone (before any split) invalidates the committed licensed extraction evidence: 2.92 newly recognizes three loader inputs the pinned artifacts already carry and populate (bus_fare_spending, employment_sector, sic_industry_division — previously in the reference's "unknown export columns" list). The parity reference, the known-gaps candidate evidence, and the coverage manifest are regenerated with their own tools from the same pinned artifacts (identities unchanged — filename/revision/sha256 all identical; every pre-existing share value byte-identical): surface 145 → 148 required, counts and version strings move, nothing else. The June candidate and the spine both carry all three columns, so the widened contract is satisfiable on both sides. This is not the efrs-post-calibration input-mass reference re-pin — that stays deferred behind the uk-data mirror per the UK WAS stage: split private pension wealth out of corporate_wealth (uk-data#452 M2) #750 ordering, and the input-mass totals evidence (fd41cb5f…) is untouched.

What this deliberately does not do

  • No input-mass exclusion for corporate_wealth: the gate's reviewed tolerance is 4.5218 and the post-split drift is ~−0.85, comfortably inside — and the UK wrapper fails stale exclusions, so adding one would fail the gate. The drift gets recorded in the seam receipt when the split spine runs.
  • No signed-differences re-mint yet: was-wealth-qrf-incidence bounds and the private_pension_wealth net-new-column entry are re-minted from the spine-j measurement in the licensed phase (register entries unused on a compared surface fail --strict, so they cannot precede the spine that carries the column).
  • No doctrine or seed changes; the stage is row-preserving.

Licensed phase (rides this branch before review completes)

Spine-j + twin build under the armed battery, partial engine-drift control (segment-1 shares vs spine-i; a full commit-1-only control spine only if segment 1 moves), strict-parity re-mints, e4–e8 ladder, acceptance re-cut, then the seam run with the #792 fix in place — the 8 blocked UC cells' new relative errors are the adjudication input for the #736 queue — and the measurement receipt experiments/452-uk-pension-wealth-split.md. The certification cut itself stays with #793/#736 adjudication per the 2026-08-27 ruling.

Verification

Per-commit: ruff, ci_test_groups.py --verify, targeted UK/spec suites; full build+data shard after the spec changes; licensed bounds --check byte-identity; the two licensed-cache regeneration tests (skipped in CI, run locally against the pinned HF-cache artifacts) green after commit 3. The branch is main-based, so PR CI runs the engine lanes and the wheels gate once, authoritatively; local heavy suites were not duplicated (per the 2026-08-27 compute ruling) — the full build+data shard ran locally at the pre-rebase tip, and the post-rebase delta (the #803 wheels-lane reorder) was covered by its three calibration test files plus a wheel metadata build of all five shards.

🤖 Generated with Claude Code

juaristi22 and others added 5 commits August 27, 2026 18:05
Banded measures took each band's upper edge from the lower edges of
sibling specs in the live registry, so excluding a band silently widened
its lower neighbour to the next surviving edge (or to infinity at the
top) — the #792 movers: 12 of 361 shared measures shifted by up to +858%
when the exclusion register grew 5->47, each by exactly the mass of the
absorbed excluded siblings. materialize_target_bindings and
resolve_target_measures now accept a band_edge_registry (default: the
materialized registry, bit-identical for existing callers), and
_band_bounds refuses a spec whose own lower edge is absent from the
supplied edge set rather than guessing. Regression fence: survivors must
be bit-identical under sibling exclusion, for numeric edges and
published range labels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The seam prunes measure exclusions before materialization, so the stage
only ever saw the pruned registry. run_uk_calibration now takes the
compiled pre-exclusion register as band_edge_registry and refuses, before
any artifact is written, a non-empty exclusion receipt without it — and
a register whose extra names do not exactly reconstitute the receipt
(the applier already fails exclusions that match zero specs, so
pruned + receipt == compiled is exact). The registry content hash is
recorded as run_config.band_edge_register_sha256, flowing into the build
record and identity digest. The seam tool passes compilation.registry;
census counts are unchanged — only survivors' measured values return to
published band widths. The stale prose in test_uk_measure_simulation
that read the widening artifact as the resolution mechanism is corrected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2.92.0 carries private_pension_wealth (policyengine-uk#1824): the
explicit capital disregard, the total_wealth re-sum, and the
corporate_sector_wealth allocation key. The #750 split emits that column
from the WAS stage, and an older engine's loader would silently drop it,
narrowing total_wealth and the exposure keys. Locked upgrade moves
policyengine-uk alone (2.89.0 -> 2.92.0, no transitive churn).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The stage folded WAS private pension wealth less current-employment DB
(totalpenr8_aggr - dvvaldbt_scaper8_aggr) into corporate_wealth, which
policyengine-uk counts as capital in every means test - the largest
mechanism behind the UC caseload shortfall (uk-data#452 M2). The stage
now emits private_pension_wealth as its own household output, drawn
first in the second chain segment with the share-like holdings
conditional on it and the third segment conditional on both;
corporate_wealth keeps employee shares/options, UK shares, unit and
investment trusts and the stocks-and-shares-ISA fold. Row identity
old corporate_wealth == corporate_wealth + private_pension_wealth holds
on the donor by construction and is pinned in tests, alongside a
segment-1 containment test documenting why the first segment's draws
cannot move. Lockstep spec pair, support bounds (operator-measured on
the pinned tab: min 0.00 over 15,128 rows, outward bound [0, 8000000],
byte-verified via the tool's --check), export allow-list, stage-support
gate columns and zero allowances, coverage manifest, and the moved
digests (battery policy/manifest/fingerprint, spine and release_cut
certification part mirrors, UK spec bundle sha) re-cut in the same
change. The QRF household-tail note records the 13th unarmed household
column; arming stays the microcosm#796 follow-up.

Closes microcosm#750 machinery; the spine rebuild, parity re-mints and
seam measurement ride the follow-up commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2.92 engine newly recognizes three loader inputs the pinned
artifacts already carry and populate - bus_fare_spending (household),
employment_sector and sic_industry_division (person), previously in the
reference's unknown-export-columns list - so the committed extraction
evidence recorded under 2.89 no longer matches its own regeneration.
The parity reference, the known-gaps candidate evidence and the coverage
manifest are regenerated with their own tools from the same pinned
artifacts: identities unchanged (filename/revision/sha256 identical),
every pre-existing share value byte-identical, surface 145 -> 148
required, count pins and version strings move, nothing else. The June
candidate and the spine both carry all three columns, so the widened
contract is satisfiable on both sides. This is not the
efrs-post-calibration input-mass reference re-pin - that stays deferred
behind the uk-data mirror per the #750 ordering, and the input-mass
totals evidence is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22 juaristi22 closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant