UK WAS stage: split private pension wealth out of corporate_wealth (#750)

[MaxGhenis]

Part of #750 (tracking issue for PolicyEngine/policyengine-uk-data#452 mechanism M2). Branched from origin/main at 2aa9679; merged up to 2807e99 (#747). Draft — held for Max's go. Model-side companion: PolicyEngine/policyengine-uk#1824 (draft, lands first).

What moved

Stage arithmetic (uk_runtime/was_wealth.py): the WAS stage folded WAS private pension wealth less the current-employment defined-benefit component (totalpenr8_aggr − dvvaldbt_scaper8_aggr) into corporate_wealth, which policyengine-uk counts as capital in every means test. It now emits that component as its own household output, private_pension_wealth, and keeps corporate_wealth to employee shares/options, UK shares, unit and investment trusts and the stocks-and-shares ISA fold. Row identity on the donor: old corporate_wealth == corporate_wealth + private_pension_wealth.

Chain: segment 2 draws (private_pension_wealth, corporate_wealth_excl_isa, stocks_and_shares_isa) — the pension component first, in the slot the old folded column (84.7% pension by donor mass) occupied, the share-like holdings conditional on it — and segment 3 conditions on both components, so the downstream targets keep the information the folded column carried. Segment 1 (owned_land, property_wealth) is untouched and its draws are identical at seed 0 (verified on the re-imputation receipt).

Second commit — one child seed per chain segment (review finding): impute_was_wealth reused one RegimeGatedQRF, and start_chain respawns the fit/draw streams from the model seed on every call, so the k-th target of every segment consumed the same quantile and sign-gate uniforms per recipient. On train/hold-out halves of the donor the review measured P(shares > 0 | no property) at 0.011 against 0.055 observed; the production child seeds (was_wealth_segment_seeds, declared seed stays the root) recover 0.039 — the receipt was re-run with exactly the shipped derivation after cross-family review caught the first variant validating differently-truncated seeds, and the unit test golden-pins the tuple.

Lockstep: uk/spec/sources.yaml + uk/source_stages.json (derived, chain_order, outputs, nonnegative_outputs, notes); was_wealth_support_bounds.json regenerated from the pinned tab (private_pension_wealth [0, 8000000]; corporate_wealth's committed bound is unchanged at one significant figure); UK_ALLOWED_EXTRA_EXPORT_COLUMNS + gates.json allow-list gain household.private_pension_wealth in the cash_isa / stocks_and_shares_isa slot; release_input_coverage_manifest.json regenerated (source_stages.json sha pins and the was_wealth outputs list; 145 required / 0 exclusions unchanged); towncrier fragment; measurement receipts in experiments/452-uk-pension-wealth-split*.{md,json} (aggregates only).

Digests (stated, per the re-pin discipline; re-cut over the union on each merge of main — twice so far, past #744/#674/#754 and past #747, the d70ea39 pattern): currently UK spec_sha256 e3b171bd → e6594bd3; gate-battery policy → 61c1e22c, gates manifest → 409e2aa6, spec fingerprint → a47f0752 (microcosm-data contract + test mirror). They move again mechanically on the next merge of main.

Why this column and this name

Donor aggregates (WAS round 8, weighted £bn): total private pension wealth 4,807.7; current-employment DB (SCAPE) 1,148.7; remainder 3,659.0 = current occupational DC 313.9 + AVCs 12.2 + retained DC 550.0 + current personal pensions 203.7 + retained DB 863.6 + pensions in payment 1,692.4 + expected from a former spouse 23.3 (ten-component identity exact on 100% of rows; remainder never negative). Share-like holdings 659.4. So the folded component is not DB-free — retained DB rights and DB pensions in payment stay inside it — hence private_pension_wealth (WAS/ONS private pension wealth less current-employment DB, stated first in its documentation), not non_db_pension_wealth. Every part is "the value of a right to receive a pension under an occupational or personal pension scheme": disregarded capital under UC Regs 2013 Sch 10 ¶10, HB Regs 2006 Sch 6 ¶31–32 / HB(SPC) Sch 6 ¶24, JSA Regs 1996 Sch 8 ¶28–29, ESA Regs 2008 Sch 9 ¶28–29, IS Regs 1987 Sch 10 ¶23–23A, SPC Regs 2002 Sch V ¶22–23 (all fetched).

Licensed acceptance (aggregates only; scripts outside the repo)

The fixed-draw legal effect (cross-family review design; the causal number for the disregard): on the final code's own draws (per-segment seeds, seed 0, 100 trees, enhanced_frs_2024_25.h5 v1.56.16, weights unchanged), folding the drawn private_pension_wealth back into corporate_wealth on the same rows and recomputing UC at 2025 gives 4.87m benefit units counted vs 6.27m disregarded — +1.399m UK (+1.329m GB), spend +£11.7bn, eligible benefit units +6.90m, reporter records over £16k −1,453; GB elements +0.82 / +0.35 / +0.06 / +0.17 / +0.61m (housing / LCWRA / carer / childcare / child); anchored +1.456m. Every draw identical; only the capital-test classification differs.

Whole-implementation comparison (secondary; the pre-fix shared-seed redraw pair — OLD and NEW independently fitted and drawn, so machinery and classification move together, and no matched OLD′ exists for the final branch):

	incumbent artifact	OLD re-draw	NEW re-draw	NEW − OLD
UC benefit units, UK (m)	6.346	5.193	6.266	+1.073
UC spend (£bn)	75.3	63.1	71.7	+8.6
eligible benefit units (m)	10.52	9.77	16.26	+6.49
reporter records over £16k	2,958	3,165	1,757	−1,408
GB housing / LCWRA / carer / childcare / child elements (m)	4.87 / 2.10 / 0.64 / 0.47 / 3.02	4.08 / 1.70 / 0.56 / 0.26 / 2.34	4.59 / 2.05 / 0.65 / 0.40 / 2.87	+0.51 / +0.36 / +0.08 / +0.14 / +0.52
GB single / lone parent / couple no children / couple with children (m)	2.97 / 2.13 / 0.23 / 0.87	2.47 / 1.66 / 0.22 / 0.69	2.88 / 1.92 / 0.33 / 0.94	+0.41 / +0.27 / +0.10 / +0.25

With per-segment seeds (NEW′): 6.268m, eligible 15.57m, reporters over £16k 1,742. With FRS reporters anchored: OLD 5.22m → NEW 6.34m. The incumbent's own upper bound (counterfactual C, corporate_wealth → 0) is 6.88m. Defensible quotation for the redraw pair: at seed 0, with shared-seed microcosm redraws and unchanged incumbent weights, the old-stage-to-split-stage implementation moved the modelled 2025 UC caseload from 5.19m to 6.27m — not a fixed-draw legal effect (the table above is), not a recalibrated estimate, and not a national forecast.

Allocation-key check on the NEW draw: corporate_wealth alone puts 84–87% of its mass in the top 10% of households (27–29% in the top 1%); corporate_wealth + private_pension_wealth 54% / 14% — why the model keeps the preserved key for shareholding, corporate land value and the employer-NI response.

Caveats that travel with the numbers (full list in the receipt): weights not recalibrated (the UK surface now compiles a UC caseload target, #735); OLD is this package's draw, not the incumbent's (continuous-quantile full-leaf draws vs microimpute's 10-point grid; released-h5 predictor surface) — the OLD re-draw sits 1.15m below the incumbent artifact on the same households, a machinery difference registered on #750; private_pension_wealth is engine-unknown at policyengine-uk 2.91.0.

Gates

uk_input_mass_parity: corporate_wealth will report about −0.73 to −0.85 against the efrs-post-calibration reference (pension mass moved to a column the reference lacks); inside the armed tolerance 4.52, so no exclusion receipt (adding one would itself fail as stale). uk_export_surface: allow-list entry is the sanctioned mechanism; note the gate is evidence_absent in production builds at this pin (#686). uk_nonnegative_columns / uk_support pick the new column up from nonnegative_outputs and the regenerated bounds. Coverage manifest: required set unchanged (the new column is candidate-only).

Sequencing and engine coupling

1. policyengine-uk companion PR (inert on current data) → 2. this PR → 3. uk-data mirror (patch held for Max; uk-data then freezes) → 4. re-pin the efrs-post-calibration reference (re-pinning before step 2 would breach the input-mass gate on corporate_wealth). Before any national build or calibration on a split dataset, bump the uk extra lock to the policyengine-uk release carrying private_pension_wealth: the UK targets ons.land.corporate_land_value / land_value are materialised through the engine's key, and an older engine drops the column and narrows it.

Tests

uv run ruff check .                                                    # clean
uv run python tools/build_uk_release_input_coverage_manifest.py --check
POPULACE_UK_WAS_TAB=<tab> uv run python tools/build_uk_was_wealth_support_bounds.py --was-tab <tab> --check
for shard in packages/*/; do uv run pytest "${shard}tests" -p no:cacheprovider; done   # see run log in the PR thread

Licensed round-trip (test_was_support_bounds_round_trip_against_licensed_tab) run locally with the pinned tab.

Follow-ups registered on #750 by the cross-family review

Raise the uk extra floors (policyengine-uk>=) and re-lock once the #1824 release exists, before any split build or calibration; add a composite mass audit (old corporate wealth vs new corporate + pension) so the input-mass gate can distinguish the intended transfer from pension-mass loss; multi-seed / recalibrated results before any forecast-like use of the impact.

Out of scope

Current-employment DB entitlements as their own column (current_db_pension_wealth, would change total_wealth); the cash-ISA gap (counted nowhere as capital); the WAS-stage realisation variance and the OLD-vs-incumbent machinery gap; the same-seed-per-segment pattern in the E6 chains — all registered on #750.

Authored by Claude (Fable) from the #452 diagnosis; adversarial verification by six refuters (statute, chain, model consumers, gates, measurement, completeness) whose blocking findings — coverage-manifest regeneration, gate-battery digest re-cut, the allow-list constant lockstep, the per-segment seed coupling — are all in this branch.

🤖 Generated with Claude Code

[MaxGhenis]

Local verification log (machine under heavy load from concurrent build lanes, so the full microcosm-build shard is left to this PR's CI, which runs the identical per-shard command on a clean runner):

• targeted (all green, run repeatedly during development): test_uk_was_wealth.py, test_uk_wealth_resources.py incl. the licensed bounds round-trip with POPULACE_UK_WAS_TAB, test_uk_e5_identity.py, test_uk_source_stages.py, test_country_spec.py, test_spec_engine_country_bundles.py, test_uk_release_input_coverage.py, test_gate_battery_contract_pins.py
• full shards locally: microcosm-data, microcosm-fit, microcosm-frame, microcosm-calibrate — exit 0
• ruff check . clean; ruff format --check clean on every file this branch touches
• tools/build_uk_release_input_coverage_manifest.py --check and tools/build_uk_was_wealth_support_bounds.py --check (licensed tab) both current