diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 3b1f886d..ae204751 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -827,7 +827,7 @@ deliberate owner pass rather than bundled into this harness change. > 🛠 **Decline overturned (2026-07-09).** A prioritization pass recommended DECLINE; the stated reason was **invalid**. Purity binds `@router` / `@handler` — **not connectors** (CLAUDE.md §8: “side effects (DB, network) belong in connections/transports”). This is an unfired **demand-gate**, not an architectural impossibility. -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **9/10** · _big bet_. The only order-preserving way to push one ordered feed past the ~60 msg/s one-lane-one-core bound; the engine-shard "workaround" is void (shards partition by connection) and the in-engine router-fanout substitute leaves transform serialized, so a real gap with only an awkward workaround. Nothing keyed exists (`partition_key`/`sequence_key`: zero hits in `messagefoundry/`), and keyed lane assignment with single-writer-per-lane over the durable outbox plus the A40 cross-key hazard is multi-week work sitting directly on the strict-FIFO invariant. Quadrant becomes big bet. _(was 5/10 · 9/10.)_ +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **9/10** · _big bet_. The only order-preserving way to push one ordered feed past the ~60 msg/s one-lane-one-core bound; the engine-shard "workaround" is void (shards partition by connection) and the in-engine router-fanout substitute leaves transform serialized, so a real gap with only an awkward workaround. Nothing keyed exists (`partition_key`/`sequence_key`: zero hits in `messagefoundry/`), and keyed lane assignment with single-writer-per-lane over the durable outbox plus the A40 cross-key hazard is multi-week work sitting directly on the strict-FIFO invariant. Quadrant becomes big bet. _(was 5/10 · 9/10.)_ **Reviewed 2026-08-10 -- deferral stands, with the arithmetic recorded.** The 45M/day capability target DOES NOT IMPLY this item: 45M/day across 1,500 connections is ~0.35 ev/s per connection, roughly 170x below the ~60 msg/s one-lane bound, so the target is met by CONCENTRATION rather than per-lane speed. The trigger is specifically ONE ORDERED INTERFACE EXCEEDING ~60 msg/s. Note also that the 2026-07-09 decline was OVERTURNED and must not be re-declined on the purity argument: purity binds `@router`/`@handler`, not connectors. **Type:** feature — throughput/ordering enhancement. Deferred by design: the near-term model is **FIFO per outbound connection** (simple, safe). Per-key ordering is the leading-edge refinement to @@ -932,7 +932,7 @@ active-passive HA). Ranked for v0.2+; the priority tier is on each item. ## 62. Binary body carriage — store ciphertext / raw bodies as `VARBINARY`/`BLOB`/`bytea` instead of base64-in-`NVARCHAR` (storage efficiency) (P3, measure-gated) -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **5/10** · Difficulty **7/10** · _money pit_. Corepoint-class ~60% at-rest win on SQL Server where the only workaround is a bigger disk, but it is measure-gated and never load-bearing on correctness; a carriage format change that re-opens ADR 0028's NUL-safe str/TEXT decision, needs its own ADR, and drags a dual-read migration over three backends and two live `mfenc:` versions. +> ⛔ **DECLINED 2026-08-10 -- owner ruling at the demand-gate triage.** `VARBINARY`/`BLOB` body carriage. The win is a ~60 percent at-rest CONSTANT FACTOR with no correctness stake; the cost is reopening ADR 0028's NUL-safe carriage decision, where being wrong CORRUPTS MESSAGE BODIES. A larger disk is an adequate answer to a constant factor. Its migration plan was also already stale -- the 2026-08-03 amendment records that the engine no longer writes the `mfenc:v1` format the plan proposed to migrate. Declining is reversible: re-filing costs one item. > ⚠️ **AMENDED 2026-08-03 — the format this item plans to migrate is no longer `mfenc:v1`, and the default writer now binds each ciphertext to its cell.** The Type paragraph describes stored bodies as `mfenc:v1::` and the catch plans a dual-read over "existing `mfenc:v1` base64 rows", but cell-bound **`mfenc:v2`** is the default writer: `[store].aad_bind` defaults `True` (`messagefoundry/config/settings.py:383`) and is passed straight through as `write_v2` when the cipher is built (`messagefoundry/store/base.py:1841`; `messagefoundry/store/crypto.py:36`), while legacy v1 rows still read dual-read and are upgraded in place by `rotate-key` (`settings.py:379`) — so a migration must expect **both** markers, not one. That **tightens** the catch rather than easing it: v2 folds `(table, column, primary-key)` into the GCM tag (`messagefoundry/store/crypto.py:155-175`), and the two columns this item would retype are bound that way on the write path today — `cell_aad("messages", "raw", mid)` and `cell_aad("queue", "payload", row_id)` (`messagefoundry/store/sqlserver.py:3430`, `:3452`) — so a carriage that lands a body under a different column name must **re-encrypt**, not merely re-encode. The carriage itself is untouched — `messages.raw` and `queue.payload` are still `NVARCHAR(MAX)` (`messagefoundry/store/sqlserver.py:1105`, `:1123`) — so the win, the own-ADR requirement and the measure-gate all stand as written. @@ -965,7 +965,7 @@ Corepoint 45M/day spec parity analysis. ## 64. Throughput parity with Corepoint — measure-first performance roadmap (group-commit + lean-writes, gated on the enterprise-box validation) (P2, owner / measure-gated) -> 🔢 **OPEN — and what survives under this number is ONLY its index role over the storage-efficiency cluster (#62/#63/#47/#34).** Prose reconciled 2026-08-10 (BACKLOG #64). The throughput half of this roadmap is closed **by measurement, not deferred by it**: step 1 RAN, step 2 is **REFUSED rather than gated**, and the 2026-06-28 "honest verdict" figures below are corrected in place against the runs that superseded them. **Nothing schedulable remains here** — the cluster is scheduled under its own numbers, and steps 4–5 are re-read-first. Whether this umbrella is therefore discharged is an **owner decision**, deliberately not taken by the reconciliation. Value **1/10** · Difficulty **1/10** · _fill-in_ (re-scored 2026-08-03 → P3 once the demand-gate fired; _was 1/10 · 2/10_). +> ⛔ **DECLINED 2026-08-10 -- owner ruling at the demand-gate triage.** Throughput-parity index. Its only surviving role after the 2026-08-10 reconciliation was indexing the storage-efficiency cluster; with #94 and #62 declined it indexes nothing. Declining is reversible: re-filing costs one item. > ⚠️ **AMENDED 2026-08-03 — the measure-first gate has RUN, and step 2 of the ordered plan is REFUSED, not gated.** The plan below still reads live — "Nothing builds before it" at step 1, and group-commit as "the #1 unbuilt durable-write lever … when built — *iff* the run shows durable-write-bound" at step 2 — but the measure-first phase completed 2026-07-12 on rig runs C1–C7 ([ADR 0051](adr/0051-corepoint-throughput-parity-strategy.md), banner at `:3`), group-commit itself was withdrawn ([ADR 0055](adr/0055-group-commit-durable-write.md)`:3-4`, "⛔ SUPERSEDED / WITHDRAWN … DO NOT BUILD THIS"), and the one surviving transaction-reduction lever was falsified by the pre-registered P0 run of 2026-07-13 — the intervention engaged (`committed_txns/msg` −28.5%) while throughput moved −0.56%, inside the pre-registered null band — closing Phase 4 ([ADR 0107](adr/0107-phase-4-is-closed-transaction-reduction-is-a-measured-dead-end.md)`:3` "Do not build F2 or F3", `:7` terminating ADR 0057 as "⛔ DO NOT PROMOTE", table at `:38-40`). ⚠️ **Do not read step 2 as schedulable.** What survives is at least step 3 — this item's index role over the storage-efficiency cluster (#62/#63/#47/#34), which the throughput measurement does not bear on; **re-read steps 4–5 against [ADR 0098](adr/0098-store-side-scaling-levers-are-exhausted-transaction-amortization-is-the-only-path-to-45m-day.md) before scheduling either**, and read the 2026-06-28 "honest verdict" figures below as pre-measurement history, not as the current state. @@ -1049,7 +1049,7 @@ parity analysis. ## 78. Custom message-definition data model + conformance validator; NCPDP codec -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **5/10** · Difficulty **6/10** · _money pit_. Corepoint-parity persisted-definition model plus a report-only validator and an additive NCPDP codec, all cleanly worked around today by a code-first Handler, so useful breadth rather than a blocker; the whole scope is still remainder — NCPDP appears nowhere in `messagefoundry/` and `profile` is merely "reserved for a conformance-profile" (`messagefoundry/parsing/validate.py:56`) — spanning a new stored model the code reads, a validator, and a new codec class. +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **5/10** · Difficulty **6/10** · _money pit_. Corepoint-parity persisted-definition model plus a report-only validator and an additive NCPDP codec, all cleanly worked around today by a code-first Handler, so useful breadth rather than a blocker; the whole scope is still remainder — NCPDP appears nowhere in `messagefoundry/` and `profile` is merely "reserved for a conformance-profile" (`messagefoundry/parsing/validate.py:56`) — spanning a new stored model the code reads, a validator, and a new codec class. **SPLIT 2026-08-10:** the NCPDP codec moves to #1213 (clean, additive, precedented by X12 ADR 0012 and DICOM ADR 0025). What stays here is the PERSISTED-DEFINITION MODEL, and its #26 adjacency is now stated: visual/template-driven authoring is declined-by-design, and this threads that needle only as DATA THE CODE READS, NEVER A GUI MODELER. Trigger restated as **a migration a code-first Handler genuinely cannot express** -- the 75-feed estate port completed USING code-first Handlers, so "modeling-heavy" must not be read as having fired it. > **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). @@ -1146,7 +1146,7 @@ parity analysis. > 🛠 **Decline overturned (2026-07-09).** A prioritization pass recommended DECLINE; the stated reason was **invalid**. Purity binds `@router` / `@handler` — **not connectors** (CLAUDE.md §8: “side effects (DB, network) belong in connections/transports”). This is an unfired **demand-gate**, not an architectural impossibility. -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **6/10** · _big bet_. The strongest store-bloat lever for document-heavy feeds with only awkward workarounds (more disk, purge history), and ADR 0105 already reserved the pointer format and deref seam it plugs into (`messagefoundry/parsing/binary.py:55-62` `DOC_REF_MARKER`, shared-seam note at `:252`, content-address contract at `:264-266`); the remainder is still a pluggable BLOB connector family, a per-connection offload setting across three backends, and an ADR fixing where a write side-effect sits against the at-least-once invariant. _(was 6/10 · 5/10.)_ +> ⛔ **DECLINED 2026-08-10 -- owner ruling at the demand-gate triage.** External BLOB-server offload. Its whole cluster is retired with it: the storage-efficiency group was #94, #62 and #64's index over them. Declining is reversible: re-filing costs one item. **Type:** feature — storage minimization + customer-infrastructure integration. The ingest-time **offload** half of **#47** (its deferred fork (b)), but targeting the **customer's existing object/BLOB store** instead @@ -1327,7 +1327,7 @@ reconciled the same day. ## 96. Built-in "setup tester" — self-service capacity estimator that benchmarks the deployed setup and reports how much traffic it can handle (P2, adopter-facing) -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **6/10** · _big bet_. An adopter-run pre-cutover capacity number has no substitute but the manual dev-harness-plus-TUNING-BASELINE exercise, so a real gap with an awkward workaround. The reuse premise is measured false — `knee` appears in `harness/` only in TOML profile comments and `__main__.py` has no `capacity`/`setup-test` subcommand — so the knee-finder, the non-filling per-step gate, the `/stats` staleness precondition and the isolated-store guard are net-new across CLI + engine + store + metrics: rubric band 6. It is not a 7: there is no 3-backend migration, and ADR 0074 already exists and needs amending, not writing. Quadrant stays big bet. _(was 6/10 · 5/10.)_ +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **6/10** · _big bet_. An adopter-run pre-cutover capacity number has no substitute but the manual dev-harness-plus-TUNING-BASELINE exercise, so a real gap with an awkward workaround. The reuse premise is measured false — `knee` appears in `harness/` only in TOML profile comments and `__main__.py` has no `capacity`/`setup-test` subcommand — so the knee-finder, the non-filling per-step gate, the `/stats` staleness precondition and the isolated-store guard are net-new across CLI + engine + store + metrics: rubric band 6. It is not a 7: there is no 3-backend migration, and ADR 0074 already exists and needs amending, not writing. Quadrant stays big bet. _(was 6/10 · 5/10.)_ **Reviewed 2026-08-10 -- NOT demand-gated; BUILD-GATED on an owner re-ratification (ADR 0074, gated 2026-07-14).** The banner's demand framing was wrong: this waits on the owner, not on a customer. 14 confirmed blockers each OVER-REPORT capacity -- the named "only success gate" admits 3 to 5.5x the true sustainable rate and the poller-zero failure mode satisfies it. The fail-closed GUARD layer (AC-1/3/5/6) remains BUILDABLE; only the MEASUREMENT layer is gated. > > ⚠️ **BUILD GATED (2026-07-14) — the MEASUREMENT layer only.** A validity re-check of the governing > [ADR 0074](adr/0074-adopter-capacity-estimator.md) against STEP-4 Arm 0 returned **14 confirmed blockers**, each @@ -1506,7 +1506,7 @@ that first domain-joined box — run this spike alongside it. ## 99. AD/gMSA production-deployment hardening — turnkey enterprise (Windows/AD) install (P3, on-trigger) -> 🚧 **PARTIAL (built 2026-07-12).** Value **5/10** · Difficulty **3/10** · _fill-in_. Turnkey polish shipped; the **live domain-lab smoke deferred** (needs a real DC + AD CS + gMSA, same gate as #98). **Shipped:** (a) `install-service.ps1` gMSA preflight — `Test-ADServiceAccount` for a `-ServiceAccount` ending in `$` + `secedit`-granted **`SeServiceLogonRight`** before NSSM registration, both **degrading gracefully** on a non-domain/RSAT-less box (skip-with-message, never abort); `-SkipGmsaPreflight` to opt out. (b) `-AllowLocalSystem` opt-out + enhanced LocalSystem warning — enforced **now** as warn + acknowledgement; the **default-FLIP to refuse** is honestly recorded as **gated on the `windows-service-smoke` CI leg** (not flipped live, so no unattended install breaks). (d) **IIS + ARR** reverse-proxy-mTLS reference config added to `docs/security/OFF-LOOPBACK-DEPLOYMENT.md` (require client cert, preserve `X-Forwarded-Proto`/`-For`, exact-peer `trusted_proxies`, placeholders only) beside the existing nginx/Caddy. (f) integrated + gMSA **worked example** in `docs/DEPLOY-SERVER-DB.md §1.1` (`[store].auth=integrated` → `Trusted_Connection=yes`, NSSM `ObjectName=CORP\svc$`, `CREATE LOGIN [CORP\svc$] FROM WINDOWS` least-priv grant) + cross-ref in `CONFIGURATION.md`; **SPN checklist finalized** in OFF-LOOPBACK-DEPLOYMENT.md (gMSA SPN on the account object, "Log on as a service", `PrincipalsAllowedToRetrieveManagedPassword`, IIS/ARR `Negotiate` pass-through). **(c) Windows cert-store (thumbprint) sourcing for `[api]` TLS — SCOPED OUT** (documented, not built): Python `ssl` is OpenSSL not SChannel, and `load_cert_chain` needs cert+key **files**; a non-exportable CNG key in `LocalMachine\My` cannot be handed to OpenSSL, so a store-thumbprint `[api]` TLS source is stdlib-infeasible (same shape as the ECH scope-out, ADR 0093) — supported paths documented instead (terminate at IIS/ARR which *can* use the machine store by thumbprint, or export an AD CS cert to PEM). **Deferred/scoped-out:** (e) real domain-lab gMSA/SSO/reverse-proxy smoke (live DC + AD CS + gMSA — same gate as #98); (g) engine-side "require an AD MFA claim" hook (build only on a customer requirement). No ADR (decisions folded into the deployment docs, per the item plan). _(was 🔢 DEMAND-GATE · Value 6/10 · Difficulty 6/10.)_ +> 🚧 **PARTIAL (built 2026-07-12).** Value **5/10** · Difficulty **3/10** · _fill-in_. Turnkey polish shipped; the **live domain-lab smoke deferred** (needs a real DC + AD CS + gMSA, same gate as #98). **Shipped:** (a) `install-service.ps1` gMSA preflight — `Test-ADServiceAccount` for a `-ServiceAccount` ending in `$` + `secedit`-granted **`SeServiceLogonRight`** before NSSM registration, both **degrading gracefully** on a non-domain/RSAT-less box (skip-with-message, never abort); `-SkipGmsaPreflight` to opt out. (b) `-AllowLocalSystem` opt-out + enhanced LocalSystem warning — enforced **now** as warn + acknowledgement; the **default-FLIP to refuse** is honestly recorded as **gated on the `windows-service-smoke` CI leg** (not flipped live, so no unattended install breaks). (d) **IIS + ARR** reverse-proxy-mTLS reference config added to `docs/security/OFF-LOOPBACK-DEPLOYMENT.md` (require client cert, preserve `X-Forwarded-Proto`/`-For`, exact-peer `trusted_proxies`, placeholders only) beside the existing nginx/Caddy. (f) integrated + gMSA **worked example** in `docs/DEPLOY-SERVER-DB.md §1.1` (`[store].auth=integrated` → `Trusted_Connection=yes`, NSSM `ObjectName=CORP\svc$`, `CREATE LOGIN [CORP\svc$] FROM WINDOWS` least-priv grant) + cross-ref in `CONFIGURATION.md`; **SPN checklist finalized** in OFF-LOOPBACK-DEPLOYMENT.md (gMSA SPN on the account object, "Log on as a service", `PrincipalsAllowedToRetrieveManagedPassword`, IIS/ARR `Negotiate` pass-through). **(c) Windows cert-store (thumbprint) sourcing for `[api]` TLS — SCOPED OUT** (documented, not built): Python `ssl` is OpenSSL not SChannel, and `load_cert_chain` needs cert+key **files**; a non-exportable CNG key in `LocalMachine\My` cannot be handed to OpenSSL, so a store-thumbprint `[api]` TLS source is stdlib-infeasible (same shape as the ECH scope-out, ADR 0093) — supported paths documented instead (terminate at IIS/ARR which *can* use the machine store by thumbprint, or export an AD CS cert to PEM). **Deferred/scoped-out:** (e) real domain-lab gMSA/SSO/reverse-proxy smoke (live DC + AD CS + gMSA — same gate as #98); (g) engine-side "require an AD MFA claim" hook (build only on a customer requirement). No ADR (decisions folded into the deployment docs, per the item plan). _(was 🔢 DEMAND-GATE · Value 6/10 · Difficulty 6/10.)_ **Re-tiered 2026-08-10: NOT demand-gate -- RIG-BLOCKED.** Its own 2026-07-28 amendment already said so: one sub-item remains and it is PROVISIONING, not code. (g) shipped via #274/ADR 0142, (b) closed via #224, (c) is a documented scope-out. Only the live domain-lab smoke remains, needing a real DC + AD CS + gMSA -- the same gate as #98, under #1003. Six items now wait on that one rig. > **AMENDED 2026-07-28 — this is no longer a 6/6 engineering build; ONE sub-item remains, and it is PROVISIONING, not code.** ⚠️ **Do not schedule this as a build.** > @@ -1569,7 +1569,7 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 105. Deterministic Corepoint-import tooling — Action-List → code-first scaffold (P3, deferred, owner decision) -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **2/10** · Difficulty **4/10** · _fill-in_. The adopter already hand-ported and the AI `/migrate` covers the rest, with no named demand, so it ships little worth even if finished; the mapper and CLI are built, leaving reconciliation of the emitted mapping against a real Corepoint export and the deferred `ide/` wrapper — behind #313's multi-message Handler model, which this item cannot buy. _(was 2/10 · 6/10.)_ +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **2/10** · Difficulty **4/10** · _fill-in_. The adopter already hand-ported and the AI `/migrate` covers the rest, with no named demand, so it ships little worth even if finished; the mapper and CLI are built, leaving reconciliation of the emitted mapping against a real Corepoint export and the deferred `ide/` wrapper — behind #313's multi-message Handler model, which this item cannot buy. _(was 2/10 · 6/10.)_ **Reviewed 2026-08-10 -- deferral stands; two corrections to the record.** (1) Its gate #313 sits ABOVE the published #231 baseline BY DESIGN and is sound evidence a public reader cannot open -- per this repo's own convention, do not disclaim it, do not call it unverifiable, do not strip it. (2) The Corepoint ROLE LAYER IS WIRED, so SPEC-105's "Vocabulary Action objects produced: 0" is a PRE-WIRING measurement and must not be quoted as current state. > **AMENDED 2026-07-28 — the stated blocker is discharged; the real gate is a different item.** This item has been carried as blocked on an *"input schema SYNTHETIC-until-validated"* premise. That premise no longer holds: [ADR 0086](adr/0086-deterministic-corepoint-import.md) **Amendment 2026-07-24 §2(a′)** supersedes the old JSON model (`:46-49` marks the synthetic format *SUPERSEDED*) — the input is now a **validated XML** format, parsed through `defusedxml` (`messagefoundry/corepoint_import.py:81`, with the security rationale at `0086:124`). ⚠️ **This does NOT make the item schedulable.** The real gate is **#313** (the multi-message Handler model — the import refuses ~2,000 statements without it), and #313 is **invisible from this published baseline**, which ends at #231. Do not read the discharged blocker as a green light; the item stays P3 and demand-gated behind #313. @@ -1951,7 +1951,7 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 141. TCP connection role selectable independently of direction (act-as-server vs act-as-client) -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **6/10** · _big bet_. Real firewall role-inversion gap that an external relay (socat/stunnel) works around awkwardly but genuinely, which is why it stays at moderate severity and P2; the outbound half is not a knob — `DestinationConnector` (`transports/base.py:459`) exposes only `send` (`:480`) and every destination dials (`tcp.py:189`, `mllp.py:849`, `x12.py:158`), so a listening outbound needs an accept loop handing a peer socket to the per-outbound delivery worker and reconciled with retry/backoff and the connection-lifecycle status vocabulary. _(was 6/10 · 5/10.)_ +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **6/10** · _big bet_. Real firewall role-inversion gap that an external relay (socat/stunnel) works around awkwardly but genuinely, which is why it stays at moderate severity and P2; the outbound half is not a knob — `DestinationConnector` (`transports/base.py:459`) exposes only `send` (`:480`) and every destination dials (`tcp.py:189`, `mllp.py:849`, `x12.py:158`), so a listening outbound needs an accept loop handing a peer socket to the per-outbound delivery worker and reconciled with retry/backoff and the connection-lifecycle status vocabulary. _(was 6/10 · 5/10.)_ **Reviewed 2026-08-10 -- POLICY, not demand; deferral stands.** The socat/stunnel relay does work, and what it costs was not previously written down: it places a hop OUTSIDE the engine's supervision -- outside count-and-log, and outside the forward-secrecy floor `harden_cipher_suites` asserts at 12 call sites across 5 files. The trigger is therefore a POLICY question (is an out-of-engine relay acceptable), not a demand one. > **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). @@ -2009,7 +2009,7 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 155. Server-to-server migration runbook -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **2/10** · Difficulty **1/10** · _fill-in_. Every constituent step already ships documented — install, backup/restore/DR, decommission at `docs/EARLY-ADOPTER-GUIDE.md` §4/§10/§16 — so the gap is prose stitching, not capability; one new doc that orders them end-to-end, no code. +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **2/10** · Difficulty **1/10** · _fill-in_. Every constituent step already ships documented — install, backup/restore/DR, decommission at `docs/EARLY-ADOPTER-GUIDE.md` §4/§10/§16 — so the gap is prose stitching, not capability; one new doc that orders them end-to-end, no code. **Reviewed 2026-08-10 -- deferred by owner ruling: build after more testing.** Pure prose stitching over steps that all ship and are documented separately; no capability gap, 1/10 difficulty. Whoever writes it must write the quiesce/drain ordering against the actual drain semantics rather than assembling section titles -- getting that order wrong loses messages. > **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). @@ -2045,7 +2045,7 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 158. Per-message dynamic FTP host/path/credentials -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **3/10** · _quick win_. Real dynamic-destination gap the shipped code closes off at both ends — host/credentials/`remote_dir` freeze at construction (`messagefoundry/transports/remotefile.py:626-627`) and `render_filename` is hard-capped to one path component (`messagefoundry/transports/file.py:105-127`), so a data-driven target subdirectory cannot be expressed by a static per-folder connection fan-out nor smuggled through the filename; awkward workaround, not a clean one. Build rides the already-shipped #68 per-message metadata carry (`messagefoundry/pipeline/wiring_runner.py:4526-4531`) plus a multi-component path sanitizer — a setting into one connector. _(was 5/10 · 3/10.)_ +> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **3/10** · _quick win_. Real dynamic-destination gap the shipped code closes off at both ends — host/credentials/`remote_dir` freeze at construction (`messagefoundry/transports/remotefile.py:626-627`) and `render_filename` is hard-capped to one path component (`messagefoundry/transports/file.py:105-127`), so a data-driven target subdirectory cannot be expressed by a static per-folder connection fan-out nor smuggled through the filename; awkward workaround, not a clean one. Build rides the already-shipped #68 per-message metadata carry (`messagefoundry/pipeline/wiring_runner.py:4526-4531`) plus a multi-component path sanitizer — a setting into one connector. _(was 5/10 · 3/10.)_ **Reviewed 2026-08-10 -- deferral stands, and the trap is now recorded.** BUILDING THIS REQUIRES WIDENING FILE-1: `render_filename` constrains output to one path component precisely "so an attacker-controlled field can't write outside the target directory or shadow `.processed`/`.error`". Per-message subdirectories is more than one component, and message-selected host/credentials extends attacker-influenceable HL7 content into connection targeting. This is a path-traversal control widening, not a connector knob. > **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). @@ -2302,9 +2302,9 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 179. Archive-aged-rows to separate store -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **4/10** · _quick win_. Real CIEArchive parity gap — `RetentionRunner` deletes and never tiers, and the fallback it names is a whole-store snapshot two backends refuse outright; a copy-then-purge step across the store seam, tested on SQLite, PostgreSQL and SQL Server. +> ⛔ **DECLINED 2026-08-10 -- owner ruling at the demand-gate triage.** Archive aged rows to a separate store. **The unbounded-growth premise does not hold**: `allow_unbounded_phi` defaults false (#186a, ASVS 14.2.4), so a PHI instance REFUSES TO START in prod unless both `messages_days` and `dead_letter_days` are bounded, and per-connection overrides (#34, ADR 0027) let an operator bound noisy feeds individually. Bounded retention plus a larger disk is a complete answer. See #1212, which makes that bound the default. Declining is reversible: re-filing costs one item. -> **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). +> **No longer demand-gated -- declined by design.** > ⚠️ **AMENDED 2026-08-03 — the delete side is per-connection, not store-wide, and the `.mfbak` fallback this item names does not snapshot the store on two of three backends.** The Why says the `RetentionRunner` "purges aged bodies and dead-letters **store-wide by age**", but the pass resolves a **per-connection** window off the live registry every run — `_resolve_overrides` reads each inbound's `messages_days` and each outbound's `dead_letter_days` (`messagefoundry/pipeline/retention.py:625`), `_cutoff_map` turns them into the `connection_cutoffs=` the purges take (`:681`, applied at `:427-437`), and `0` means keep-forever (`_KEEP_FOREVER`, `:76`) — beside a separate per-inbound embedded-document strip on its own window (`_resolve_document_prune`, `:649`, #47/ADR 0042). The named fallback is narrower still: the store snapshot applies **only** to `[store].backend = "sqlite"` (`messagefoundry/pipeline/dr_backup.py:15`) and a server-DB store is forced config-only (`:294`), so on Postgres and SQL Server there is no whole-store `.mfbak` to fall back to. ⚠️ **The gap itself stands, unchanged** — every purge path in that pass still deletes without tiering — but the build must now tier **per connection** to match the window it is tiering out of, and cannot lean on `.mfbak` as the interim answer on a server backend. @@ -2325,9 +2325,9 @@ lane; demand-gated on a first enterprise Windows/AD deployment. ## 180. Cross-backend store migration tool -> 🔢 **Re-scored 2026-08-03 → DEMAND-GATE.** Value **6/10** · Difficulty **5/10** · _quick win_. Real gap — `open_store` picks a backend but nothing moves rows between them (no such subcommand exists in messagefoundry/__main__.py), so the only path discards retained history and audit; an offline row copy that re-wraps every `mfenc` body and reproduces the staged plus history shapes on all three backends. +> ⛔ **DECLINED 2026-08-10 -- owner ruling at the demand-gate triage.** Cross-backend store migration tool. **Consequence recorded deliberately:** greenfield drain-before-cutover, which discards retained history AND audit, is now the shipped answer for backend promotion. That is a position, not an oversight, and it should be stated wherever backend promotion is documented rather than left implicit here. Declining is reversible: re-filing costs one item. -> **On-trigger / demand-gate.** Numbered for tracking only — build when the trigger below fires (“demand-gate, don’t schedule”). +> **No longer demand-gated -- declined by design.** **Cluster:** Store / Operations. **Priority:** P3. **Verdict:** demand-gate. **Severity (vs Corepoint):** minor. @@ -7548,3 +7548,33 @@ gate is the wrong shape, validation of the walk is the right one. **Cluster:** Developer Experience & CI. **Priority:** P2. **Verdict:** build. **Severity:** conditional -- it reds a required merge context on unrelated pull requests; there is no deployment consequence. +## 1212. Bound PHI-body retention by default: `messages_days` and `dead_letter_days` should default to 60 days, not keep-forever + +> 🔢 **Filed 2026-08-10 -- owner-requested. Measured, not assumed.** Value **6/10** · Difficulty **2/10** · _quick win_. Every window in `RetentionSettings` currently defaults to **`0`**, and in this codebase **`0` means KEEP FOREVER** -- `messages_days`, `dead_letter_days`, `state_max_age_days`, `connection_event_retention_hours`, `app_log_days`, `app_log_compress_days`, `search_preset_days`, `reference_snapshot_days`, `audit_days`. Per-connection overrides exist for the two PHI-body windows (#34, ADR 0027): `None` inherits the global, `0` keeps forever, `>0` is days. **Scope: change `messages_days` and `dead_letter_days` to default `60`. Nothing else.** + +> **ASVS: this lands inside a partial cell's subject matter, not beside it.** `14.2.4` (partial, L2) is DOC-TO-CODE FIDELITY for named control domains and **retention-classification is one of them**; `14.2.7` (partial, L3) records that the classification half shipped and made primary PHI-body deletion a shipped default, with the deletion half still missing three classified tiers. Score against both -- do not land this as a config tweak. + +> **FIVE TRAPS, and the first two are the ones that would do real damage.** +> 1. **`audit_days` MUST STAY `0`.** The audit log is a tamper-evident hash chain and HIPAA expects ~6-year retention -- keep-forever there is BY DESIGN. A sweep that "fixes all the zero defaults" would turn a compliance property into a bug while looking like it finished the job. +> 2. **THE SEMANTIC FLIP IS THE REAL RISK.** Today, omitting the setting means *keep forever*; afterwards the identical file means *delete after 60 days*. Same config, opposite behaviour. Zero deployments is exactly why now is the moment -- and every example config, `docs/CONFIGURATION.md` and `docs/PHI.md` must say it loudly rather than let it be discovered. +> 3. The sentinel does not change: `0` still means keep-forever for anyone who sets it explicitly. Only the DEFAULT moves. +> 4. One default change moves EVERY interface at once, because per-connection `None` inherits the global. That is the intent; state it anyway. +> 5. **It weakens a forcing function, and that trade should be made deliberately.** `allow_unbounded_phi` (#186a) currently makes a PHI instance REFUSE TO START in prod unless both windows are bounded -- which forces every operator to choose a retention window consciously. With a bounded default that gate rarely fires: **more secure by default, less explicit by default.** + +> **Framing note:** 60 days is an OPERATIONAL default, not a compliance number. The compliance retention here is the audit chain, which stays keep-forever. Nothing in this item should imply 60 days satisfies a regulatory requirement. + +**Cluster:** Store / Retention. **Priority:** P2. **Verdict:** build. **Severity:** conditional -- on a first deployment an operator who omits the setting would retain full raw PHI bodies indefinitely; there are zero deployments today. + +--- + +## 1213. NCPDP codec -- additive tolerant codec on payload-agnostic ingress, split from #78 + +> 🔢 **Filed 2026-08-10 -- split out of #78 at the demand-gate triage.** Value **4/10** · Difficulty **4/10** · _fill-in_. #78 bolted two unrelated conditions to one trigger: a persisted custom-definition model AND an NCPDP codec. They have very different risk, so they are now separate rows. **This one is the clean half.** + +> **Why it is clean:** it is the shape this project has already shipped twice -- X12 ([ADR 0012](adr/0012-x12-edi-codec.md)) and DICOM ([ADR 0025](adr/0025-dicom-codec-store-connectors.md)). A pure tolerant codec under `parsing/`, riding payload-agnostic ingress ([ADR 0004](adr/0004-payload-agnostic-ingress.md)) behind its own `content_type`, called ON DEMAND by a Router or Handler against a `RawMessage`, and NEVER pushed through the pipeline. It carries no declarative-authoring risk, which is what makes it separable from #78's remaining half. + +> **On-trigger / demand-gate.** Trigger: **a real NCPDP feed appears.** Pharmacy claims are outside the HL7/FHIR/X12/DICOM scope the project has taken on, so this waits for a genuine feed rather than being built on parity grounds. + +**Cluster:** Codecs & Parsing. **Priority:** P3. **Verdict:** demand-gate. **Severity:** minor. + +