From aa791d127d40d02f81f798c244878a7be9d45bc5 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 17:31:55 -0500 Subject: [PATCH 1/3] docs: convert 21 dead BACKLOG line anchors to item references A `BACKLOG.md:` anchor cannot survive an actively edited ledger. The file moved 6,318 -> 6,616 lines during this work alone, and of 43 anchors repo-wide, every one examined had drifted onto unrelated text. Five that were past EOF a day earlier are now IN range and land on plausible-looking wrong items, which is worse than dangling. Item numbers survive both editing and archival, so that is what these become. Root cause, and it is not gradual drift: commit 4ea15017 (the master test plan) is NOT a descendant of 03f1fbd7 (the 185-item archival). They were parallel branches. The plan's anchors were authored against an 8,742-line ledger and landed beside a commit that cut it to 3,858, so they were stale on arrival, not over time. Recovery was per-site reading with the citing prose as the primary evidence -- it usually names the item inline -- backed by resolving each anchor against the pre-archival ledger at 03f1fbd7^. Every mapping was then put to an adversarial reviewer instructed to refute by default. 32 of 43 survived; the 11 refusals were not noise and are deliberately NOT converted here: - Eight sites where the citing CLAIM is dead, not just the pointer. The alerting chapter still asserts #139 and PHI.md "currently contradict each other" when #139 was corrected 2026-08-01 and the code fixed by #323 on 2026-08-02; the ranked-backlog row for #338 is a pre-shipping snapshot. Repointing those would relaunder a false present-tense claim as a fresh, durable-looking reference. They need a content fix, which is a different change. - Six anchors that cannot be attributed to any item, including one pointing at an un-numbered narrative bullet. Left uniform rather than half-repaired: uniform staleness is at least detectable, and a confident wrong pointer is not. - Three that are not citations at all -- two inside a fenced transcript in #1083 reproducing scanner output, and #347's own Source paragraph narrating the falsification test that made the status checker fail on purpose. One mapping was recovered after the review: 16-security-phi:891's "no dormancy contingency" quotes #89's title ("hl7apy security hardening -- dormant-upstream contingency") almost verbatim. One proposal was refuted only on fragment uniqueness, not on the item; applying line-scoped rather than globally makes it safe, which is how all of these were applied. --- docs/BACKLOG.md | 4 ++-- docs/adr/0074-adopter-capacity-estimator.md | 2 +- .../master-test-plan/04-high-availability-and-dr.md | 4 ++-- docs/testing/master-test-plan/06-parsing-and-codecs.md | 4 ++-- .../master-test-plan/10-auth-rbac-and-active-directory.md | 8 ++++---- docs/testing/master-test-plan/13-steps-editor.md | 2 +- .../master-test-plan/16-security-phi-and-supply-chain.md | 2 +- .../master-test-plan/18-interop-migration-and-uat.md | 8 ++++---- 8 files changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 7d7ffdc4..6e4db0c8 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -2765,7 +2765,7 @@ No test covers it. `tests/test_scan_tokens_source.py:559-583` (`test_absolute_ho 5. **Same fix site, sibling defect:** `_WORKTREE_SLUG` at `scripts/security/scan_forbidden.py:92` is case-blind the same way (`[a-z0-9]+`); an upper-cased slug — `claude/` followed by `Some-Task-a1b2c3` — is MISSED. (Written split on purpose, for the reason in the note above: once the fix lands, the joined literal trips the very detector it documents, and unlike `_HOME_PATH` the slug pattern has no `<…>` exemption to write it into.) `scripts/worktree/new.ps1:43,86` passes `-Name` through verbatim with no lowercasing, so an upper-cased worktree name is reachable. Narrower than the home-path case (agent-created slugs are lowercase by convention), but it is a two-character edit in the same block — take it in the same change or say why not. -**Related:** `scripts/security/scan_forbidden.py` (`_HOME_PATH` :99-106, `_WORKTREE_SLUG` :92, call site :758-759), `tests/test_scan_tokens_source.py:559-583`, `.github/workflows/security.yml:446-493`, `.github/required-contexts.txt`, `scripts/worktree/new.ps1`. Sibling **#321** — same gate, same "green gate that cannot see the class" root cause, but the **opposite mechanism**: #321 is an incomplete *token source* (data, fixed by the owner updating a private secret) and explicitly scopes itself away from scanner defects at `docs/BACKLOG.md:7356`; this is a *structural detector* defect (code, fixed by a regex edit) that is live even with no token source. Also **#322**, and the anonymizer's structural-detector item from this same audit. Note #321's **Related:** line at `docs/BACKLOG.md:7363` cites `tests/test_scan_forbidden.py` for regression tests, but the home-path test actually lives in `tests/test_scan_tokens_source.py` — worth correcting when someone next touches #321. +**Related:** `scripts/security/scan_forbidden.py` (`_HOME_PATH` :99-106, `_WORKTREE_SLUG` :92, call site :758-759), `tests/test_scan_tokens_source.py:559-583`, `.github/workflows/security.yml:446-493`, `.github/required-contexts.txt`, `scripts/worktree/new.ps1`. Sibling **#321** — same gate, same "green gate that cannot see the class" root cause, but the **opposite mechanism**: #321 is an incomplete *token source* (data, fixed by the owner updating a private secret) and explicitly scopes itself away from scanner defects; this is a *structural detector* defect (code, fixed by a regex edit) that is live even with no token source. Also **#322**, and the anonymizer's structural-detector item from this same audit. Note #321's **Related:** line cites `tests/test_scan_forbidden.py` for regression tests, but the home-path test actually lives in `tests/test_scan_tokens_source.py` — worth correcting when someone next touches #321. **Source:** public-repo disclosure audit, 2026-08-01. Verified open at HEAD (`12efbffc`) by executing the compiled pattern and by diffing the current, proposed and naive-`re.I` variants across every git-tracked file. @@ -3116,7 +3116,7 @@ _check_handler_security(cfg, strict=True) -> ok=True, skipped=True, "no handler ADR 0144:193-195 records the decorated-scope trade, but justifies it with an **`impure-transform`** false positive ("the trade that keeps the shipped `_pdf_mdm_transforms.py` timestamp fallback clean") and then applies it to `phi-to-log` as well. `samples/config/_demo_oru_transforms.py` and `_pdf_mdm_transforms.py` exist, [`docs/CONNECTIONS.md`](CONNECTIONS.md) §"Decomposing by role" tells authors to put field-level transform logic there, and #226 is an estate-wide sweep to do exactly that — so the one CLAUDE.md §9 rule the lint encodes systematically skips the file the convention steers PHI handling into. -**The third gap the audit named is narrower than described.** The non-recursive `base.glob("*.py")` at `checks.py:893`/`:898` (ADR 0144:196) is **not** an unscanned execution path. `load_config` globs `directory.glob("*.py")` non-recursively too (`config/wiring.py:3969`, and `:4392` for `validate_config`), and `_SiblingHelperFinder.find_spec` returns `None` for any dotted name and serves only `_`-prefixed top-level helpers from the config dir (`wiring.py:3902`, `:3908-3912`). A `.py` in a config subdirectory is therefore neither executed by the loader nor importable by a sibling — and `_assert_safe_config_source` is non-recursive for the same reason (`wiring.py:4194`, `:4324`). The lint's file set already equals the executable set. A recursive walk here would make the lint report on files the safe-source ownership gate never vets — an asymmetry in the other direction. #226 (`docs/BACKLOG.md:6917`) already parks recursion as a *loader* question; it belongs there, not here. +**The third gap the audit named is narrower than described.** The non-recursive `base.glob("*.py")` at `checks.py:893`/`:898` (ADR 0144:196) is **not** an unscanned execution path. `load_config` globs `directory.glob("*.py")` non-recursively too (`config/wiring.py:3969`, and `:4392` for `validate_config`), and `_SiblingHelperFinder.find_spec` returns `None` for any dotted name and serves only `_`-prefixed top-level helpers from the config dir (`wiring.py:3902`, `:3908-3912`). A `.py` in a config subdirectory is therefore neither executed by the loader nor importable by a sibling — and `_assert_safe_config_source` is non-recursive for the same reason (`wiring.py:4194`, `:4324`). The lint's file set already equals the executable set. A recursive walk here would make the lint report on files the safe-source ownership gate never vets — an asymmetry in the other direction. #226 already parks recursion as a *loader* question; it belongs there, not here. **Why:** Bounded, and bounded hard. The lint is advisory by default (`checks.py:953`, `ok=not strict, required=strict`), so a finding blocks nobody unless an adopter opts into `--strict-handler-security` on their own CI. It governs code the adopter's own administrator authors, inside a directory whose write access is already the trust boundary (`_assert_safe_config_source`, `wiring.py:4194`/`:4324`) — anyone who can drop a `.py` there already has arbitrary in-process execution under the engine account, so this is **not** a privilege boundary and evading it buys an attacker nothing they did not already have. diff --git a/docs/adr/0074-adopter-capacity-estimator.md b/docs/adr/0074-adopter-capacity-estimator.md index d78d7a4d..5588f438 100644 --- a/docs/adr/0074-adopter-capacity-estimator.md +++ b/docs/adr/0074-adopter-capacity-estimator.md @@ -176,7 +176,7 @@ If the halved rate is the **daily average**, the peak hour lands at 0.5 × 2.7 = `harness/load/runner.py:59-65` constructs **one** `CorrelationSink` with `ports=tuple(sink_port + i for i in range(sink_ports))` — those are *ports inside one process*, and the ~135–144 msg/s cap is **per sink *process***. In the path §Method names, the cap **cannot be relieved at all**. Multi-*process* sinks exist only in `shardcert.py`'s drive, which this ADR never mentions, and it fails loud when `sink_count > dests` — so the **~36 msg/s single-hospital adopter with 1–2 outbound destinations can provision at most 1–2 sink processes**, hard-capping the *measurable* delivered rate with no remedy. **B14 — Downstream artifacts carry the stale text and would ship the gate holes even if only this ADR is amended.** — ✅ **RESOLVED: all four now corrected** (see item 13 below). -`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; `docs/BACKLOG.md:4077-4079` (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule). +`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; BACKLOG #96 (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule). --- diff --git a/docs/testing/master-test-plan/04-high-availability-and-dr.md b/docs/testing/master-test-plan/04-high-availability-and-dr.md index 82b91f66..cceceefa 100644 --- a/docs/testing/master-test-plan/04-high-availability-and-dr.md +++ b/docs/testing/master-test-plan/04-high-availability-and-dr.md @@ -94,7 +94,7 @@ bare `HA-nn` in this chapter always means **this chapter's own §3.4 row** — t | The PG store-side epoch guard is live-proven nowhere | `test_resumed_ex_leader_is_fenced_after_real_handover` exists in both live suites but only the SS one runs. The PG claim guard appears in **three** separate statements (`postgres.py:2686-2688, 2816-2820, 2934-2938`). | One missed append across three claim paths is invisible to fake-pool tests and shows up only as duplicate/out-of-order delivery under a real handover | No | P1 | | DR readiness is asserted nowhere on the box that needs it | `messagefoundry verify`'s sections are exactly `("host", "store", "smoke", "manual", "federation")` (`messagefoundry/verify/runner.py:22`) — no cluster, HA or DR group. `harness/acceptance/matrix.py:429-470` has G1-G5 with no DR row; `WIN2025-TEST-MATRIX.md` §G likewise. | An operator gets zero signal about DEK availability at the DR site, backup-destination writability, or cluster posture | No | P1 | | Windows port-rebind lag is a one-off manual number | Captured only as prose in `W25:S4.9`; not in the report JSON, no trend, no threshold. | On the intended production OS this dominates observed recovery; a regression is indistinguishable from normal | Manual only | P2 | -| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. `BACKLOG.md:152` (#100) still reads "P1" though it shipped 2026-07-10 (`BACKLOG.md:4229`); `BACKLOG.md:159` (#101) reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 | +| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. BACKLOG #100's ranked-backlog Tier cell still reads "P1" though it shipped 2026-07-10 per its own SHIPPED banner; BACKLOG #101's ranked-backlog Tier cell reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 | ### 3.4 Test matrix @@ -180,7 +180,7 @@ deliverables that STORE-44 and STORE-46 point at — do not descope them here. | HA-56 | Config-only `.mfbak` + `DbaDelegatedError` on a live server DB — **pointer** | Cross-backend | — | — | — | T | P1 | Covered by STORE-45; no separate work scoped. (The files themselves reach CI through HA-02's wiring; the assertion is STORE's.) | | HA-57 | FEATURE-MAP drift guard for Accepted HA/DR ADRs — **pointer** | Compat | — | — | — | T | P1 | Covered by the MIG FEATURE-MAP drift-guard row (one consolidated row extending `tests/test_feature_map_claims.py`); no separate work scoped. HA's contribution is the input list — ADR 0047, 0048, 0049, 0096, 0102 named in `docs/FEATURE-MAP.md` §6, and ADR 0056 carried with a non-shipped marker. | | HA-58 | Correct the two stale FEATURE-COVERAGE-PLAN rows | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `FCP:HA-20`'s "Web console has no HA surface / coverage: none" (§16, `FEATURE-COVERAGE-PLAN.md:1201`) is corrected to reflect `monitoring.py:471-508` + `routes/status.py:88-97` + `test_webui.py:1437-1532`; `FCP:DEPLOY-26`'s "coverage: none" (§24, line 1552) is reconciled with the same document's P6 closure (line 317). Outcome is a dated doc edit, not a falsifiable assertion — the recurring guard is HA-57's owner row. | -| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `docs/BACKLOG.md:152` (#100, `MultiSubnetFailover`) changes from "P1" to SHIPPED, consistent with its own detail section at line 4229; `docs/BACKLOG.md:159` (#101, leader preference / non-promotable) changes from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. | +| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: BACKLOG #100 (`MultiSubnetFailover`) changes its ranked-backlog Tier cell from "P1" to SHIPPED, consistent with its own SHIPPED banner; BACKLOG #101 (leader preference / non-promotable) changes its ranked-backlog Tier cell from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. | | HA-60 | Live PG expired-lease reclaim sweep does not steal a live sibling's rows | HA/Resilience | pytest | container-CI | x2 (PG) | T | P1 | On real PostgreSQL with two owners: rows leased by a crashed owner past `lease_ttl_seconds` are re-pended by `reclaim_expired_leases`, while rows leased by a live, renewing owner are untouched. Asserts the multi-node-safety claim in `leader_tasks.py:129-133` against real SQL rather than the single-node path. | | HA-61 | `[store].lease_ttl_seconds` is inert on SQL Server | Cross-backend | pytest | container-CI | x2 (SS) | T | P2 | On a SQL Server store the setting changes no observable claim/recovery behaviour, and `reclaim_expired_leases` is absent from the SS store surface, so the SS coordinator's `reclaims_inflight()` path never reaches it. Locks in the documented asymmetry (`AOAG-DEPLOYMENT.md:527-531`) that operators otherwise mis-tune. | | HA-62 | Live PG reference-set and transform-state convergence across nodes | Cross-backend | pytest | container-CI | x2 (PG) | T | P1 | On a real two-node PG cluster: the leader materializes a declared `Reference(...)` set, the follower read-throughs the shared snapshot without re-materializing; a transform-state write on one node bumps the namespace `state_version` and reaches the sibling's cache within one convergence interval. Today only the SS half has live coverage (`test_sqlserver_coordinator.py:183-200`). | diff --git a/docs/testing/master-test-plan/06-parsing-and-codecs.md b/docs/testing/master-test-plan/06-parsing-and-codecs.md index 94135cf1..f7ac6184 100644 --- a/docs/testing/master-test-plan/06-parsing-and-codecs.md +++ b/docs/testing/master-test-plan/06-parsing-and-codecs.md @@ -79,7 +79,7 @@ | Codec extras silently uninstall | 41 test files use `importorskip`; there is no skip-counting guard in `tests/conftest.py` or `pyproject.toml` `addopts` | ~100 DICOM/FHIR/X12/XML/WSDL tests vanish and the build is GREEN. `pyproject.toml:119-123` documents exactly this class of breakage (annotated-types 0.8.0 dropping `SLOTS`) already happening once | No — the existing plan flags it for `FCP:X12-6` only | P1 | | `parse_tree()` has its own separator derivation | `tree.py:65-72` blind-slices `msh[4:8]` and takes `sub_sep = enc[3]`. **Verified divergence:** for a legal 3-character MSH-2 (`^~\`), the tree derives `sub_sep = '\|'` (the *field* separator) while `_extract_separators` correctly returns `'&'`. Leaves are also never unescaped, so the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien` | The operator's only structural view of what actually arrived renders wrong in **both** the web console and the harness Qt pane, while routing stays correct — nothing else signals it | No. `test_parse_tree.py` is 8 tests over one default-separator ADT | P1 | | ADR 0054 AC-6 (≥6× multi-core, ~14× single-thread on cp314t) has no executable gate | `test_benchmark_parser.py:207-235` is `skipif`-not-freethreaded and asserts only `ratio > 1.0`; `freethread-smoke.yml` runs only `test_parsing.py` + `test_wiring.py`, so the measure never executes anywhere | The entire justification for reimplementing python-hl7 (ADR 0053/0052) is unverified on any automated surface; only the crude `> 200 msg/s` floor would catch an orders-of-magnitude fall | Partially — the floor exists, the scaling gate does not | P1 | -| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `` selector. Disclosed at `docs/BACKLOG.md:3188` | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 | +| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `` selector. Disclosed at BACKLOG #69 | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 | | No malformed/oversized corpus generator | `generators/_core.generate_message` emits only hl7apy-conformant bodies and raises on an unknown trigger; `messagefoundry generate` exposes only `--type/--triggers/--count/--out/--seed/--list/--json` (`__main__.py:349-362`) | The error/dead-letter path — the count-and-log invariant itself — is never exercised at load or repeatably. Both P0s above are exactly the shapes such a generator would emit. Blocks automating `W25:S4.8` (`WIN2025-TEST-PLAN.md:926,1044,1351`) | No — `W25:S4.8` bad input is GUI-injected manually | P1 | | `ValidationResult.errors` unasserted PHI-redacted | `validate.py:91` appends `str(exc)` from hl7apy verbatim; `wiring_runner.py:3659` does pass it through `safe_text` before persisting, but nothing pins that the **library** result is safe for other callers (dry-run, IDE, harness) | A datatype/table conformance failure can carry the offending field VALUE into any surface that renders `result.errors` directly | Partly — the persisted path is scrubbed; the library return is not asserted | P2 | | No property-based or coverage-guided fuzzing | `pyproject.toml` carries no hypothesis/atheris; both "fuzz" corpora (`test_parsing.py:228-244`, 15 cases; `test_builtin_hl7_hardening.py`) are hand-enumerated. `docs/quality-gates/HANDOFF-mutation-coverage.md:223` proposes mutmut over `parsing/` but is a handoff doc only | Hand-enumerated corpora find only shapes an author imagined — demonstrated by both P0s, each trivially reachable and each missed. Parsing is the largest attacker-influenced surface in the product | No | P2 | @@ -390,6 +390,6 @@ 9. **Does the code-set `UnmappedSink` persistence half get built?** If yes it creates a new PHI-at-rest surface needing encryption, audit and retention decisions, and the `_sink is None` pin (PARSE-48) becomes a pin on the *default*, not on the absence of the feature. **Blocks:** PARSE-48's framing. 10. **Should `parse_tree()` unescape leaf values so the operator's view matches the router's?** Today the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien`. Either answer is defensible; neither is written down or asserted. **Blocks:** PARSE-24, PARSE-51, PARSE-52. 11. **Does `validate(profile=…)` stay a silent no-op (`validate.py:56`), or raise `NotImplementedError` until conformance profiles exist?** A caller passing a profile today gets an `ok=True` that means nothing on a safety-critical strict feed. **Blocks:** PARSE-19. -12. **Is the WSDL `` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at `docs/BACKLOG.md:3188`. **Blocks:** PARSE-31 — a test written now would encode the defect. +12. **Is the WSDL `` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at BACKLOG #69. **Blocks:** PARSE-31 — a test written now would encode the defect. 13. **Should the stale ratings and doc drift be corrected as part of this work?** Concretely: `FEATURE-COVERAGE-PLAN.md` `FCP:PARSE-14` (`:859`) says "partial" (closed by `test_builtin_hl7_hardening.py:595-703`) and `FCP:PARSE-15` (`:860`) says the bench file is "MISSING" (it exists, 235 lines); `docs/HL7-VALIDATION.md:10` still names python-hl7 as the Tier-1 engine though ADR 0054's built-ins parser is the default; ADR 0012 §5 still calls the X12 strict validator deferred though `parsing/x12/validate.py` ships; ADR 0033's header still reads "Proposed … ratified-on-build" though the code sets are built; and `docs/FEATURE-MAP.md` §3 omits ADR 0054, 0122, 0123, 0033, `parsing/xml/`, `consistency.py`, `split.py` and `sniff.py`. **Blocks:** nothing technically — but the acceptance report and this plan both cite those documents as authority. 14. **Is a silent U+FFFD substitution an acceptable ingress outcome?** `normalize()` defaults to `errors="replace"` (`peek.py:152`) precisely so the hot path keeps routing a slightly-off body — but a substitution is an unrecorded, irreversible corruption of every *derived* value while the raw stays byte-faithful, which is count-and-log-clean by construction. Options: keep `replace` and record a warning/`ERROR` disposition when the decoded text differs from a strict decode; switch the ingress decode to `errors="strict"` and let it NAK (PARSE-43's path); or declare it accepted-by-design in `docs/CONNECTIONS.md`. **Blocks:** PARSE-64's unconditional half and the framing of PARSE-43. diff --git a/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md b/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md index 15d9dce1..81785d36 100644 --- a/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md +++ b/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md @@ -42,7 +42,7 @@ This chapter covers the whole authentication/authorization core and, specificall **Objective.** Move the four directory-facing P0 exposures from "asserted against a mock" to "observed against a real domain controller, an AD FS farm and a real browser", and close the cross-backend and meta-guard holes that let a silent regression through CI. -**Objective's hard constraint — the AD lab has not been stood up, so the gate is split.** Nine of this chapter's eighteen P0 rows can only run against a real Domain Controller / AD FS farm, and **that rig does not exist yet**. Standing it up is the genuine blocker — *not* a missing document. The lab's authority document (`docs/security/AD-FEDERATION-LAB-RUNBOOK.md`, cells L0–L18) exists for the owner; the whole `docs/security/` tree is simply **withheld from the public repo** (`.gitignore:144`; `docs/BACKLOG.md:271` records it as "gitignored post-cutover", and `:6041` states outright that such absences are "a publishing boundary, not evidence of completion"), so it is not readable from this worktree. A single undifferentiated P0 set would nonetheless be **unsatisfiable today**, which makes the release gate meaningless rather than strict. §9.4 splits it into a **P0-automated** gate that blocks every release and a **P0 AD campaign gate** that blocks only a release claiming AD / Kerberos / federated-SSO support; §9.7 carries **standing the lab up** — plus the chapter author's read access to the withheld runbook — as the blocking environment-phase exit item. +**Objective's hard constraint — the AD lab has not been stood up, so the gate is split.** Nine of this chapter's eighteen P0 rows can only run against a real Domain Controller / AD FS farm, and **that rig does not exist yet**. Standing it up is the genuine blocker — *not* a missing document. The lab's authority document (`docs/security/AD-FEDERATION-LAB-RUNBOOK.md`, cells L0–L18) exists for the owner; the whole `docs/security/` tree is simply **withheld from the public repo** (`.gitignore:144`; BACKLOG #185 records it as "gitignored post-cutover" and states outright that such absences are "a publishing boundary, not evidence of completion"), so it is not readable from this worktree. A single undifferentiated P0 set would nonetheless be **unsatisfiable today**, which makes the release gate meaningless rather than strict. §9.4 splits it into a **P0-automated** gate that blocks every release and a **P0 AD campaign gate** that blocks only a release claiming AD / Kerberos / federated-SSO support; §9.7 carries **standing the lab up** — plus the chapter author's read access to the withheld runbook — as the blocking environment-phase exit item. ### 9.2 Already covered — do not re-test @@ -105,7 +105,7 @@ This chapter covers the whole authentication/authorization core and, specificall | Hand-maintained gate exemption sets have no derived drift guard: `_MUST_CHANGE_EXEMPT_PATHS` (`api/security.py:54`) and `_MFA_EXEMPT_ROUTES` (`:72`). `tests/test_mfa_access_gate.py:145` pins the method-keyed **shape**, explicitly not membership against the live route table | Adding a path, or renaming a route so an exemption silently widens (the `DELETE /me/mfa` vs `GET /me/mfa` trap the code comment itself calls out at `:58-61`), lets a half-authenticated session reach a sensitive route | Authenticated-but-unverified privilege escalation | Shape only | **P2** | | AD group → **custom** role mapping is only indirect. `roles_for_ad_groups` can return a `custom:`-prefixed id and `_custom_permissions_for_ids` resolves it (`auth/service.py:1592`, used at `:1103`), but no test drives an AD login whose group maps to a custom role | Silent mis-resolution grants an AD operator nothing (lockout) or more than intended; the audit row records only the opaque role id, so the over-grant is illegible after the fact | Any org using custom roles with AD | **No** | **P2** | | No mutation or diff-coverage signal over the auth suite. `docs/quality-gates/HANDOFF-mutation-coverage.md` is an explicit **DRAFT / ready-to-run**; `.github/workflows/quality-advisory.yml` ships only complexity + clone gates | ~480 auth assertions of unmeasured strength guard the security boundary; a weakened predicate (an `or` where an `and` belongs inside a gate) can survive the whole suite | Any auth gate | **No** | **P2** | -| AD service-account least-privilege and gMSA logon rights unproven. `#99(e)` is the only residual and is provisioning-gated (`docs/BACKLOG.md:177`, `:4172`); nothing tests or documents the minimum directory rights the bind account needs | Deployments over-privilege the bind account; a compromised engine host then holds a directory-**write** credential | Directory-wide, on host compromise | **No** | **P2** | +| AD service-account least-privilege and gMSA logon rights unproven. `#99(e)` is the only residual and is provisioning-gated (BACKLOG #99); nothing tests or documents the minimum directory rights the bind account needs | Deployments over-privilege the bind account; a compromised engine host then holds a directory-**write** credential | Directory-wide, on host compromise | **No** | **P2** | | Hardened-DC interop untested. A DC with `LdapEnforceChannelBinding` / LDAP signing required may refuse the simple binds `auth/ldap.py` issues; the engine passes no `channel_bindings` anywhere (#98 unbuilt, `auth/ldap.py:299-303`) | Every AD login becomes a generic `LdapError` with no engine-side classification distinguishing it from an outage — an unsupportable first-contact failure | Any customer on Microsoft's hardened LDAP defaults | **No** | **P2** | | Reconciler default drift. `config/settings.py:1815` ships `ad_session_recheck_seconds = 300`, but `api/app.py:5620` comments "Default OFF (`ad_session_recheck_seconds = 0`)" and `auth/service.py:1189-1191` repeats the claim | An operator reading the code believes no directory traffic is generated; enabling `ad_enabled` silently starts a 300 s bind loop against their DC | Unexpected DC load; misjudged blast radius during an outage | **No** — the setting default is asserted, the two comments are not | **P2** | | No validator floor on any limiter. `docs/SECURITY.md:1571-1575` records that none of the eleven `*_rate_limit_*` fields nor `lockout_threshold`/`lockout_minutes` carries a Pydantic validator, so `per_key=0` or `window_seconds=0` disables enforcement **while the limiter still reports enabled** | A typo or a copy-pasted profile silently removes brute-force protection with no startup complaint | Password spray against the whole user base | Documented, not asserted | **P2** | @@ -135,7 +135,7 @@ Nine of the eighteen P0 rows sit behind the AD campaign gate, and that gate is * | AUTH-06 | Referral posture pinned statically at both `ldap3.Connection` sites | Negative/Security | pytest | dev-PC | n/a | T | P1 | An AST walk (modelled on `tests/test_ldap_timeouts.py:267`) finds both `Connection(...)` sites in `auth/ldap.py` and asserts each carries an **explicit** `auto_referrals` keyword whose value is not a bare literal `True`; the walker self-test proves it sees the bare-import call form | | AUTH-07 | Referral chase does not redirect the password-verifying bind to an untrusted target | Negative/Security | manual | AD-lab | n/a | T | P1 | With a referral configured from domain A to domain B, a login as an A user either (a) never opens a bind to B carrying the user's password, or (b) is refused; captured by a packet count / connection log on the B side showing **zero** simple binds with the user's DN | | AUTH-08 | Real-domain AD login end to end with RBAC applied and the AD identity on the audit row | Functional | manual | AD-lab | SQLite | T | P0 | Domain user `jdoe@mefor.lab` authenticates; a role-gated route returns 200 and a route outside the role returns 403; an `auth.login_success` row exists with `actor=jdoe`, `detail` containing `{"provider":"ad","roles":[…]}` and a non-NULL `client`. (Closes `W25:S1.AC-AD` / matrix `W25:F1` — record evidence there, not a second time) | -| AUTH-09 | Pin the exact kwargs handed to `spnego.server` at **both** acceptor sites | Negative/Security | pytest | dev-PC | n/a | T | P0 | With `kerberos_spn="HTTP/engine.mefor.lab"`, both `kerberos_principal` (`auth/ldap.py:300`) and `kerberos_acceptor_preflight` (`:360`) call `spnego.server(service="HTTP", hostname="engine.mefor.lab")`; a call passing the whole SPN to `service=` fails the test. Test is written to fail RED against today's code until the SPN fix lands (tracked by the owner as **#275**, above the published #231 baseline — `docs/BACKLOG.md:177` and `:4172` both gate the #99(e) residual behind it) | +| AUTH-09 | Pin the exact kwargs handed to `spnego.server` at **both** acceptor sites | Negative/Security | pytest | dev-PC | n/a | T | P0 | With `kerberos_spn="HTTP/engine.mefor.lab"`, both `kerberos_principal` (`auth/ldap.py:300`) and `kerberos_acceptor_preflight` (`:360`) call `spnego.server(service="HTTP", hostname="engine.mefor.lab")`; a call passing the whole SPN to `service=` fails the test. Test is written to fail RED against today's code until the SPN fix lands (tracked by the owner as **#275**, above the published #231 baseline — BACKLOG #99 gates its (e) residual behind it) | | AUTH-10 | #275 confirmation on a real acceptor (handoff cell **L1**) | Functional | manual | AD-lab | n/a | C | P0 | The acceptor principal is captured verbatim. Defect **confirmed** if it reads `HTTP/engine.mefor.lab/` (or `/unspecified`); **refuted** if it reads `HTTP/engine.mefor.lab@MEFOR.LAB`. Verdict recorded verbatim in the run record; L6 and the #99(e) cells stay blocked until it reports | | AUTH-11 | `POST /auth/negotiate` single-leg SPNEGO against a real KDC | Functional | manual | AD-lab | SQLite | T | P0 | A ticket for `HTTP/engine.mefor.lab` obtained on a domain-joined client is accepted; the response carries a session token; `auth.login_success` records `actor=` with `provider=ad`; **no** `WWW-Authenticate` continuation is sent (single-leg is a hard line) | | AUTH-12 | Browser Kerberos SSO `GET /ui/sso` from a domain-joined client, Chrome **and** Firefox | Functional | browser | AD-lab | SQLite | T | P0 | First navigation returns 401 with `WWW-Authenticate: Negotiate`; the browser re-issues with `Authorization: Negotiate …`; the response is a 303 to `/ui` with exactly one session cookie set; the session has **no** step-up window (`seed_reauth=False`) so the first sensitive action bounces to `/ui/reauth`. **Both** browsers must pass — a failure in either is a FAIL for that browser, recorded per browser and never averaged into one verdict | @@ -502,7 +502,7 @@ This area is signed off for release when **all** of the following hold and are r ### 9.9 Open questions -1. **Read access to the withheld `docs/security/` tree.** `AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18), `KERBEROS-EPA-SPIKE-RUNBOOK.md`, `THREAT-MODEL.md` and the ASVS assessment all **exist**; the whole directory is deliberately **withheld from the public repo** (`.gitignore:144`, ~32 files of posture / assessment / risk-register / runbook detail held back as an attacker roadmap; `docs/BACKLOG.md:271` records it as "gitignored post-cutover"). That is why `docs/SECURITY.md:1556` and `HANDOFF-AD-LAB-aws.md:7-9` link to paths that do not resolve from this tree — a publishing boundary, not a gap, so this chapter cites them as sound evidence. The question is operational: will the chapter author get read access, so the AD-lab rows can cite cell numbers as authoritative instead of restating acceptance criteria inline? **Blocks:** nothing structural — only whether AUTH-10/16/17/18/19/20/21 keep their inline restatements (they currently restate, which is safe either way). +1. **Read access to the withheld `docs/security/` tree.** `AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18), `KERBEROS-EPA-SPIKE-RUNBOOK.md`, `THREAT-MODEL.md` and the ASVS assessment all **exist**; the whole directory is deliberately **withheld from the public repo** (`.gitignore:144`, ~32 files of posture / assessment / risk-register / runbook detail held back as an attacker roadmap; BACKLOG #185 records it as "gitignored post-cutover"). That is why `docs/SECURITY.md:1556` and `HANDOFF-AD-LAB-aws.md:7-9` link to paths that do not resolve from this tree — a publishing boundary, not a gap, so this chapter cites them as sound evidence. The question is operational: will the chapter author get read access, so the AD-lab rows can cite cell numbers as authoritative instead of restating acceptance criteria inline? **Blocks:** nothing structural — only whether AUTH-10/16/17/18/19/20/21 keep their inline restatements (they currently restate, which is safe either way). 2. **Where does the chapter read the fuller backlog ledger?** The committed `docs/BACKLOG.md` is a **published baseline that stops at #231** (`:269`), and `:6041` states the rule outright: an item's absence from it is "a publishing boundary, not evidence of completion". The programme continued past it, so **#274 and #275 are valid, owner-confirmed items** — they surface here only as cross-references (`:177`, `:4170`, `:4172`) because their own entries sit above the baseline. This chapter therefore cites them as sound evidence, annotating "above the published #231 baseline" only where a reader would otherwise go looking for an entry. The open question is access alone: may the author read the fuller ledger to quote #275's own acceptance criteria? **Blocks:** nothing — AUTH-09 and AUTH-10 stand on the code evidence at `auth/ldap.py:300` / `:360` regardless. 3. **May the #275 SPN fix land BEFORE the lab confirms it?** The handoff instructs that L1 runs first and hard-blocks L6 and the #99(e) cells. AUTH-09 is written to fail RED against today's code. **Blocks:** whether AUTH-09 lands now as a failing guard, lands after AUTH-10, or is written to pin today's behaviour and inverted later. 4. **May ADR 0142 / OIDC be described as shipped while its status reads "Proposed — code COMPLETE, awaiting lab validation"?** This chapter labels it *built but unvalidated* throughout. **Blocks:** the wording of AUTH-17..25 pass criteria and whether federated SSO appears in release notes as a feature. diff --git a/docs/testing/master-test-plan/13-steps-editor.md b/docs/testing/master-test-plan/13-steps-editor.md index ad4025d1..2e52b2d5 100644 --- a/docs/testing/master-test-plan/13-steps-editor.md +++ b/docs/testing/master-test-plan/13-steps-editor.md @@ -9,7 +9,7 @@ ### 12.1 Scope & objectives -**In scope.** The BACKLOG #26 carve-out (#222, SHIPPED — `docs/BACKLOG.md:6689`): a structured Steps view over real Python Handlers via a typed action vocabulary, where **plain `.py` stays the only artifact and the only execution path**. Concretely: +**In scope.** The BACKLOG #26 carve-out (#222, SHIPPED): a structured Steps view over real Python Handlers via a typed action vocabulary, where **plain `.py` stays the only artifact and the only execution path**. Concretely: - The **projection contract** — `messagefoundry lens parse [--json]` (`messagefoundry/__main__.py:374-390`, `:2770`) → `messagefoundry/lens.py:270 parse_module` / `:285 parse_source`, and its **coverage-partition invariant** (`lens.py:11-21`). - The **rewrite contract** — `messagefoundry lens rewrite` (`__main__.py:392-404`, `:2793`) → `lens.py:1406 rewrite_module` / `:1423 rewrite_source`, its 11 ops (`lens.py:1382-1396 _SUPPORTED_OPS`), the row-scoped byte-space splice, the F7 stale-coordinate guard (`lens.py:1533 _check_expect_src`), the re-parse gate (`:1582 _assert_reparses`) and the 100-column refusal (`:1403 _MAX_LINE_LENGTH`). diff --git a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md index edb5d2af..65c8a4bb 100644 --- a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md +++ b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md @@ -888,7 +888,7 @@ This area is signed off for release when **all** of the following hold: campaign? Which dependency (`hypothesis` / `atheris` / `schemathesis`) is approved?** *Blocks:* SEC-67. The HL7, X12, DICOM and base64-carriage parsers are the most attacker-exposed code in the product and today have only hand-written adversarial cases — and two - of them sit on single-maintainer upstreams with no dormancy contingency (`docs/BACKLOG.md:394`). + of them sit on single-maintainer upstreams with no dormancy contingency (BACKLOG #89). 13. **Which licence-resolution tool becomes the SEC-77 gate, and who ratifies the allow-list — specifically the rule that a term acceptable for AGPL distribution but not sublicensable under diff --git a/docs/testing/master-test-plan/18-interop-migration-and-uat.md b/docs/testing/master-test-plan/18-interop-migration-and-uat.md index 4a02a42c..fdae2560 100644 --- a/docs/testing/master-test-plan/18-interop-migration-and-uat.md +++ b/docs/testing/master-test-plan/18-interop-migration-and-uat.md @@ -164,7 +164,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-32 | The import `--json` ledger exposes its own denominator | Functional | pytest | dev-PC | n/a | T | P1 | `ImportResult.to_json()` (`corepoint_import.py:252-288`) emits a `total_statements` alongside mapped/unmapped/disabled, and `mapped + unmapped + disabled == total_statements` for every fixture. Today the CLI reports three numerators and no denominator, so a migrator cannot see the accounting invariant the tests assert internally | | MIG-33 | `import corepoint` CLI wrapper — end-to-end | Functional | pytest | dev-PC | n/a | C | P2 | **Owned as FCP:CLI-17** (FEATURE-COVERAGE-PLAN :185, :478, :1442). Close it there: drive `main(["import","corepoint",...])` for text and `--json`, assert exit codes and the summary shape. Two cases already exist (`test_corepoint_import.py:945`, `:968`) — the row is the argparse-surface and flag coverage, not a rewrite. **C**, because the falsifiable assertion lives in another document's ID space: this row records that FCP:CLI-17 moved off `partial`, so it cannot fail here and must not gate a MessageFoundry release. It becomes **T** the day the argparse-surface coverage is scoped into this plan instead | | MIG-34 | A `SPEC-105` specified-but-unbuilt mapping is reported as unmapped **with its cause**, never claimed | Functional | pytest | dev-PC | n/a | T | P2 | `EnvLogText` has a built vocabulary target (`messagefoundry/diagnostics.py:36` `log_note`) but no importer mapping (`grep EnvLogText corepoint_import.py` → none). Assert an `EnvLogText` statement lands in the unmapped bucket with a named `_decline_reason`, and that the emitted `# TODO: Corepoint …` marker names `log_note` as the hand-finish target. The same shape for the three-part `MsgSend` fix | -| MIG-35 | Every BACKLOG cross-reference resolves or is classified above the published baseline → **pointer** | Compat | — | dev-PC | n/a | T | P1 | **Covered by MIG-74; no separate work scoped.** The BACKLOG-reference classifier folds into MIG-74's doc-path linter so the publishing boundary is declared in exactly one allowlist rather than a second time inside `scripts/docs/backlog_status_check.py`. Context retained for this chapter's readers: a `#NNN` reference must resolve to a `## NNN.` section **or** be classified above-baseline — the published index ends at `## 231.` (`docs/BACKLOG.md:6883`, boundary prose at `:6041`) — and an above-baseline number such as #313 (cited at `:269` and `:4394`) is **valid evidence**, so the classifier must assert `NNN > 231` rather than tolerating any unresolved reference | +| MIG-35 | Every BACKLOG cross-reference resolves or is classified above the published baseline → **pointer** | Compat | — | dev-PC | n/a | T | P1 | **Covered by MIG-74; no separate work scoped.** The BACKLOG-reference classifier folds into MIG-74's doc-path linter so the publishing boundary is declared in exactly one allowlist rather than a second time inside `scripts/docs/backlog_status_check.py`. Context retained for this chapter's readers: a `#NNN` reference must resolve to a `## NNN.` section **or** be classified above-baseline — the published index ends at `## 231.` (boundary prose in BACKLOG #185) — and an above-baseline number such as #313 (cited at `:269` and `:4394`) is **valid evidence**, so the classifier must assert `NNN > 231` rather than tolerating any unresolved reference | | MIG-36 | `reconcile compare` default output is PHI-safe | PHI | pytest | dev-PC | n/a | T | P0 | Build a needle-bearing synthetic corpus (unique tokens in PID-5, PID-3, PID-7, OBX-5) with a deliberate mismatch. With no diff flag, assert **no needle** appears in stdout, stderr, or the `--report-json` artifact; only counts and match keys. Fails today: `report.py` `render_text` prints field values at :63-66 and `render_json` always emits `left`/`right` at :31-42 | | MIG-37 | `reconcile compare` gates diff disclosure behind an explicit, warned flag | PHI | pytest | dev-PC | n/a | T | P0 | A `--show-diffs` flag (mirroring `tee/__main__.py:269`) is required to reveal field values, and using it prints a PHI/TEST-DATA-ONLY warning to stderr before any diff (mirroring `tee/__main__.py:451-453`). Without it the exit code and counts are unchanged | | MIG-38 | The reconcile capture CLI makes de-identification the reachable path | PHI | pytest | dev-PC | n/a | T | P0 | The `capture` subparser exposes `--anonymize --salt-env ` wiring `CaptureSink(anonymizer=…)` (the kwarg already exists at `capture.py:46`); with it set, no un-anonymized body reaches the JSONL, verified with the ADR 0030 fail-closed leak gate. Without it, a stderr banner states that bodies are written verbatim. Fails today — `_run_capture` never passes the kwarg and no flag exists | @@ -203,7 +203,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-71 | Naive-user INSTALL-GUIDE walkthrough executed verbatim | Usability | manual | W2025-box | SQLite | T | P1 | A first-time operator with no prior Python or engine install follows `docs/INSTALL-GUIDE.md` top to bottom on a clean Windows Server 2025 box, typing only what the document prints. Every deviation, guess, or dead end is recorded as a finding with the line number. PASS = a serving engine, a reachable `/ui`, and one synthetic message through to `PROCESSED`, with **zero** undocumented steps. Repeat from a clean VM snapshot after each doc change | | MIG-72 | Naive-user USER-GUIDE + EARLY-ADOPTER-GUIDE Stage 0 walkthrough | Usability | manual | W2025-box | SQLite | C | P2 | Same discipline over `docs/USER-GUIDE.md` and `EARLY-ADOPTER-GUIDE.md` §11 Stage 0's four exit criteria. Findings are filed against line numbers. Synthetic corpus only, produced by `messagefoundry generate` | | MIG-73 | Area go/no-go roll-up is a signed artifact | Compat | manual | any | n/a | C | P1 | A per-release MIG sign-off sheet: every P0 row PASS; MIG-71 executed on the release candidate with zero undocumented steps; MIG-08/MIG-47's downgrade answer recorded in EARLY-ADOPTER-GUIDE §13; the MIG-60 partner checklist filled for every named partner in scope; a named approver accepting residual risk (mirroring **W25:S6.3**'s shape, not duplicating its content) | -| MIG-74 | **Every doc path named in shipped text or a docstring resolves** — one linter, one boundary allowlist | Compat | pytest | dev-PC | n/a | T | P1 | **Owner of the plan-wide "doc paths resolve" deliverable** — WEB-56 and SEC-09 point here and MIG-35 folds in; no separate work is scoped there. One new module walks every repo-relative path reference and markdown link target in `docs/**/*.md`, `README.md`, `CLAUDE.md` and the docstrings/comments of `messagefoundry/**`, `harness/**`, `tee/**`, `scripts/**`, `ide/src/**`, and resolves each on disk. Three outcomes, none of them silence: **RESOLVED**; **WITHHELD** — a path under `docs/security/`, `docs/reviews/` or `docs/marketing/` (gitignored post-cutover, `.gitignore:144-146`) matched against a checked-in manifest of expected names, reported as withheld and never a FAIL in a public checkout — and the manifest cannot rot vacuous (an empty manifest, or a WITHHELD claim for a path *outside* those three directories, FAILS); **BROKEN** — anything else reds the gate (`TEST-ENVIRONMENT-PLAN.md §5`, MIG-42, is today's example). Folds in MIG-35: a `#NNN` BACKLOG reference must resolve to a `## NNN.` section **or** classify as above-baseline, where `NNN > 231` (`docs/BACKLOG.md:6883`) is valid evidence and an unresolved `NNN ≤ 231` is a FAIL. A `file:line` citation is checked for file existence, and where the anchor names a symbol, that the symbol is present | +| MIG-74 | **Every doc path named in shipped text or a docstring resolves** — one linter, one boundary allowlist | Compat | pytest | dev-PC | n/a | T | P1 | **Owner of the plan-wide "doc paths resolve" deliverable** — WEB-56 and SEC-09 point here and MIG-35 folds in; no separate work is scoped there. One new module walks every repo-relative path reference and markdown link target in `docs/**/*.md`, `README.md`, `CLAUDE.md` and the docstrings/comments of `messagefoundry/**`, `harness/**`, `tee/**`, `scripts/**`, `ide/src/**`, and resolves each on disk. Three outcomes, none of them silence: **RESOLVED**; **WITHHELD** — a path under `docs/security/`, `docs/reviews/` or `docs/marketing/` (gitignored post-cutover, `.gitignore:144-146`) matched against a checked-in manifest of expected names, reported as withheld and never a FAIL in a public checkout — and the manifest cannot rot vacuous (an empty manifest, or a WITHHELD claim for a path *outside* those three directories, FAILS); **BROKEN** — anything else reds the gate (`TEST-ENVIRONMENT-PLAN.md §5`, MIG-42, is today's example). Folds in MIG-35: a `#NNN` BACKLOG reference must resolve to a `## NNN.` section **or** classify as above-baseline, where `NNN > 231` is valid evidence and an unresolved `NNN ≤ 231` is a FAIL. A `file:line` citation is checked for file existence, and where the anchor names a symbol, that the symbol is present | | MIG-75 | **Non-ASCII patient data survives ingest → store → console → export byte-for-byte** | Functional | pytest | dev-PC | x3 | T | P0 | A synthetic corpus whose PID-5, PID-3 and OBX-5 carry accented Latin (`Zoë Ångström-Muñoz`), CJK (`山田 太郎`) and a declared `MSH-18` character set (one `UNICODE UTF-8` case, one `8859/1` case), driven through every hop and byte-compared at each. **(1) Ingest:** the inbound decodes with the connection's declared `encoding` at `errors="strict"` (`pipeline/wiring_runner.py:3485-3494`) and `normalize()` collapses line endings only (`parsing/peek.py:152-163`); on an `MSH-18`-vs-configured-`encoding` conflict the engine must either honour `MSH-18` or record a **named `ERROR` disposition** — silently decoding under the other charset, or substituting U+FFFD into the stored raw, is the FAIL. **(2) Store:** the raw round-trips identical on all three backends and the at-rest-encrypted body decrypts to the same bytes — SQL Server text columns must be `NVARCHAR`, never `VARCHAR`, since a code-page collation silently mangling CJK is the classic version of this defect. **(3) Console:** `GET /messages/{id}` and the `/ui` render return the same code points (rides the console suite's seam, `packaging/messagefoundry-webconsole/tests/test_webui.py`). **(4) Export/attachment:** the NDJSON from `GET /messages/export` (`api/app.py:3035`, #124 / ADR 0131) and the reconstructed bytes from `GET /messages/{id}/attachments/{attachment_id}` (`:3225`, #149 / ADR 0105) are byte-identical to what arrived, base64 attachment path included (ADR 0028 `mfb64:v1:` — **never** latin-1). Finally the outbound re-encode emits the same bytes. **Gap being closed:** `MSH-18` occurs **nowhere** in the tree (0 hits repo-wide), so the declared character set is never consulted; the only adjacent coverage is `tests/test_mllp_encoding_override.py` (17 tests, delimiter re-encode, accented names incidental) and the IDE hex-pane row (ADR 0119). Synthetic names only | | MIG-76 | Wall clock stepped **backwards** while lanes drain — no reordering, no duplicate delivery, no stalled lane | HA/Resilience | pytest | dev-PC | x3 | T | P1 | Under a steady synthetic send with all three stages draining, step the wall clock back 90 s (monkeypatched `time.time` in-process; a container clock step on the CI leg) and assert four things. **(a) No reordering:** per-lane delivery order stays strict FIFO. ADR 0059 makes ordering `seq`-only precisely so a backward step cannot reorder — this row is the regression pin for that decision and for the **#285** per-lane-FIFO no-skip conformance gate (above the published #231 baseline); a claim ordered by `created_at, seq` would fail here, which is what makes the row falsifiable. **(b) No duplicate delivery:** each message is delivered exactly once — no in-flight row is re-claimed because a lease or due-time comparison went backwards (`queue.next_attempt_at`, `store/store.py:1183`; `ix_queue_ready`, `:1189`; `reset_stale_inflight`, `:5762`), and the leader lease neither flaps nor is stolen (`pipeline/cluster.py:94`, `lease_expires_at`). **(c) No stalled lane:** a retry stamped before the step still becomes due and every lane reaches `in_pipeline_depth() == 0` within the retry budget instead of waiting out the skew. **(d)** The forward-step direction fires no retry storm and no duplicate delivery. **Leap-second note:** a positive leap second is a repeated second, not a step, so (a)–(c) subsume it — but no assertion in this row (or in the code it guards) may depend on `time.time()` being *strictly* increasing: equal consecutive timestamps are legal and must not be treated as an ordering error | | MIG-77 | `connections.toml` forward/backward schema compatibility across engine versions | Compat | pytest | dev-PC | n/a | T | P1 | The existing rows test parity (code-first vs data-authored desugaring to identical `Registry` entries); this row tests **versioning**, which nothing does. `connections.toml` carries **no version marker** and its reader is a strict allowlist (`config/connections_file.py` `_INBOUND_KEYS`/`_OUTBOUND_KEYS`, `_reject_unknown` at :300-306) — unlike `connection schema --json`, which does emit `schemaVersion` (`config/connection_schema.py:45`) and which the IDE refuses forward-incompatibly. Pin both directions against a per-vintage fixture set built with MIG-06's generator discipline: **newer file → older engine** — a table carrying a key added after that engine (`flagged`, `deployed`, `auto_start` are the worked examples, `connections_file.py:110-119`) fails loud as a `WiringError` naming the unknown key **and** the file, never a silent drop of the key's effect and never a partial `Registry`; **older file → newer engine** — a table written before those keys existed loads unchanged and desugars into `Registry` entries byte-identical to today's, with the documented defaults (`auto_start`/`deployed` TRUE, `flagged` FALSE), so an adopter's committed file survives an engine upgrade untouched. Additionally assert that every key added to either allowlist since the previous release is named in the upgrade notes MIG-16 checks. If the owner later rules that a version marker is wanted, this row becomes its conformance test; either way the *behaviour* is pinned now | @@ -498,8 +498,8 @@ This area is signed off for a release when **all** of the following hold and are ### 17.9 Open questions -1. **Is `messagefoundry import corepoint` an adopter-facing capability or an internal migration aid?** `docs/AI-OFF-MATRIX.md:21` marks it "✅ shipped"; `docs/BACKLOG.md:4390` keeps #105 OPEN at P3 demand-gated; `docs/FEATURE-MAP.md` has **no row for it at all**. *Blocks:* the testing bar for MIG-31 through MIG-34, which differs by an order of magnitude between the two answers, and whether the importer appears in FEATURE-MAP at all (MIG-28). -2. **What is the decision on BACKLOG #313 (the multi-message Handler model), and is Corepoint verb coverage in scope for any release?** ~2,032 statements are refused for want of it (`HANDOFF-105:24-28`); #313 sits above the published #231 baseline (`docs/BACKLOG.md:6041`), which makes it sound evidence a public-only reader cannot open, not a doubtful citation. *Blocks:* whether MIG-31's pinned ledger is a *floor that must rise* or a *permanent cap*, and whether the SPEC-105 "specified but unbuilt" items (MIG-34) are scheduled or declined. +1. **Is `messagefoundry import corepoint` an adopter-facing capability or an internal migration aid?** `docs/AI-OFF-MATRIX.md:21` marks it "✅ shipped"; BACKLOG #105 is OPEN at P3, demand-gated; `docs/FEATURE-MAP.md` has **no row for it at all**. *Blocks:* the testing bar for MIG-31 through MIG-34, which differs by an order of magnitude between the two answers, and whether the importer appears in FEATURE-MAP at all (MIG-28). +2. **What is the decision on BACKLOG #313 (the multi-message Handler model), and is Corepoint verb coverage in scope for any release?** ~2,032 statements are refused for want of it (`HANDOFF-105:24-28`); #313 sits above the published #231 baseline (stated in BACKLOG #185), which makes it sound evidence a public-only reader cannot open, not a doubtful citation. *Blocks:* whether MIG-31's pinned ledger is a *floor that must rise* or a *permanent cap*, and whether the SPEC-105 "specified but unbuilt" items (MIG-34) are scheduled or declined. 3. **Is Mirth import ever in scope, or is the AI chat participant (`ide/src/chat.ts:40`, `ide/package.json:584-585`) the permanent answer?** This chapter's scope names "Corepoint/Mirth import"; only Corepoint exists as a deterministic path. *Blocks:* whether a MIG row for a Mirth importer belongs in the matrix at all, and whether shipped text may say "Mirth migration" without qualification. 4. **Should the greenfield-only SQLite→server-DB constraint be ENFORCED — refuse or loudly warn on a backend switch with a populated prior store — or remain documentation-only?** *Blocks:* MIG-01 and MIG-02 (a code change plus a test) vs. MIG-03 alone (a doc-drift test plus a recorded risk acceptance). This is the single highest-value decision in the chapter. 5. **What is the supported engine↔console version relationship: strict lockstep as coded (`SUPPORTED_ENGINE_SEAMS == frozenset({15})`), or an N-1 range?** The constant's own comment says re-widening requires "the CI matrix that installs the MIN and MAX supported engine builds — the claim and its test land together, or not at all." *Blocks:* whether MIG-15's PEP 508 range is `>=X, Date: Fri, 7 Aug 2026 17:31:58 -0500 Subject: [PATCH 2/3] docs(testing): move the line number out of 23 source-link targets The repo's file_path:line_number citation convention had leaked inside the href: [`pipeline/alerts.py:27`](messagefoundry/pipeline/alerts.py:27). No prefix makes that resolve. Owner's ruling was to move the line out of the target and keep it in the link text, where it already sits -- no second convention, and nothing lost. These carry a second defect that hid behind the first. The targets were also root-relative, the class #280 repaired across 333 hrefs in this same subtree; they were skipped there because link_check.py deliberately skips ":" targets, so each defect concealed the other. Both are fixed here: the line comes out and the ../../../ prefix goes in, matching the form #280 established in these files. All 23 targets were confirmed to exist before rewriting. Repo-wide there are now zero file:line hrefs, so --include-line-cites reports nothing new -- the checker's skip is now a safety net rather than a suppression. --- .../00-strategy-and-governance.md | 2 +- .../15-alerting-and-observability.md | 36 +++++++++---------- 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/testing/master-test-plan/00-strategy-and-governance.md b/docs/testing/master-test-plan/00-strategy-and-governance.md index cfa0e207..aa914482 100644 --- a/docs/testing/master-test-plan/00-strategy-and-governance.md +++ b/docs/testing/master-test-plan/00-strategy-and-governance.md @@ -204,7 +204,7 @@ drives): | `harness/load` (`--load` / `--failover` / `--estate`) | Performance, resilience | Reports metrics only; SLO verdicts and exit codes per [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) §"Exit codes" | | `harness/acceptance` (`python -m harness.acceptance`) | Host acceptance | Executes probes + re-runs backing pytest suites; **never fakes green** — unautomatable rows report `MANUAL` | | `harness/reconcile` (`capture` / `compare`) | Migration / parallel run | Offline comparison core; normalizes engine-non-deterministic fields; exits non-zero on a real diff so it can gate a per-Connection sign-off | -| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](messagefoundry/verify/runner.py:22)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message | +| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](../../../messagefoundry/verify/runner.py)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message | | `messagefoundry check` | Pre-merge config gate | Required checks `validate`, `dryrun`, `posture`, `build-check`, `reference-backend`; `ruff`/`mypy`/`raise-fstring`/`accepts-candidate` are **advisory and never block** | | CI legs (`ci.yml`) | All automated levels | `test` + `ide` + `ci-gate` on every PR; the heavy legs (`sqlserver-store`, `postgres-store`, `load-test`, `load-test-sqlserver`, `windows-service-smoke`, `docker-smoke`) run **nightly at 03:17 UTC** or on `workflow_dispatch`, plus a PR path-gate arm where `changes` provides one | | `quality-advisory.yml` | Meta / test-quality | `complexity`, `clone`, `coverage` (diff-cover), `mutation` — all **advisory, never required**; the `liveness` job is the meta-gate that demands proof a signal actually executed | diff --git a/docs/testing/master-test-plan/15-alerting-and-observability.md b/docs/testing/master-test-plan/15-alerting-and-observability.md index 608c7607..76f21c5c 100644 --- a/docs/testing/master-test-plan/15-alerting-and-observability.md +++ b/docs/testing/master-test-plan/15-alerting-and-observability.md @@ -14,13 +14,13 @@ off-box, and nothing in CI notices. **In scope — Alerting.** The ADR 0014 rules engine (`AlertRuleSet.decide` matcher, first-match-wins, per-rule severity/transport-subset/cooldown/suppress, `mute`, `recipients`, `id`, `control_action`/ `control_target`, `escalate` tiers, `schedule`, `content_label`) and its five amendments; the -`AlertSink` Protocol ([`pipeline/alerts.py:27`](messagefoundry/pipeline/alerts.py:27) — **20** public +`AlertSink` Protocol ([`pipeline/alerts.py:27`](../../../messagefoundry/pipeline/alerts.py) — **20** public methods) + its `LoggingAlertSink` fallback (same file, `:231`, the same 20) and the **separate** -`NotifierAlertSink` ([`pipeline/alert_sinks.py:578`](messagefoundry/pipeline/alert_sinks.py:578) — a +`NotifierAlertSink` ([`pipeline/alert_sinks.py:578`](../../../messagefoundry/pipeline/alert_sinks.py) — a different class: 35 defs, 27 public = those 20 emit methods + `content_match` + six sink-lifecycle methods; do not conflate its count with the Protocol's); the **18**-member operator-routable set `_ALERT_EVENT_TYPES` -([`config/settings.py:2499-2526`](messagefoundry/config/settings.py:2499) — `lane_stuck` (`:2515`) +([`config/settings.py:2499-2526`](../../../messagefoundry/config/settings.py) — `lane_stuck` (`:2515`) and `rcsi_off_degraded` (`:2516`) **are** members, see 14.2); the webhook and SMTP transports and their failure behaviour; ADR 0127 email templates; ADR 0044 durable `alert_instance` ack/mute(suspend)/resume/resolve lifecycle and auto-resolve inverses; ADR 0128 @@ -60,7 +60,7 @@ has its own `FCP:ALERT-1..FCP:ALERT-24` (different subjects from ours) and the W |---|---| | The 24-row subsystem coverage-gap audit `FCP:ALERT-1..FCP:ALERT-24` (six dimensions, per-feature verdicts) | [`docs/testing/FEATURE-COVERAGE-PLAN.md` §19, lines 1300-1341](../FEATURE-COVERAGE-PLAN.md) — this chapter **supersedes four stale rows** (see 14.2) and otherwise inherits it | | The PHI posture of every log/alert stream (14 rows: format, sink, ACL, retention, PHI class) | [`docs/PHI.md` §7 "Logging inventory (16.1.1 / 16.2.3)", rows 1-14](../../PHI.md) — CI-guarded by `tests/test_phi_logging_inventory.py` | -| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](harness/acceptance/matrix.py:454)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | +| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](../../../harness/acceptance/matrix.py)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | | On-box deployment acceptance (`messagefoundry verify` host/store/smoke/manual/federation sections) | [`docs/testing/VERIFY.md`](../VERIFY.md) — it has no alerting or logging section; nothing here duplicates it | | Load generation mechanics, profiles, governor, SLO verdicts | [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) — this chapter only *consumes* the `connscale` / `sustained-overload` profiles | | Cluster leadership/lease/failover mechanics themselves | the HA/DR chapter — here only the **alerting behaviour across** a leadership move | @@ -120,7 +120,7 @@ tree; carry the correction here, leave that document as a historical artifact): | Stale row | Re-verified status | |---|---| | `FCP:ALERT-4` (line 483: webhook http/SSRF/3xx refusal "partial", P3) | **CLOSED** by `tests/test_asvs_phase0.py:158-190` | -| `FCP:ALERT-12` (lines 431/1319/1341: `lane_stuck` + `rcsi_off_degraded` "absent from `_ALERT_EVENT_TYPES`", P0) | **CLOSED** — both are members at [`settings.py:2515-2516`](messagefoundry/config/settings.py:2515) with rule-targetable tests at `test_alert_rules.py:357/370/395`. The auditor note at line 11 and its `settings.py:1755` line reference are stale (the constant is now at `:2499`) | +| `FCP:ALERT-12` (lines 431/1319/1341: `lane_stuck` + `rcsi_off_degraded` "absent from `_ALERT_EVENT_TYPES`", P0) | **CLOSED** — both are members at [`settings.py:2515-2516`](../../../messagefoundry/config/settings.py) with rule-targetable tests at `test_alert_rules.py:357/370/395`. The auditor note at line 11 and its `settings.py:1755` line reference are stale (the constant is now at `:2499`) | | `FCP:ALERT-19` (line 438: three-backend parity "structural only", P1) | **CLOSED** by live execution on both server backends | | `FCP:ALERT-24` (line 1331) | Cites `tests/test_console_alerts.py`, which **no longer exists**; the PySide6 console is retired. The live surface is `/ui/alerts`, covered by the webconsole suite | @@ -132,27 +132,27 @@ ALERT-13 and ALERT-11. | Risk | Failure mode | Blast radius | Detected today? | Priority | |---|---|---|---|---| -| **G1 — three alert vocabularies bound only by convention** | An emit site's `type` literal, `_ALERT_EVENT_TYPES`, and the `AlertSink` Protocol / `LoggingAlertSink` / `NotifierAlertSink` method sets drift. A type absent from the routable set can never be escalated, routed, suppressed or muted by any operator rule; a method present only on `NotifierAlertSink` raises `AttributeError` on the logging fallback | Every deployment. Drift has fired **twice** already: `lane_stuck` + `rcsi_off_degraded` once shipped unroutable — **both are members today** ([`settings.py:2515-2516`](messagefoundry/config/settings.py:2515)), rule-targetable end to end at `test_alert_rules.py:357/370/395`, so the only live drift is that no guard stops the next one; `content_match` is **still** divergent (only on `NotifierAlertSink` at [`alert_sinks.py:676`](messagefoundry/pipeline/alert_sinks.py:676), absent from the Protocol and the fallback). Note the non-obvious third form: `AlertSink.saturation_rising()` emits type `"saturation"` — method name ≠ event type, so a naive guard would false-fail | **No.** No guard exists in either direction | **P0** | +| **G1 — three alert vocabularies bound only by convention** | An emit site's `type` literal, `_ALERT_EVENT_TYPES`, and the `AlertSink` Protocol / `LoggingAlertSink` / `NotifierAlertSink` method sets drift. A type absent from the routable set can never be escalated, routed, suppressed or muted by any operator rule; a method present only on `NotifierAlertSink` raises `AttributeError` on the logging fallback | Every deployment. Drift has fired **twice** already: `lane_stuck` + `rcsi_off_degraded` once shipped unroutable — **both are members today** ([`settings.py:2515-2516`](../../../messagefoundry/config/settings.py)), rule-targetable end to end at `test_alert_rules.py:357/370/395`, so the only live drift is that no guard stops the next one; `content_match` is **still** divergent (only on `NotifierAlertSink` at [`alert_sinks.py:676`](../../../messagefoundry/pipeline/alert_sinks.py), absent from the Protocol and the fallback). Note the non-obvious third form: `AlertSink.saturation_rising()` emits type `"saturation"` — method name ≠ event type, so a naive guard would false-fail | **No.** No guard exists in either direction | **P0** | | **G2 — `content_match` has no Handler-reachable surface** | ADR 0133 AC-3 says "WHEN a Handler emits a `content_match`" — but there is no export in `messagefoundry/__init__.py` (unlike `db_lookup`/`fhir_lookup` at lines 32-33/150/153), no injected sink on a Handler context, and no dry-run path. Every test calls `sink.content_match(...)` directly | The differentiating Corepoint "Action Point" parity capability is unusable in practice; the PHI-free-by-contract guarantee has never been exercised through real Handler code | **No** — AC-3/AC-4 pass at the sink so nothing fails | **P0** | -| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](messagefoundry/pipeline/alert_sinks.py:100)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](../../../messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | -| **G4 — the alert/security-notify SMTP hop is encrypted but unauthenticated** | `send_plain_email` calls `smtp.starttls()` with **no** `SSLContext` ([`alert_sinks.py:384`](messagefoundry/pipeline/alert_sinks.py:384)), so the stdlib default applies (`check_hostname=False`, `verify_mode=CERT_NONE`). The same transport carries the ASVS 6.3.5/6.3.7 per-user security notifications | A MITM on the SMTP hop reads every alert body (connection names, queue shapes, `safe_exc` detail strings) and every account-security notification (username, source IP, failed-attempt count) — and can suppress them | **No, and the docs disagree with themselves.** `docs/PHI.md` row 11 states the caveat plainly; `docs/BACKLOG.md:5152` claims the sink "uses STARTTLS with a verifying context by design". No test pins either | **P0** | -| **G5 — `[alerts]` is startup-only** | `app.state.alerts_settings` is assigned only at app construction ([`api/app.py:1120`](messagefoundry/api/app.py:1120)) and lifespan startup (`:5485`). `POST /config/reload` (`:2741`) re-runs the `--config` graph, never the service-settings TOML | An operator adds a suppression rule mid-incident via the IDE or `messagefoundry alert add`; the IDE re-lists from the **file** and shows it; `/alerts/rules` still shows the **startup** set; the running notifier keeps paging until a restart. The requirement is documented only in `alerts_edit.py:19-21` and the CLI docstring — nowhere an operator looks | **No** | **P0** | -| **G6 — IDE alert editor offers 4 of the 18 event types and 7 of the 15 fields** | [`ide/src/alertEditor.ts:13-19`](ide/src/alertEditor.ts:13) offers a 5-entry dropdown — `any` plus only `connection_stopped`/`queue_buildup`/`storage_threshold`/`cert_expiry`; `:25-32` supports only `event_type`/`connection`/`min_depth`/`min_oldest_seconds`/`severity`/`transports`/`cooldown_seconds` (7 of the 15 `AlertRule` fields). `ide/src/test/suite/` has **no** alert test file (35 suites, none for the alert editor) | An operator on the supported GUI authoring path cannot rule on **14 of the 18** signals, nor set `id`, `recipients`, `mute`, `escalate`, `schedule`, `content_label`, `control_action`, `control_target` | **No** — the `ide` CI leg runs and tests nothing here | **P1** | +| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](../../../messagefoundry/pipeline/alert_sinks.py)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](../../../messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | +| **G4 — the alert/security-notify SMTP hop is encrypted but unauthenticated** | `send_plain_email` calls `smtp.starttls()` with **no** `SSLContext` ([`alert_sinks.py:384`](../../../messagefoundry/pipeline/alert_sinks.py)), so the stdlib default applies (`check_hostname=False`, `verify_mode=CERT_NONE`). The same transport carries the ASVS 6.3.5/6.3.7 per-user security notifications | A MITM on the SMTP hop reads every alert body (connection names, queue shapes, `safe_exc` detail strings) and every account-security notification (username, source IP, failed-attempt count) — and can suppress them | **No, and the docs disagree with themselves.** `docs/PHI.md` row 11 states the caveat plainly; `docs/BACKLOG.md:5152` claims the sink "uses STARTTLS with a verifying context by design". No test pins either | **P0** | +| **G5 — `[alerts]` is startup-only** | `app.state.alerts_settings` is assigned only at app construction ([`api/app.py:1120`](../../../messagefoundry/api/app.py)) and lifespan startup (`:5485`). `POST /config/reload` (`:2741`) re-runs the `--config` graph, never the service-settings TOML | An operator adds a suppression rule mid-incident via the IDE or `messagefoundry alert add`; the IDE re-lists from the **file** and shows it; `/alerts/rules` still shows the **startup** set; the running notifier keeps paging until a restart. The requirement is documented only in `alerts_edit.py:19-21` and the CLI docstring — nowhere an operator looks | **No** | **P0** | +| **G6 — IDE alert editor offers 4 of the 18 event types and 7 of the 15 fields** | [`ide/src/alertEditor.ts:13-19`](../../../ide/src/alertEditor.ts) offers a 5-entry dropdown — `any` plus only `connection_stopped`/`queue_buildup`/`storage_threshold`/`cert_expiry`; `:25-32` supports only `event_type`/`connection`/`min_depth`/`min_oldest_seconds`/`severity`/`transports`/`cooldown_seconds` (7 of the 15 `AlertRule` fields). `ide/src/test/suite/` has **no** alert test file (35 suites, none for the alert editor) | An operator on the supported GUI authoring path cannot rule on **14 of the 18** signals, nor set `id`, `recipients`, `mute`, `escalate`, `schedule`, `content_label`, `control_action`, `control_target` | **No** — the `ide` CI leg runs and tests nothing here | **P1** | | **G7 — no end-to-end drive of the runner's buildup / stall / saturation emit sites** | `_maybe_alert_buildup` (`:5401`), `_maybe_alert_saturation` (`:5444`), `_maybe_alert_stall` (`:5511`) in `wiring_runner.py`. Only the pure `SaturationDetector` and the **engine-shard** non-owned-lane watchdog are tested | These are the three alerts an operator relies on to notice a stalled or drowning feed. Threshold resolution, the `_outbound_paused` suppression guard, the per-`(stage,lane)` `_BUILDUP_REALERT_SECONDS` throttle and the `pending_depth` read could all break silently | **No** — FEATURE-COVERAGE-PLAN `FCP:ALERT-10` flagged it and it is still open | **P1** | -| **G8 — the alert-storm bound is untested** | `_MAX_QUEUE = 1000` with drop-with-warning ([`alert_sinks.py:126-131`](messagefoundry/pipeline/alert_sinks.py:126)). With a wedged webhook (a hung POST inside the 10 s timeout) and a large estate, a burst silently exceeds the bound and the excess is dropped with only a `WARNING` — there is **no dropped-alert counter or metric** | The operator sees neither the alerts nor a countable drop signal. A regression that lowers the bound, blocks the drain loop, or turns the drop into a *stall* on the emitting delivery worker is invisible | **No** — `tests/test_communications_inventory.py:372` pins `_MAX_QUEUE` as documentation only | **P1** | +| **G8 — the alert-storm bound is untested** | `_MAX_QUEUE = 1000` with drop-with-warning ([`alert_sinks.py:126-131`](../../../messagefoundry/pipeline/alert_sinks.py)). With a wedged webhook (a hung POST inside the 10 s timeout) and a large estate, a burst silently exceeds the bound and the excess is dropped with only a `WARNING` — there is **no dropped-alert counter or metric** | The operator sees neither the alerts nor a countable drop signal. A regression that lowers the bound, blocks the drain loop, or turns the drop into a *stall* on the emitting delivery worker is invisible | **No** — `tests/test_communications_inventory.py:372` pins `_MAX_QUEUE` as documentation only | **P1** | | **G9 — post-failover / post-restart re-page volume is unbounded and untested** | `_last_sent` (throttle) and `_occurrences` (escalation counter) are per-node, in-memory and **not** primed from the store — only `_suspended` is (`prime_suspensions`, `:873`) | On an HA leadership move, a DR promotion, or an ordinary restart, the new node re-pages **every** open condition at once and every escalation tier resets to base, so a long-running critical drops back to warning. ADR 0014 §4 (lines 91-92, 117) records this as accepted — but nothing bounds it and nothing tests it | **No** | **P1** | | **G10 — no boundary assertion that every emit site's `detail`/`reason` is PHI-scrubbed** | `docs/PHI.md` row 10: the webhook carries this free text "`safe_exc()`-scrubbed at the emit sites, but **not** re-run through `safe_text` on this path". The scrub is a convention across ~20 scattered call sites | One new emit site passing `str(exc)` instead of `safe_exc(exc)` sends an HL7 fragment straight off-box to a third-party webhook (Slack/Teams/PagerDuty) — no store, no ACL, no audit. **Silent**: the `alert_instance.reason` IS `safe_text`'d at the store, so an at-rest test passes while the wire payload leaks | **No** | **P1** | | **G11 — outbound saturation blind spot** | `_maybe_alert_saturation` is reached only from `_maybe_alert_buildup`, which runs on the delivery-failure/retry tick — a **healthy-but-behind** outbound (delivering successfully while its backlog climbs) is never sampled. ADR 0014 amendment records it as the "BACKLOG #93 residual" | The single most common real overload — a downstream that is slow but not failing — produces no saturation page, while an operator who opted in believes they are covered | **No** — the known non-coverage is documented but not pinned, so a refactor could widen or narrow it unnoticed | **P1** | | **G12 — four periodic alert runners are not leader-gated** | `cert_expiry.py`, `secret_rotation.py`, `update_check.py`, `gcm_invocations.py` have no `is_leader()` call; only `retention.py:357/:380` does. `gcm_invocations.py:28-29` documents its non-gating *for block refill*, which is a different question from the alert | In an N-node active-passive cluster each node independently observes the **same cluster-wide facts** (a cert file, the DEK rotation age, the installed version, the shared DEK invocation count) and pages every cooldown — N duplicate pages per condition, plus N `alert_instance` upserts contending on `ux_alert_instance_open`. ADR 0014 §4's per-node reasoning covers *lane* events, not these | **No** | **P1** | | **G13 — `support/redact.py` has no adversarial corpus test** | `tests/test_support_bundle.py:93-234` asserts ~7 shapes. The module is a small `re` pass plus the shared `messagefoundry.redaction.redact`; its stated residual is the single-token identifier | The support bundle is the **one** stream designed to leave the box, and `docs/PHI.md` row 14 records it carries **no RBAC, no audit row, no retention** — the operator emails it to a third party. A missed residual is an unlogged, uncontrolled PHI disclosure | **No** — the residual is stated but never measured | **P1** | | **G14 — no cross-surface alert-rule parity check** | Three views of one rule set: `/alerts/rules` (startup `app.state`), the IDE editor list (the TOML **file**), and `/ui/alerts` (rendering `/alerts/rules`). During the restart window they disagree **by construction** (G5) | An operator confirming "the suppression rule is in place" in the IDE while the engine still pages is a real incident-time failure. Nothing detects or signals the divergence | **No** | **P1** | -| **G15 — an alert-triggered `control_action` writes no audit row** | `_dispatch_control` ([`alert_sinks.py:1039`](messagefoundry/pipeline/alert_sinks.py:1039)) fires the injected callback and only logs on failure; `_alert_control` at [`api/app.py:5469`](messagefoundry/api/app.py:5469) has no `record_audit` | A rule can restart a production inbound or outbound automatically with **no** entry in the tamper-evident audit chain. Every other connection-control path in the API is audited | **No** | **P2** | +| **G15 — an alert-triggered `control_action` writes no audit row** | `_dispatch_control` ([`alert_sinks.py:1039`](../../../messagefoundry/pipeline/alert_sinks.py)) fires the injected callback and only logs on failure; `_alert_control` at [`api/app.py:5469`](../../../messagefoundry/api/app.py) has no `record_audit` | A rule can restart a production inbound or outbound automatically with **no** entry in the tamper-evident audit chain. Every other connection-control path in the API is audited | **No** | **P2** | | **G16 — never-resolved `alert_instance` rows grow without bound** | Retention purges **RESOLVED** rows only (`docs/PHI.md` row 8: "an open or acknowledged condition is never aged out from under an operator"). With G3, open `connection_stopped` rows accumulate permanently | The `ux_alert_instance_open` partial unique index and every `/alerts/active` read carry them forever; the nav bell caps at 200 so the count silently understates | **No** — no growth or soak test | **P2** | -| **G17 — OpenTelemetry export is smoke-only** | `tests/test_metrics_exporter.py:345` records against an in-process provider. No OTLP collector round-trip; no CI leg exercises the `[otel]` extra present **or** absent (the guarded-import `RuntimeError` at [`api/metrics.py:463-465`](messagefoundry/api/metrics.py:463)) | `[otel]` is an advertised capability (`docs/FEATURE-MAP.md:159`). A broken export or a wrong resource/attribute set regresses silently while the default Prometheus path keeps passing | **No** | **P2** | +| **G17 — OpenTelemetry export is smoke-only** | `tests/test_metrics_exporter.py:345` records against an in-process provider. No OTLP collector round-trip; no CI leg exercises the `[otel]` extra present **or** absent (the guarded-import `RuntimeError` at [`api/metrics.py:463-465`](../../../messagefoundry/api/metrics.py)) | `[otel]` is an advertised capability (`docs/FEATURE-MAP.md:159`). A broken export or a wrong resource/attribute set regresses silently while the default Prometheus path keeps passing | **No** | **P2** | | **G18 — NSSM log rotation is never asserted** | `scripts/service/install-service.ps1:456-458` sets `AppRotateFiles 1` / `AppRotateOnline 1` / `AppRotateBytes 10485760`; the `windows-service-smoke` leg only captures and uploads the logs (`ci.yml:1239-1259`) | The app log is the **sole** sink for PHI.md stream 1 (the engine installs no file handler) and the source of `/logs/tail` and the support bundle. If rotation stops, the volume fills or the tail becomes unusable — precisely during an incident | **No** — `[retention].app_log_days` deletion and `app_log_compress_days` gzip are unit-tested but never on a real Windows service | **P2** | | **G19 — `/metrics` has no cardinality or scrape-cost bound** | `gather_snapshot` does per-scrape store reads; exporter tests use a handful of connections | A 200+ connection estate on a 15 s Prometheus scrape adds measurable store load, and an unbounded label set (connection × destination × status) can blow up a TSDB | **No** | **P2** | | **G20 — off-box log forwarding has no live-collector round-trip** | `tests/test_logging.py:371-660` covers handler construction, TLS context, CA anchoring, client cert and unreachable-collector tolerance — nothing asserts a record **arrives**, correctly framed, at a collector | This is the SIEM evidence path (`docs/PHI.md` rows 3/4). An RFC 5425 length-prefix or framing regression means audit rows silently never reach the SIEM while the engine reports nothing wrong | **No** | **P2** | -| **G21 — the notifier is constructed only on the `serve` path** | `notifier_from_settings` is called once, in the `create_serve_app` lifespan ([`api/app.py:5316-5318`](messagefoundry/api/app.py:5316)). Any other engine path (embedded `Engine`, `messagefoundry check`, dryrun, most tests) silently runs on `LoggingAlertSink` | A future deployment mode or a lifespan reorder would leave a production engine with no webhook/email notifier and only `WARNING` log lines | **No** — nothing asserts the runner's sink identity after a `serve` lifespan | **P2** | +| **G21 — the notifier is constructed only on the `serve` path** | `notifier_from_settings` is called once, in the `create_serve_app` lifespan ([`api/app.py:5316-5318`](../../../messagefoundry/api/app.py)). Any other engine path (embedded `Engine`, `messagefoundry check`, dryrun, most tests) silently runs on `LoggingAlertSink` | A future deployment mode or a lifespan reorder would leave a production engine with no webhook/email notifier and only `WARNING` log lines | **No** — nothing asserts the runner's sink identity after a `serve` lifespan | **P2** | | **G22 — no bounded-latency assertion that a new alert reaches an operator surface** | The nav bell + `/ui/alerts` are poll-driven (`/ui/nav-status` ~every 15 s, [`webconsole/routes/status.py:128`](../../../messagefoundry_webconsole/routes/status.py)); `/ws/stats` pushes queue counts, **not** alerts | An operator watching the console can be a poll interval behind a stopped connection with no indication the view is stale | **No** — the webconsole tests assert rendering and RBAC, not freshness | **P2** | | **G23 — three ledger/catalog documents lie about this area** | `docs/FEATURE-MAP.md` §9 lists 8 rows and omits ADR 0044 alert state, escalation, templates, per-rule recipients, control actions, **16 of the 18** event types (§9 names only `connection_stopped` and `queue_buildup`), the support bundle, crashdump suppression, the `connection_event` log, `/logs/tail`, `/metrics/history` and host metrics; §10 is still titled "Surfaces — Admin Console (PySide6)" with an Alerts page row at `:172`, and `:131` asserts "The PySide6 desktop console stays (additive)" though `messagefoundry/console/` does not exist. `docs/BACKLOG.md` #171 (lines 5768-5780) still banners DEMAND-GATE and states "there is no runtime/per-area verbosity control and no interactive in-console log viewer" — both are BUILT | FEATURE-MAP reaches the **public mirror** (`tests/test_feature_map_claims.py:3`). It understates the shipped alerting surface and overstates a retired one; the BACKLOG lies about build state — exactly what `backlog-hygiene.yml` exists to prevent | **No** — the existing guard tests links and ASVS score claims, not row currency | **P2** | @@ -172,9 +172,9 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ID | Test | Type | Method | Env | Backend | Cls | Pri | Pass criteria | |---|---|---|---|---|---|---|---|---| -| ALERT-01 | Alert-vocabulary mirror invariant: every `type` literal emitted by a `NotifierAlertSink` method is in `_ALERT_EVENT_TYPES` or in `_AUTO_RESOLVE` | Functional | pytest | dev-PC | n/a | T | P0 | AST-walk `alert_sinks.py`; collect every `"type": ` in a `self._emit({...})` / `self._record_state({...})` call. Each literal is in `_ALERT_EVENT_TYPES ∪ set(_AUTO_RESOLVE)`. The set of emitted-and-routable literals equals `_ALERT_EVENT_TYPES` exactly (currently **18** members, [`config/settings.py:2499-2526`](messagefoundry/config/settings.py:2499)). `saturation_rising` → `"saturation"` is asserted by name so the method-name≠type case is pinned, not a false failure. A planted extra emit type fails the test | +| ALERT-01 | Alert-vocabulary mirror invariant: every `type` literal emitted by a `NotifierAlertSink` method is in `_ALERT_EVENT_TYPES` or in `_AUTO_RESOLVE` | Functional | pytest | dev-PC | n/a | T | P0 | AST-walk `alert_sinks.py`; collect every `"type": ` in a `self._emit({...})` / `self._record_state({...})` call. Each literal is in `_ALERT_EVENT_TYPES ∪ set(_AUTO_RESOLVE)`. The set of emitted-and-routable literals equals `_ALERT_EVENT_TYPES` exactly (currently **18** members, [`config/settings.py:2499-2526`](../../../messagefoundry/config/settings.py)). `saturation_rising` → `"saturation"` is asserted by name so the method-name≠type case is pinned, not a false failure. A planted extra emit type fails the test | | ALERT-02 | Alert-vocabulary mirror invariant: the three classes expose identical public method sets | Functional | pytest | dev-PC | n/a | T | P0 | `{m for m in dir(AlertSink) if not m.startswith("_")}` == the same for `LoggingAlertSink` == the emit-method subset of `NotifierAlertSink` (excluding the sink-lifecycle methods `start`/`aclose`/`set_store`/`set_control_callback`/`suspend`/`resume`/`forget`/`prime_suspensions`, named explicitly as an allowlist). Test **currently FAILS** on `content_match` — record the failure as the acceptance evidence for the G2 decision, then make it pass by whichever way OQ-1 resolves | -| ALERT-03 | `LoggingAlertSink` fallback survives every emit the runner can make | Negative/Security | pytest | dev-PC | n/a | T | P0 | For each of the **20** methods on the `AlertSink` Protocol ([`pipeline/alerts.py:27`](messagefoundry/pipeline/alerts.py:27) — the Protocol lives in `alerts.py`, **not** `alert_sinks.py`; the 35-def `NotifierAlertSink` at [`alert_sinks.py:578`](messagefoundry/pipeline/alert_sinks.py:578) is a different class and its count is not the Protocol's), call it on `LoggingAlertSink` ([`alerts.py:231`](messagefoundry/pipeline/alerts.py:231), the same 20) with representative PHI-free args; no `AttributeError`, no exception, and exactly one log record at the documented level (`WARNING`, except `leadership_lost`/`dr_released` at `INFO` and `connection_restored` a no-op). Calling `content_match` on the fallback raises `AttributeError` today — assert the *current* behaviour explicitly so the divergence is a pinned fact | +| ALERT-03 | `LoggingAlertSink` fallback survives every emit the runner can make | Negative/Security | pytest | dev-PC | n/a | T | P0 | For each of the **20** methods on the `AlertSink` Protocol ([`pipeline/alerts.py:27`](../../../messagefoundry/pipeline/alerts.py) — the Protocol lives in `alerts.py`, **not** `alert_sinks.py`; the 35-def `NotifierAlertSink` at [`alert_sinks.py:578`](../../../messagefoundry/pipeline/alert_sinks.py) is a different class and its count is not the Protocol's), call it on `LoggingAlertSink` ([`alerts.py:231`](../../../messagefoundry/pipeline/alerts.py), the same 20) with representative PHI-free args; no `AttributeError`, no exception, and exactly one log record at the documented level (`WARNING`, except `leadership_lost`/`dr_released` at `INFO` and `connection_restored` a no-op). Calling `content_match` on the fallback raises `AttributeError` today — assert the *current* behaviour explicitly so the divergence is a pinned fact | | ALERT-04 | `content_match` reachability from a real Handler | Functional | pytest | dev-PC | SQLite | T | P0 | Author a sample Handler under a temp config dir that attempts to raise a content alert using only the public `messagefoundry` surface. Assert either (a) a working export (`messagefoundry.alert_content` or an injected sink) produces exactly one `content_match` event whose payload keys are `{type, connection, label}` (+ optional `rule_id`) and **no** other key, **or** (b) an explicit, documented refusal. Assert `"content_match"` in `messagefoundry.__all__` iff (a) | | ALERT-05 | `content_match` unavailable in dryrun and on a Router | Negative/Security | pytest | dev-PC | n/a | T | P0 | If OQ-1 resolves to a Handler export: calling it from a Router or under `messagefoundry dryrun` raises, matching the `db_lookup`/`fhir_lookup` carve-out posture (CLAUDE.md §2). No event is enqueued in either case | | ALERT-06 | A restarted stopped lane clears its open `connection_stopped` instance | HA/Resilience | pytest | dev-PC | SQLite | T | P0 | Drive an outbound to `connection_stopped` through `RegistryRunner`; assert one `alert_instance` row `status='open'`. Call `RegistryRunner.restart_outbound(name)`; assert the instance reaches `status='resolved'` **and** `count_open_alerts_by_connection` drops that connection to 0 within one poll. Fails today (nothing emits `connection_started`) — see 14.5 §S1 | @@ -200,7 +200,7 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ALERT-26 | `_suspended` survives a restart but `_last_sent`/`_occurrences` do not — pinned | HA/Resilience | pytest | dev-PC | SQLite | T | P1 | Suspend `(type, connection)` until `now+3600`, restart the sink, call `prime_suspensions()`; assert the next emit is muted. In the same test assert `_last_sent` and `_occurrences` are empty after the restart (the accepted ADR 0014 §4 asymmetry is a pinned fact, not an accident) | | ALERT-27 | Cluster duplicate-page behaviour for the four ungated periodic runners | HA/Resilience | pytest | dev-PC | n/a | T | P1 | Under a fixture with three coordinators where only one is leader, run one pass each of `CertExpiryRunner`, `SecretRotationRunner`, `UpdateCheckRunner`, `GcmInvocationRunner`. Assert the observed emit count per condition (3 today) against the decided expectation (OQ-5). `RetentionRunner` in the same fixture emits 1 (leader-gated) — the contrast is the point | | ALERT-28 | Duplicate `alert_instance` upserts under concurrent nodes do not violate the open-unique index | Cross-backend | pytest | container-CI | x2 | T | P1 | Three concurrent `upsert_alert_instance` calls for the same `(event_type, connection)` against a live Postgres and a live SQL Server produce exactly one `open` row with `count == 3`; no `IntegrityError` escapes; `ux_alert_instance_open` holds | -| ALERT-29 | IDE alert editor event-type parity | Compat | ide-mocha | container-CI | n/a | T | P1 | `EVENT_TYPES` in `ide/src/alertEditor.ts` equals `["any", ...sorted(_ALERT_EVENT_TYPES)]` read from a generated constant. New file `ide/src/test/suite/alert-editor.test.ts`. Fails today: the dropdown holds **5** entries (`any` + 4 real types, [`alertEditor.ts:13-19`](ide/src/alertEditor.ts:13)) against the required **19** (`any` + the 18 `_ALERT_EVENT_TYPES` members) | +| ALERT-29 | IDE alert editor event-type parity | Compat | ide-mocha | container-CI | n/a | T | P1 | `EVENT_TYPES` in `ide/src/alertEditor.ts` equals `["any", ...sorted(_ALERT_EVENT_TYPES)]` read from a generated constant. New file `ide/src/test/suite/alert-editor.test.ts`. Fails today: the dropdown holds **5** entries (`any` + 4 real types, [`alertEditor.ts:13-19`](../../../ide/src/alertEditor.ts)) against the required **19** (`any` + the 18 `_ALERT_EVENT_TYPES` members) | | ALERT-30 | IDE alert editor field parity | Compat | ide-mocha | container-CI | n/a | T | P1 | The `NewRule` field set equals `AlertRule.model_fields` from the same generated constant, or a decided, explicitly-listed subset with the omitted fields named. Mirrors the `tests/test_alerts_edit.py:153` `_RULE_FIELDS` pattern | | ALERT-31 | The generated IDE constant is regenerated from the engine and drift fails CI | Compat | pytest | container-CI | n/a | T | P1 | An engine-side test regenerates the constant from `_ALERT_EVENT_TYPES` + `AlertRule.model_fields` and asserts the committed file is byte-identical. Adding an event type without regenerating fails the `test` leg (not only the `ide` leg) | | ALERT-32 | Cross-surface rule parity after a restart | Functional | pytest | dev-PC | SQLite | T | P1 | Add a rule via `messagefoundry alert add`, restart the serve app, then assert `messagefoundry alert list --json`, `GET /alerts/rules`, and the `/ui/alerts` rendered rule table all contain the same rule count, ids and event types | @@ -211,7 +211,7 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ALERT-37 | A newly-opened alert reaches the console within a bounded number of polls | Usability | pytest | dev-PC | SQLite | T | P2 | Upsert an `alert_instance`, then poll `/ui/nav-status`; the alerts `count` reflects it on the first poll after the write and the `severity` field equals the worst open severity. Assert the documented ~15 s poll cadence is what the page's script uses | | ALERT-38 | `/ui/alerts` ack / resolve / suspend / resume round-trip and CSRF posture | Functional | pytest | dev-PC | SQLite | T | P2 | Each of the four POST routes in `messagefoundry_webconsole/routes/monitoring_writes.py` mutates the instance and 303-redirects to `/ui/alerts`; a cross-site POST is refused; an operator without `monitoring:diagnose` gets no forms rendered | | ALERT-39 | `POST /alerts/test-email` exercises the identical send path a real alert uses | Functional | pytest | dev-PC | SQLite | T | P2 | With a capturing SMTP stub, the frames produced by `/alerts/test-email` and by a real fired alert differ only in subject/body content — the same `EmailTransport` → `send_plain_email` → `starttls` → `send_message` sequence, the same allowlist check, the same timeout. Guards against the test path drifting into a false-confidence stub | -| ALERT-40 | `/metrics` cardinality and scrape cost on a large estate — **the owning row** (the API chapter's `API-54` points here) | Performance | load-harness | dev-PC | SQLite | T | P2 | Derive the expected series count **from the registry**, don't guess it: `_MetricsCollector.collect` ([`api/metrics.py:246-441`](messagefoundry/api/metrics.py:246)) yields a fixed block (`build_info`, `in_pipeline`, `store_committed_txns`, `store_body_copies`, + 4 host gauges when psutil reads, + the 9-series pool block on a server backend), `outbox_status` at one series per outbox status, **2 per inbound connection** (`messages_received`, `messages_errored`), **4 per (connection, destination) pair** (`deliveries`, `deliveries_dead`, `queue_depth`, `oldest_pending_age_seconds`) and **15 per pair** for `delivery_latency_seconds` (12 `DEFAULT_LATENCY_BUCKETS` + `+Inf` + `_sum` + `_count`). On the `harness/load/profiles/connscale.toml` estate assert total series `== K + 2·I + 19·D` **exactly** — i.e. growth is linear in inbound connections and in *existing* destination pairs, never the connection×destination×status product — that the scrape stays under a committed latency ceiling at that estate size, and that the label-**name** set is exactly the strict allowlist `{connection, destination, status, version, le}` (extending `tests/test_metrics_exporter.py:135`) with no PHI-shaped label value | +| ALERT-40 | `/metrics` cardinality and scrape cost on a large estate — **the owning row** (the API chapter's `API-54` points here) | Performance | load-harness | dev-PC | SQLite | T | P2 | Derive the expected series count **from the registry**, don't guess it: `_MetricsCollector.collect` ([`api/metrics.py:246-441`](../../../messagefoundry/api/metrics.py)) yields a fixed block (`build_info`, `in_pipeline`, `store_committed_txns`, `store_body_copies`, + 4 host gauges when psutil reads, + the 9-series pool block on a server backend), `outbox_status` at one series per outbox status, **2 per inbound connection** (`messages_received`, `messages_errored`), **4 per (connection, destination) pair** (`deliveries`, `deliveries_dead`, `queue_depth`, `oldest_pending_age_seconds`) and **15 per pair** for `delivery_latency_seconds` (12 `DEFAULT_LATENCY_BUCKETS` + `+Inf` + `_sum` + `_count`). On the `harness/load/profiles/connscale.toml` estate assert total series `== K + 2·I + 19·D` **exactly** — i.e. growth is linear in inbound connections and in *existing* destination pairs, never the connection×destination×status product — that the scrape stays under a committed latency ceiling at that estate size, and that the label-**name** set is exactly the strict allowlist `{connection, destination, status, version, le}` (extending `tests/test_metrics_exporter.py:135`) with no PHI-shaped label value | | ALERT-41 | `/metrics` counter semantics survive a service restart | Compat | pytest | dev-PC | SQLite | T | P2 | Counters that are store-derived resume from the store value after a restart; process-local counters reset to 0 and are documented as such in the exporter's HELP text. No counter silently goes backwards without a documented reason | | ALERT-42 | `/metrics/history` is in-process and lost on restart, by design | Functional | pytest | dev-PC | n/a | T | P2 | Record samples, restart the app, assert `GET /metrics/history` returns an empty ring; assert `capacity == 900` and `min_interval == 0.9`; assert two concurrent `/ws/stats` samplers do not double-append the same instant | | ALERT-43 | OTel guarded-import failure message when `[otel]` is absent | Negative/Security | pytest | container-CI | n/a | T | P2 | With `opentelemetry` uninstalled, `build_otel_meter_provider()` and `OtelMetricsExporter(...)` each raise `RuntimeError` whose message names `pip install messagefoundry[otel]`. Currently marked `pragma: no cover` — this row removes the blind spot | From ee9f10b4d5b6c9a72ad5bdb27ffd697289bef803 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 17:36:02 -0500 Subject: [PATCH 3/3] backlog: correct #1095's own counts, and file #1098, #1099, #1100 #1095 recorded four measured counts and three of them were wrong -- in the way the item itself is about. Its href figure of 13 was low by two orders of magnitude against at least 629 repaired sites. Its two anchor counts moved between filing and repair, and not because anything improved: this file grew 6,318 -> 6,616 lines, so five anchors that were safely past EOF came back INTO range and now land on plausible-looking wrong items. A dangling pointer degrades into a confident one as the file grows, which is the sharpest argument for the item-number convention. Root cause of the anchor class recorded, because it is not gradual drift: commit 4ea15017 (the master test plan) is not a descendant of 03f1fbd7 (the 185-item archival). Parallel branches, so anchors written against an 8,742-line ledger landed beside a commit that cut it to 3,858 -- stale on arrival. It also explains the anchor citing line 8429, absurd against 6,616 and ordinary against 8,742. The markdown-only scope note is discharged rather than left open. The predicted sweep of harness/, ide/src/, messagefoundry/, scripts/, tests/, packaging/ and .github/workflows/ ran over 1,219 files: 68 nonexistent paths, none of them a rotted citation -- test fixtures, withheld directories, and past-tense historical comments. The prediction was wrong and the reason is kept: citations in code are written about the past, prose citations are written as pointers. Three items filed, numbers allocated with alloc.ps1: #1098 -- the coordination hook prints a session UUID in the column a commit SHA occupies. Small, but it is an instrument answering a different question from the one its header asks, with nothing in the output saying so. #1099 -- #1094 says "the archival pass generates the anchor". There is no archival tooling at all; the move is manual. The sentence sits in the archive as settled record and points maintenance at a generator that was never built. #1100 -- the nine sites where the citing CLAIM is dead, not just the pointer, split out of #1095 rather than repointed. The alerting chapter still sets exit criteria requiring that #139 and PHI.md be made to agree when #139 was corrected 2026-08-01 and the code fixed by #323 on 2026-08-02. Converting those pointers would attach a durable reference to a false claim, which is worse than leaving the stale anchor visible. Found by the adversarial pass: every one was proposed as a clean repoint by a first reader and refuted by a second who checked the claim against the code rather than against the anchor. --- docs/BACKLOG.md | 135 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 127 insertions(+), 8 deletions(-) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 6e4db0c8..8f9468b1 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -6464,12 +6464,36 @@ What was unwatched is the **release-lag window**: the interval between a fix lan **At least four distinct classes, and they differ in DETECTABILITY. This is the item's central constraint** — state it in any plan, or a linter closes the visible third and the rest is declared done: -| Class | Count (2026-08-07) | Catchable by a link checker? | -|---|---|---| -| Path-bearing citation names the live ledger; item is archived | at least 69 sites / 35 files | **No** — the link resolves perfectly. Only the human-readable number beside it is stale. | -| Relative href does not resolve at all | 13 | Yes | -| `BACKLOG.md:` anchor past end of file | 12 (file is 6318 lines; one anchor cites **8429**) | Yes | -| `BACKLOG.md:` anchor in range but drifted onto unrelated text | 31 | **No** — resolves, lands somewhere plausible, says the wrong thing. | +| Class | Count as filed | Re-measured repo-wide | Catchable by a link checker? | Status | +|---|---|---|---|---| +| Path-bearing citation names the live ledger; item is archived | at least 69 sites / 35 files | 73 sites / 34 files | **No** — the link resolves perfectly. Only the human-readable number beside it is stale. | repaired | +| Relative href does not resolve at all | 13 | **at least 629** | Yes | repaired | +| `BACKLOG.md:` anchor past end of file | 12 | 7 | Yes | converted to item references | +| `BACKLOG.md:` anchor in range but drifted onto unrelated text | 31 | 36 of 43 | **No** — resolves, lands somewhere plausible, says the wrong thing. | 25 converted, 18 left deliberately | + +**Three of those four filed counts were wrong, and the way they were wrong is the item's own subject.** +The href figure of 13 was low by two orders of magnitude: the repaired sites total at least 629 (333 +root-relative hrefs under `testing/`, 270 broken by the archival move, 3 ADR-slug and depth errors, +23 `file:line` targets). The anchor counts moved in the *opposite* direction between filing and +repair, and not because anything improved — this file grew 6,318 → 6,616 lines, so five anchors that +were safely past EOF came back **into** range and now land on plausible-looking wrong items. A +dangling pointer degrades into a confident one as the file grows. + +**Root cause of the anchor class, which is not gradual drift.** Commit `4ea15017` (the master test +plan) is **not** a descendant of `03f1fbd7` (the 185-item archival) — they were parallel branches. +The plan's anchors were authored against an 8,742-line ledger and merged alongside a commit that cut +this file to 3,858. They were stale on arrival. That also explains the anchor citing line **8429**, +which is absurd against 6,616 lines and was ordinary against 8,742. + +**18 anchors were left unconverted on purpose**, in three groups. Nine sites make a present-tense +claim that the repo has since falsified — the alerting chapter still says #139 and `PHI.md` +"currently contradict each other" when #139 was corrected 2026-08-01 and the code fixed by #323 on +2026-08-02 (filed as #1100). Attaching a durable item reference to a dead claim relaunders it, so +those need a content fix, not a repoint. Six cannot be attributed to any item at all, one of them +pointing at an un-numbered narrative bullet; they stay uniformly stale, because uniform staleness is +detectable and a confident wrong pointer is not. Three are not citations — two sit inside a fenced +transcript in #1083 reproducing scanner output, and one is #347's own **Source:** paragraph +narrating a deliberate falsification. The two "No" rows are the majority and the harder half. Root cause for the anchor classes is the same archival pass that produced the first: it shortened the ledger under citations that had been correct. @@ -6481,9 +6505,11 @@ The two "No" rows are the majority and the harder half. Root cause for the ancho **Already specified elsewhere — do not file a third time.** In [`testing/master-test-plan/18-interop-migration-and-uat.md`](testing/master-test-plan/18-interop-migration-and-uat.md): `MIG-74` is a P1 row reading *"Every doc path named in shipped text or a docstring resolves — one linter, one boundary allowlist"*, and **`MIG-35` is already named there as "the BACKLOG-reference classifier", folding into MIG-74** (:128) — so the exact check this item needs is specified, just unbuilt. This item supplies what those rows lack: the measured counts, the detectability split, and the warning that `MIG-74` as worded ("paths resolve") would pass the **largest** class untouched, because those paths *do* resolve. -**Scope note — markdown only.** The sweep behind these numbers read `*.md`. Citations also live in `harness/`, `ide/src/`, `messagefoundry/` and `.github/workflows/`, and rot identically; they are **not** counted above and should be swept before anyone calls the class closed. +**Scope note — the non-markdown sweep has now RUN, and it found nothing.** The counts as filed read `*.md` only, and this note used to say citations in `harness/`, `ide/src/`, `messagefoundry/` and `.github/workflows/` "rot identically" and must be swept before the class could be called closed. That sweep ran 2026-08-07 across 1,219 tracked non-markdown files. It surfaced 68 distinct repo-relative paths that do not exist and **not one of them is a rotted citation**: they are test fixtures (`docs/adr/0001-collision.md`, `messagefoundry/x.py`, `tests/test_a.py`), targets inside withheld directories, or past-tense historical comments — `.github/workflows/release.yml:51` names the pre-cutover `scripts/publish/publish.ps1` under a heading reading "THIS GUARD WAS INVERTED UNTIL THE CUTOVER". The prediction was wrong, and the reason is worth keeping: citations in code are mostly written *about* the past, where prose citations are written as *pointers*. **Do not re-open this as unswept scope.** + +**A gate now covers the catchable half, repo-wide.** `scripts/docs/link_check.py` with `tests/test_archive_link_resolution.py` asserts that every relative markdown link in the repository resolves — 5,327 links across 347 files. Widening it beyond the archive required removing three false-positive classes first, because a gate red on arrival gets suppressed rather than fixed: links inside inline code (a regex whose character class contains `](`, two VS Code `command:` URIs, and ADR 0160 quoting the link it records as removed), and the withheld `docs/releases/` and `.claude/`. **`.claude/` is the one to carry forward** — it is gitignored but *present* in a long-lived local checkout, so the first repo-wide measurement passed there and undercounted by 7; widening on that number would have put the gate red on CI's clean clone. A gate result is a fact about the configuration it ran in. The gate was then proved able to fail, by planting a break in a real document outside the archive and watching it go red. -**Related:** #1094 (two instances of this class, closed — already satisfied when filed), #1073 (whose §12 marker prompted the original find, and which is itself the closed-but-not-archived case), #1000 (a control whose green is not evidence about what it appears to cover), [`../CLAUDE.md`](../CLAUDE.md) §11 (state a load-bearing fact once and link to it — which is what makes a link's durability load-bearing). +**Related:** #1094 (two instances of this class, closed — already satisfied when filed), #1073 (whose §12 marker prompted the original find, and which is itself the closed-but-not-archived case), #1000 (a control whose green is not evidence about what it appears to cover), #1099 (#1094's claim that an archival pass *generates* the anchor, describing tooling that does not exist), #1100 (the nine sites whose citing claim is dead, split out of this item rather than repointed), [`../CLAUDE.md`](../CLAUDE.md) §11 (state a load-bearing fact once and link to it — which is what makes a link's durability load-bearing). **Source:** repo-wide sweep 2026-08-07 while repointing the [`CONNECTIONS.md`](CONNECTIONS.md) serial/ASTM decline (PR #273, the `docs/CONNECTIONS.md` half of the same two-marker pair #1094 covered in `../CLAUDE.md`). Item locations came from `parse_items` imported from the status-check script rather than a hand-rolled scan; the resolver was validated on planted good-and-bad citations, including the closed-but-not-yet-archived case, before its output was believed. @@ -6614,3 +6640,96 @@ So `-Com` and `-Comm` are working spellings. `-Cm` is not, which bounds the fami **Source:** found 2026-08-07 by the adversarial verification of a candidate fix for #1086. Two verifiers independently measured the abbreviation family; the third accepted the candidate because its corpus contained no such payload, which is itself the #1000 shape. The pre-existing half was separated from the introduced half by measuring both against the committed gate. +## 1098. The coordination hook prints a session UUID where a commit SHA is expected + +> 🔢 **Filed 2026-08-07 - not started.** Value **4/10** · Difficulty **2/10**. The parallel-session banner +> printed at session start lists each live worktree with what reads as a commit SHA in the trailing +> column. For at least one row the value is a **session UUID**: `061726a4` resolves to +> `.git/mefor-coord/announce/061726a4-....json`, not to any object in the repository. The branch +> names in the same banner are reliable; the SHAs are not. + +**Cluster:** Coordination tooling / instrument accuracy. **Priority:** P3. **Verdict:** build. +**Severity:** no product effect. The cost is that a reader who trusts the column reasons about the +wrong tree - and the banner exists precisely to tell sessions where other sessions are. + +**Why it matters more than the size suggests.** This is a value printed in the position a SHA +occupies, so it fails the SDS-3.8 test: the instrument answers a different question from the one the +column header asks, and nothing in the output says so. A session comparing "is that worktree ahead +of mine" against a UUID gets a `git` error at best and a wrong answer if the prefix happens to +resolve. Either print the real SHA or rename the column to what it actually holds; do not leave a +field whose meaning depends on which row you are reading. + +**Related:** #1000 (a control whose green is not evidence about what it appears to cover), +[`../CLAUDE.md`](../CLAUDE.md) §11 (SDS-3.8 - confirm the instrument answers the question asked). + +**Source:** observed 2026-08-07 during the #1095 work and deferred at the time only for ledger +contention; recorded in the #1095 handoff note rather than lost. + +## 1099. BACKLOG #1094 describes an archival pass that generates anchors; no archival tooling exists + +> 🔢 **Filed 2026-08-07 - not started.** Value **4/10** · Difficulty **1/10**. #1094 states that "the archival +> pass generates the anchor". There is **no archival tooling in the repository at all** - closing an +> item is a manual move of its text from [`BACKLOG.md`](BACKLOG.md) into +> [`archive/backlog/BACKLOG-CLOSED.md`](archive/backlog/BACKLOG-CLOSED.md). That sentence describes a +> generator that was never built. + +**Cluster:** Documentation record / instrument accuracy. **Priority:** P3. **Verdict:** build (a +prose correction). **Severity:** no product effect. + +**Why this is not pedantry.** #1094 is closed, so the sentence now sits in the archive as settled +record, and it points maintenance at the wrong place: it implies the fix for anchor rot belongs in a +tool, when the only thing that can catch it is a gate at the moment the item lands - which is exactly +the reasoning `tests/test_archive_link_resolution.py` +records. A future reader looking for the generator to fix will not find one. + +**The work.** Correct the sentence in place in the archive, marked as a correction rather than a +silent rewrite, since the surrounding text is a closed record. + +**Related:** #1094, #1095 (the repo-scale instance of the same class), #1000. + +**Source:** found 2026-08-07 while resolving the #1095 anchor classes; the absence of archival +tooling was confirmed by looking for it, not assumed. + +## 1100. The master test plan asserts document contradictions that were resolved before it was written + +> 🔢 **Filed 2026-08-07 - not started.** Value **6/10** · Difficulty **3/10**. Nine sites in the master test +> plan make **present-tense claims that the repository has already falsified**. The clearest cluster +> is five cells in +> [`15-alerting-and-observability.md`](testing/master-test-plan/15-alerting-and-observability.md), +> which assert that #139 and [`PHI.md`](PHI.md) "currently contradict each other" and set exit +> criteria requiring three documents be made "to agree with the code". #139 was corrected 2026-08-01 +> and the code fixed by #323 on 2026-08-02: `alert_sinks.py` builds a verifying context, +> `tests/test_alert_smtp_tls.py` exists, and `PHI.md` row 11 already states that posture. + +**Cluster:** Documentation record / instrument accuracy. **Priority:** P3. **Verdict:** build. +**Severity:** no product effect and no security effect - but a **test plan** is the wrong document to +carry dead assertions, because its rows are written to be executed. An exit criterion demanding that +a resolved contradiction be resolved is a gate that can never inform anyone. + +**The other four sites.** [`00-strategy-and-governance.md`](testing/master-test-plan/00-strategy-and-governance.md) +and [`19-execution-phasing-and-sign-off.md`](testing/master-test-plan/19-execution-phasing-and-sign-off.md) +both say [`BACKLOG.md`](BACKLOG.md) "says so itself" about the published-baseline boundary; that prose +now lives in **#185**, in the archive, and `grep` finds zero occurrences of it in the live file. The +ranked-backlog row at `BACKLOG.md` for **#338** is a pre-shipping snapshot still scoring it P3 and +`_fill-in_` after #338 shipped 2026-08-06 and its restatements were corrected. And #323's amendment +of **#139** in the archive carries a present-tense reading that #139's own text now contradicts. + +**Why it was split out of #1095 rather than fixed there.** #1095 converts dead +`BACKLOG.md:` anchors to item references. At these nine sites the pointer is dead **and so is +the sentence around it**. Converting the pointer would attach a durable, confident-looking reference +to a claim that is false - relaundering it. That is a strictly worse outcome than leaving the stale +anchor visible, so all nine were deliberately left unconverted and collected here. **The fix is a +content edit per site, not a repoint.** + +**Do not fix these with a script.** Each needs the claim checked against the code before the sentence +is rewritten, and at least one of the nine may warrant deleting the row rather than restating it. + +**Related:** #1095 (the anchor conversion that stopped at these nine), #139, #323, #338, #185, #1000 +(a control whose green is not evidence about what it covers), +[`../CLAUDE.md`](../CLAUDE.md) §0 (present-tense claims about a not-deployed beta) and §11 (SDS-3.7 - a +compensating control must not rest on a false premise). + +**Source:** found 2026-08-07 by the adversarial verification pass over the #1095 anchor mappings. +Every one of the nine was proposed as a clean repoint by a first-pass reader and refuted by a second +reader instructed to refute by default, who checked each citing claim against the code rather than +against the anchor.