diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 7d7ffdc4..8f9468b1 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -2765,7 +2765,7 @@ No test covers it. `tests/test_scan_tokens_source.py:559-583` (`test_absolute_ho 5. **Same fix site, sibling defect:** `_WORKTREE_SLUG` at `scripts/security/scan_forbidden.py:92` is case-blind the same way (`[a-z0-9]+`); an upper-cased slug — `claude/` followed by `Some-Task-a1b2c3` — is MISSED. (Written split on purpose, for the reason in the note above: once the fix lands, the joined literal trips the very detector it documents, and unlike `_HOME_PATH` the slug pattern has no `<…>` exemption to write it into.) `scripts/worktree/new.ps1:43,86` passes `-Name` through verbatim with no lowercasing, so an upper-cased worktree name is reachable. Narrower than the home-path case (agent-created slugs are lowercase by convention), but it is a two-character edit in the same block — take it in the same change or say why not. -**Related:** `scripts/security/scan_forbidden.py` (`_HOME_PATH` :99-106, `_WORKTREE_SLUG` :92, call site :758-759), `tests/test_scan_tokens_source.py:559-583`, `.github/workflows/security.yml:446-493`, `.github/required-contexts.txt`, `scripts/worktree/new.ps1`. Sibling **#321** — same gate, same "green gate that cannot see the class" root cause, but the **opposite mechanism**: #321 is an incomplete *token source* (data, fixed by the owner updating a private secret) and explicitly scopes itself away from scanner defects at `docs/BACKLOG.md:7356`; this is a *structural detector* defect (code, fixed by a regex edit) that is live even with no token source. Also **#322**, and the anonymizer's structural-detector item from this same audit. Note #321's **Related:** line at `docs/BACKLOG.md:7363` cites `tests/test_scan_forbidden.py` for regression tests, but the home-path test actually lives in `tests/test_scan_tokens_source.py` — worth correcting when someone next touches #321. +**Related:** `scripts/security/scan_forbidden.py` (`_HOME_PATH` :99-106, `_WORKTREE_SLUG` :92, call site :758-759), `tests/test_scan_tokens_source.py:559-583`, `.github/workflows/security.yml:446-493`, `.github/required-contexts.txt`, `scripts/worktree/new.ps1`. Sibling **#321** — same gate, same "green gate that cannot see the class" root cause, but the **opposite mechanism**: #321 is an incomplete *token source* (data, fixed by the owner updating a private secret) and explicitly scopes itself away from scanner defects; this is a *structural detector* defect (code, fixed by a regex edit) that is live even with no token source. Also **#322**, and the anonymizer's structural-detector item from this same audit. Note #321's **Related:** line cites `tests/test_scan_forbidden.py` for regression tests, but the home-path test actually lives in `tests/test_scan_tokens_source.py` — worth correcting when someone next touches #321. **Source:** public-repo disclosure audit, 2026-08-01. Verified open at HEAD (`12efbffc`) by executing the compiled pattern and by diffing the current, proposed and naive-`re.I` variants across every git-tracked file. @@ -3116,7 +3116,7 @@ _check_handler_security(cfg, strict=True) -> ok=True, skipped=True, "no handler ADR 0144:193-195 records the decorated-scope trade, but justifies it with an **`impure-transform`** false positive ("the trade that keeps the shipped `_pdf_mdm_transforms.py` timestamp fallback clean") and then applies it to `phi-to-log` as well. `samples/config/_demo_oru_transforms.py` and `_pdf_mdm_transforms.py` exist, [`docs/CONNECTIONS.md`](CONNECTIONS.md) §"Decomposing by role" tells authors to put field-level transform logic there, and #226 is an estate-wide sweep to do exactly that — so the one CLAUDE.md §9 rule the lint encodes systematically skips the file the convention steers PHI handling into. -**The third gap the audit named is narrower than described.** The non-recursive `base.glob("*.py")` at `checks.py:893`/`:898` (ADR 0144:196) is **not** an unscanned execution path. `load_config` globs `directory.glob("*.py")` non-recursively too (`config/wiring.py:3969`, and `:4392` for `validate_config`), and `_SiblingHelperFinder.find_spec` returns `None` for any dotted name and serves only `_`-prefixed top-level helpers from the config dir (`wiring.py:3902`, `:3908-3912`). A `.py` in a config subdirectory is therefore neither executed by the loader nor importable by a sibling — and `_assert_safe_config_source` is non-recursive for the same reason (`wiring.py:4194`, `:4324`). The lint's file set already equals the executable set. A recursive walk here would make the lint report on files the safe-source ownership gate never vets — an asymmetry in the other direction. #226 (`docs/BACKLOG.md:6917`) already parks recursion as a *loader* question; it belongs there, not here. +**The third gap the audit named is narrower than described.** The non-recursive `base.glob("*.py")` at `checks.py:893`/`:898` (ADR 0144:196) is **not** an unscanned execution path. `load_config` globs `directory.glob("*.py")` non-recursively too (`config/wiring.py:3969`, and `:4392` for `validate_config`), and `_SiblingHelperFinder.find_spec` returns `None` for any dotted name and serves only `_`-prefixed top-level helpers from the config dir (`wiring.py:3902`, `:3908-3912`). A `.py` in a config subdirectory is therefore neither executed by the loader nor importable by a sibling — and `_assert_safe_config_source` is non-recursive for the same reason (`wiring.py:4194`, `:4324`). The lint's file set already equals the executable set. A recursive walk here would make the lint report on files the safe-source ownership gate never vets — an asymmetry in the other direction. #226 already parks recursion as a *loader* question; it belongs there, not here. **Why:** Bounded, and bounded hard. The lint is advisory by default (`checks.py:953`, `ok=not strict, required=strict`), so a finding blocks nobody unless an adopter opts into `--strict-handler-security` on their own CI. It governs code the adopter's own administrator authors, inside a directory whose write access is already the trust boundary (`_assert_safe_config_source`, `wiring.py:4194`/`:4324`) — anyone who can drop a `.py` there already has arbitrary in-process execution under the engine account, so this is **not** a privilege boundary and evading it buys an attacker nothing they did not already have. @@ -6464,12 +6464,36 @@ What was unwatched is the **release-lag window**: the interval between a fix lan **At least four distinct classes, and they differ in DETECTABILITY. This is the item's central constraint** — state it in any plan, or a linter closes the visible third and the rest is declared done: -| Class | Count (2026-08-07) | Catchable by a link checker? | -|---|---|---| -| Path-bearing citation names the live ledger; item is archived | at least 69 sites / 35 files | **No** — the link resolves perfectly. Only the human-readable number beside it is stale. | -| Relative href does not resolve at all | 13 | Yes | -| `BACKLOG.md:` anchor past end of file | 12 (file is 6318 lines; one anchor cites **8429**) | Yes | -| `BACKLOG.md:` anchor in range but drifted onto unrelated text | 31 | **No** — resolves, lands somewhere plausible, says the wrong thing. | +| Class | Count as filed | Re-measured repo-wide | Catchable by a link checker? | Status | +|---|---|---|---|---| +| Path-bearing citation names the live ledger; item is archived | at least 69 sites / 35 files | 73 sites / 34 files | **No** — the link resolves perfectly. Only the human-readable number beside it is stale. | repaired | +| Relative href does not resolve at all | 13 | **at least 629** | Yes | repaired | +| `BACKLOG.md:` anchor past end of file | 12 | 7 | Yes | converted to item references | +| `BACKLOG.md:` anchor in range but drifted onto unrelated text | 31 | 36 of 43 | **No** — resolves, lands somewhere plausible, says the wrong thing. | 25 converted, 18 left deliberately | + +**Three of those four filed counts were wrong, and the way they were wrong is the item's own subject.** +The href figure of 13 was low by two orders of magnitude: the repaired sites total at least 629 (333 +root-relative hrefs under `testing/`, 270 broken by the archival move, 3 ADR-slug and depth errors, +23 `file:line` targets). The anchor counts moved in the *opposite* direction between filing and +repair, and not because anything improved — this file grew 6,318 → 6,616 lines, so five anchors that +were safely past EOF came back **into** range and now land on plausible-looking wrong items. A +dangling pointer degrades into a confident one as the file grows. + +**Root cause of the anchor class, which is not gradual drift.** Commit `4ea15017` (the master test +plan) is **not** a descendant of `03f1fbd7` (the 185-item archival) — they were parallel branches. +The plan's anchors were authored against an 8,742-line ledger and merged alongside a commit that cut +this file to 3,858. They were stale on arrival. That also explains the anchor citing line **8429**, +which is absurd against 6,616 lines and was ordinary against 8,742. + +**18 anchors were left unconverted on purpose**, in three groups. Nine sites make a present-tense +claim that the repo has since falsified — the alerting chapter still says #139 and `PHI.md` +"currently contradict each other" when #139 was corrected 2026-08-01 and the code fixed by #323 on +2026-08-02 (filed as #1100). Attaching a durable item reference to a dead claim relaunders it, so +those need a content fix, not a repoint. Six cannot be attributed to any item at all, one of them +pointing at an un-numbered narrative bullet; they stay uniformly stale, because uniform staleness is +detectable and a confident wrong pointer is not. Three are not citations — two sit inside a fenced +transcript in #1083 reproducing scanner output, and one is #347's own **Source:** paragraph +narrating a deliberate falsification. The two "No" rows are the majority and the harder half. Root cause for the anchor classes is the same archival pass that produced the first: it shortened the ledger under citations that had been correct. @@ -6481,9 +6505,11 @@ The two "No" rows are the majority and the harder half. Root cause for the ancho **Already specified elsewhere — do not file a third time.** In [`testing/master-test-plan/18-interop-migration-and-uat.md`](testing/master-test-plan/18-interop-migration-and-uat.md): `MIG-74` is a P1 row reading *"Every doc path named in shipped text or a docstring resolves — one linter, one boundary allowlist"*, and **`MIG-35` is already named there as "the BACKLOG-reference classifier", folding into MIG-74** (:128) — so the exact check this item needs is specified, just unbuilt. This item supplies what those rows lack: the measured counts, the detectability split, and the warning that `MIG-74` as worded ("paths resolve") would pass the **largest** class untouched, because those paths *do* resolve. -**Scope note — markdown only.** The sweep behind these numbers read `*.md`. Citations also live in `harness/`, `ide/src/`, `messagefoundry/` and `.github/workflows/`, and rot identically; they are **not** counted above and should be swept before anyone calls the class closed. +**Scope note — the non-markdown sweep has now RUN, and it found nothing.** The counts as filed read `*.md` only, and this note used to say citations in `harness/`, `ide/src/`, `messagefoundry/` and `.github/workflows/` "rot identically" and must be swept before the class could be called closed. That sweep ran 2026-08-07 across 1,219 tracked non-markdown files. It surfaced 68 distinct repo-relative paths that do not exist and **not one of them is a rotted citation**: they are test fixtures (`docs/adr/0001-collision.md`, `messagefoundry/x.py`, `tests/test_a.py`), targets inside withheld directories, or past-tense historical comments — `.github/workflows/release.yml:51` names the pre-cutover `scripts/publish/publish.ps1` under a heading reading "THIS GUARD WAS INVERTED UNTIL THE CUTOVER". The prediction was wrong, and the reason is worth keeping: citations in code are mostly written *about* the past, where prose citations are written as *pointers*. **Do not re-open this as unswept scope.** + +**A gate now covers the catchable half, repo-wide.** `scripts/docs/link_check.py` with `tests/test_archive_link_resolution.py` asserts that every relative markdown link in the repository resolves — 5,327 links across 347 files. Widening it beyond the archive required removing three false-positive classes first, because a gate red on arrival gets suppressed rather than fixed: links inside inline code (a regex whose character class contains `](`, two VS Code `command:` URIs, and ADR 0160 quoting the link it records as removed), and the withheld `docs/releases/` and `.claude/`. **`.claude/` is the one to carry forward** — it is gitignored but *present* in a long-lived local checkout, so the first repo-wide measurement passed there and undercounted by 7; widening on that number would have put the gate red on CI's clean clone. A gate result is a fact about the configuration it ran in. The gate was then proved able to fail, by planting a break in a real document outside the archive and watching it go red. -**Related:** #1094 (two instances of this class, closed — already satisfied when filed), #1073 (whose §12 marker prompted the original find, and which is itself the closed-but-not-archived case), #1000 (a control whose green is not evidence about what it appears to cover), [`../CLAUDE.md`](../CLAUDE.md) §11 (state a load-bearing fact once and link to it — which is what makes a link's durability load-bearing). +**Related:** #1094 (two instances of this class, closed — already satisfied when filed), #1073 (whose §12 marker prompted the original find, and which is itself the closed-but-not-archived case), #1000 (a control whose green is not evidence about what it appears to cover), #1099 (#1094's claim that an archival pass *generates* the anchor, describing tooling that does not exist), #1100 (the nine sites whose citing claim is dead, split out of this item rather than repointed), [`../CLAUDE.md`](../CLAUDE.md) §11 (state a load-bearing fact once and link to it — which is what makes a link's durability load-bearing). **Source:** repo-wide sweep 2026-08-07 while repointing the [`CONNECTIONS.md`](CONNECTIONS.md) serial/ASTM decline (PR #273, the `docs/CONNECTIONS.md` half of the same two-marker pair #1094 covered in `../CLAUDE.md`). Item locations came from `parse_items` imported from the status-check script rather than a hand-rolled scan; the resolver was validated on planted good-and-bad citations, including the closed-but-not-yet-archived case, before its output was believed. @@ -6614,3 +6640,96 @@ So `-Com` and `-Comm` are working spellings. `-Cm` is not, which bounds the fami **Source:** found 2026-08-07 by the adversarial verification of a candidate fix for #1086. Two verifiers independently measured the abbreviation family; the third accepted the candidate because its corpus contained no such payload, which is itself the #1000 shape. The pre-existing half was separated from the introduced half by measuring both against the committed gate. +## 1098. The coordination hook prints a session UUID where a commit SHA is expected + +> 🔢 **Filed 2026-08-07 - not started.** Value **4/10** · Difficulty **2/10**. The parallel-session banner +> printed at session start lists each live worktree with what reads as a commit SHA in the trailing +> column. For at least one row the value is a **session UUID**: `061726a4` resolves to +> `.git/mefor-coord/announce/061726a4-....json`, not to any object in the repository. The branch +> names in the same banner are reliable; the SHAs are not. + +**Cluster:** Coordination tooling / instrument accuracy. **Priority:** P3. **Verdict:** build. +**Severity:** no product effect. The cost is that a reader who trusts the column reasons about the +wrong tree - and the banner exists precisely to tell sessions where other sessions are. + +**Why it matters more than the size suggests.** This is a value printed in the position a SHA +occupies, so it fails the SDS-3.8 test: the instrument answers a different question from the one the +column header asks, and nothing in the output says so. A session comparing "is that worktree ahead +of mine" against a UUID gets a `git` error at best and a wrong answer if the prefix happens to +resolve. Either print the real SHA or rename the column to what it actually holds; do not leave a +field whose meaning depends on which row you are reading. + +**Related:** #1000 (a control whose green is not evidence about what it appears to cover), +[`../CLAUDE.md`](../CLAUDE.md) §11 (SDS-3.8 - confirm the instrument answers the question asked). + +**Source:** observed 2026-08-07 during the #1095 work and deferred at the time only for ledger +contention; recorded in the #1095 handoff note rather than lost. + +## 1099. BACKLOG #1094 describes an archival pass that generates anchors; no archival tooling exists + +> 🔢 **Filed 2026-08-07 - not started.** Value **4/10** · Difficulty **1/10**. #1094 states that "the archival +> pass generates the anchor". There is **no archival tooling in the repository at all** - closing an +> item is a manual move of its text from [`BACKLOG.md`](BACKLOG.md) into +> [`archive/backlog/BACKLOG-CLOSED.md`](archive/backlog/BACKLOG-CLOSED.md). That sentence describes a +> generator that was never built. + +**Cluster:** Documentation record / instrument accuracy. **Priority:** P3. **Verdict:** build (a +prose correction). **Severity:** no product effect. + +**Why this is not pedantry.** #1094 is closed, so the sentence now sits in the archive as settled +record, and it points maintenance at the wrong place: it implies the fix for anchor rot belongs in a +tool, when the only thing that can catch it is a gate at the moment the item lands - which is exactly +the reasoning `tests/test_archive_link_resolution.py` +records. A future reader looking for the generator to fix will not find one. + +**The work.** Correct the sentence in place in the archive, marked as a correction rather than a +silent rewrite, since the surrounding text is a closed record. + +**Related:** #1094, #1095 (the repo-scale instance of the same class), #1000. + +**Source:** found 2026-08-07 while resolving the #1095 anchor classes; the absence of archival +tooling was confirmed by looking for it, not assumed. + +## 1100. The master test plan asserts document contradictions that were resolved before it was written + +> 🔢 **Filed 2026-08-07 - not started.** Value **6/10** · Difficulty **3/10**. Nine sites in the master test +> plan make **present-tense claims that the repository has already falsified**. The clearest cluster +> is five cells in +> [`15-alerting-and-observability.md`](testing/master-test-plan/15-alerting-and-observability.md), +> which assert that #139 and [`PHI.md`](PHI.md) "currently contradict each other" and set exit +> criteria requiring three documents be made "to agree with the code". #139 was corrected 2026-08-01 +> and the code fixed by #323 on 2026-08-02: `alert_sinks.py` builds a verifying context, +> `tests/test_alert_smtp_tls.py` exists, and `PHI.md` row 11 already states that posture. + +**Cluster:** Documentation record / instrument accuracy. **Priority:** P3. **Verdict:** build. +**Severity:** no product effect and no security effect - but a **test plan** is the wrong document to +carry dead assertions, because its rows are written to be executed. An exit criterion demanding that +a resolved contradiction be resolved is a gate that can never inform anyone. + +**The other four sites.** [`00-strategy-and-governance.md`](testing/master-test-plan/00-strategy-and-governance.md) +and [`19-execution-phasing-and-sign-off.md`](testing/master-test-plan/19-execution-phasing-and-sign-off.md) +both say [`BACKLOG.md`](BACKLOG.md) "says so itself" about the published-baseline boundary; that prose +now lives in **#185**, in the archive, and `grep` finds zero occurrences of it in the live file. The +ranked-backlog row at `BACKLOG.md` for **#338** is a pre-shipping snapshot still scoring it P3 and +`_fill-in_` after #338 shipped 2026-08-06 and its restatements were corrected. And #323's amendment +of **#139** in the archive carries a present-tense reading that #139's own text now contradicts. + +**Why it was split out of #1095 rather than fixed there.** #1095 converts dead +`BACKLOG.md:` anchors to item references. At these nine sites the pointer is dead **and so is +the sentence around it**. Converting the pointer would attach a durable, confident-looking reference +to a claim that is false - relaundering it. That is a strictly worse outcome than leaving the stale +anchor visible, so all nine were deliberately left unconverted and collected here. **The fix is a +content edit per site, not a repoint.** + +**Do not fix these with a script.** Each needs the claim checked against the code before the sentence +is rewritten, and at least one of the nine may warrant deleting the row rather than restating it. + +**Related:** #1095 (the anchor conversion that stopped at these nine), #139, #323, #338, #185, #1000 +(a control whose green is not evidence about what it covers), +[`../CLAUDE.md`](../CLAUDE.md) §0 (present-tense claims about a not-deployed beta) and §11 (SDS-3.7 - a +compensating control must not rest on a false premise). + +**Source:** found 2026-08-07 by the adversarial verification pass over the #1095 anchor mappings. +Every one of the nine was proposed as a clean repoint by a first-pass reader and refuted by a second +reader instructed to refute by default, who checked each citing claim against the code rather than +against the anchor. diff --git a/docs/adr/0074-adopter-capacity-estimator.md b/docs/adr/0074-adopter-capacity-estimator.md index d78d7a4d..5588f438 100644 --- a/docs/adr/0074-adopter-capacity-estimator.md +++ b/docs/adr/0074-adopter-capacity-estimator.md @@ -176,7 +176,7 @@ If the halved rate is the **daily average**, the peak hour lands at 0.5 × 2.7 = `harness/load/runner.py:59-65` constructs **one** `CorrelationSink` with `ports=tuple(sink_port + i for i in range(sink_ports))` — those are *ports inside one process*, and the ~135–144 msg/s cap is **per sink *process***. In the path §Method names, the cap **cannot be relieved at all**. Multi-*process* sinks exist only in `shardcert.py`'s drive, which this ADR never mentions, and it fails loud when `sink_count > dests` — so the **~36 msg/s single-hospital adopter with 1–2 outbound destinations can provision at most 1–2 sink processes**, hard-capping the *measurable* delivered rate with no remedy. **B14 — Downstream artifacts carry the stale text and would ship the gate holes even if only this ADR is amended.** — ✅ **RESOLVED: all four now corrected** (see item 13 below). -`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; `docs/BACKLOG.md:4077-4079` (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule). +`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; BACKLOG #96 (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule). --- diff --git a/docs/testing/master-test-plan/00-strategy-and-governance.md b/docs/testing/master-test-plan/00-strategy-and-governance.md index cfa0e207..aa914482 100644 --- a/docs/testing/master-test-plan/00-strategy-and-governance.md +++ b/docs/testing/master-test-plan/00-strategy-and-governance.md @@ -204,7 +204,7 @@ drives): | `harness/load` (`--load` / `--failover` / `--estate`) | Performance, resilience | Reports metrics only; SLO verdicts and exit codes per [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) §"Exit codes" | | `harness/acceptance` (`python -m harness.acceptance`) | Host acceptance | Executes probes + re-runs backing pytest suites; **never fakes green** — unautomatable rows report `MANUAL` | | `harness/reconcile` (`capture` / `compare`) | Migration / parallel run | Offline comparison core; normalizes engine-non-deterministic fields; exits non-zero on a real diff so it can gate a per-Connection sign-off | -| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](messagefoundry/verify/runner.py:22)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message | +| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](../../../messagefoundry/verify/runner.py)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message | | `messagefoundry check` | Pre-merge config gate | Required checks `validate`, `dryrun`, `posture`, `build-check`, `reference-backend`; `ruff`/`mypy`/`raise-fstring`/`accepts-candidate` are **advisory and never block** | | CI legs (`ci.yml`) | All automated levels | `test` + `ide` + `ci-gate` on every PR; the heavy legs (`sqlserver-store`, `postgres-store`, `load-test`, `load-test-sqlserver`, `windows-service-smoke`, `docker-smoke`) run **nightly at 03:17 UTC** or on `workflow_dispatch`, plus a PR path-gate arm where `changes` provides one | | `quality-advisory.yml` | Meta / test-quality | `complexity`, `clone`, `coverage` (diff-cover), `mutation` — all **advisory, never required**; the `liveness` job is the meta-gate that demands proof a signal actually executed | diff --git a/docs/testing/master-test-plan/04-high-availability-and-dr.md b/docs/testing/master-test-plan/04-high-availability-and-dr.md index 82b91f66..cceceefa 100644 --- a/docs/testing/master-test-plan/04-high-availability-and-dr.md +++ b/docs/testing/master-test-plan/04-high-availability-and-dr.md @@ -94,7 +94,7 @@ bare `HA-nn` in this chapter always means **this chapter's own §3.4 row** — t | The PG store-side epoch guard is live-proven nowhere | `test_resumed_ex_leader_is_fenced_after_real_handover` exists in both live suites but only the SS one runs. The PG claim guard appears in **three** separate statements (`postgres.py:2686-2688, 2816-2820, 2934-2938`). | One missed append across three claim paths is invisible to fake-pool tests and shows up only as duplicate/out-of-order delivery under a real handover | No | P1 | | DR readiness is asserted nowhere on the box that needs it | `messagefoundry verify`'s sections are exactly `("host", "store", "smoke", "manual", "federation")` (`messagefoundry/verify/runner.py:22`) — no cluster, HA or DR group. `harness/acceptance/matrix.py:429-470` has G1-G5 with no DR row; `WIN2025-TEST-MATRIX.md` §G likewise. | An operator gets zero signal about DEK availability at the DR site, backup-destination writability, or cluster posture | No | P1 | | Windows port-rebind lag is a one-off manual number | Captured only as prose in `W25:S4.9`; not in the report JSON, no trend, no threshold. | On the intended production OS this dominates observed recovery; a regression is indistinguishable from normal | Manual only | P2 | -| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. `BACKLOG.md:152` (#100) still reads "P1" though it shipped 2026-07-10 (`BACKLOG.md:4229`); `BACKLOG.md:159` (#101) reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 | +| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. BACKLOG #100's ranked-backlog Tier cell still reads "P1" though it shipped 2026-07-10 per its own SHIPPED banner; BACKLOG #101's ranked-backlog Tier cell reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 | ### 3.4 Test matrix @@ -180,7 +180,7 @@ deliverables that STORE-44 and STORE-46 point at — do not descope them here. | HA-56 | Config-only `.mfbak` + `DbaDelegatedError` on a live server DB — **pointer** | Cross-backend | — | — | — | T | P1 | Covered by STORE-45; no separate work scoped. (The files themselves reach CI through HA-02's wiring; the assertion is STORE's.) | | HA-57 | FEATURE-MAP drift guard for Accepted HA/DR ADRs — **pointer** | Compat | — | — | — | T | P1 | Covered by the MIG FEATURE-MAP drift-guard row (one consolidated row extending `tests/test_feature_map_claims.py`); no separate work scoped. HA's contribution is the input list — ADR 0047, 0048, 0049, 0096, 0102 named in `docs/FEATURE-MAP.md` §6, and ADR 0056 carried with a non-shipped marker. | | HA-58 | Correct the two stale FEATURE-COVERAGE-PLAN rows | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `FCP:HA-20`'s "Web console has no HA surface / coverage: none" (§16, `FEATURE-COVERAGE-PLAN.md:1201`) is corrected to reflect `monitoring.py:471-508` + `routes/status.py:88-97` + `test_webui.py:1437-1532`; `FCP:DEPLOY-26`'s "coverage: none" (§24, line 1552) is reconciled with the same document's P6 closure (line 317). Outcome is a dated doc edit, not a falsifiable assertion — the recurring guard is HA-57's owner row. | -| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `docs/BACKLOG.md:152` (#100, `MultiSubnetFailover`) changes from "P1" to SHIPPED, consistent with its own detail section at line 4229; `docs/BACKLOG.md:159` (#101, leader preference / non-promotable) changes from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. | +| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: BACKLOG #100 (`MultiSubnetFailover`) changes its ranked-backlog Tier cell from "P1" to SHIPPED, consistent with its own SHIPPED banner; BACKLOG #101 (leader preference / non-promotable) changes its ranked-backlog Tier cell from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. | | HA-60 | Live PG expired-lease reclaim sweep does not steal a live sibling's rows | HA/Resilience | pytest | container-CI | x2 (PG) | T | P1 | On real PostgreSQL with two owners: rows leased by a crashed owner past `lease_ttl_seconds` are re-pended by `reclaim_expired_leases`, while rows leased by a live, renewing owner are untouched. Asserts the multi-node-safety claim in `leader_tasks.py:129-133` against real SQL rather than the single-node path. | | HA-61 | `[store].lease_ttl_seconds` is inert on SQL Server | Cross-backend | pytest | container-CI | x2 (SS) | T | P2 | On a SQL Server store the setting changes no observable claim/recovery behaviour, and `reclaim_expired_leases` is absent from the SS store surface, so the SS coordinator's `reclaims_inflight()` path never reaches it. Locks in the documented asymmetry (`AOAG-DEPLOYMENT.md:527-531`) that operators otherwise mis-tune. | | HA-62 | Live PG reference-set and transform-state convergence across nodes | Cross-backend | pytest | container-CI | x2 (PG) | T | P1 | On a real two-node PG cluster: the leader materializes a declared `Reference(...)` set, the follower read-throughs the shared snapshot without re-materializing; a transform-state write on one node bumps the namespace `state_version` and reaches the sibling's cache within one convergence interval. Today only the SS half has live coverage (`test_sqlserver_coordinator.py:183-200`). | diff --git a/docs/testing/master-test-plan/06-parsing-and-codecs.md b/docs/testing/master-test-plan/06-parsing-and-codecs.md index 94135cf1..f7ac6184 100644 --- a/docs/testing/master-test-plan/06-parsing-and-codecs.md +++ b/docs/testing/master-test-plan/06-parsing-and-codecs.md @@ -79,7 +79,7 @@ | Codec extras silently uninstall | 41 test files use `importorskip`; there is no skip-counting guard in `tests/conftest.py` or `pyproject.toml` `addopts` | ~100 DICOM/FHIR/X12/XML/WSDL tests vanish and the build is GREEN. `pyproject.toml:119-123` documents exactly this class of breakage (annotated-types 0.8.0 dropping `SLOTS`) already happening once | No — the existing plan flags it for `FCP:X12-6` only | P1 | | `parse_tree()` has its own separator derivation | `tree.py:65-72` blind-slices `msh[4:8]` and takes `sub_sep = enc[3]`. **Verified divergence:** for a legal 3-character MSH-2 (`^~\`), the tree derives `sub_sep = '\|'` (the *field* separator) while `_extract_separators` correctly returns `'&'`. Leaves are also never unescaped, so the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien` | The operator's only structural view of what actually arrived renders wrong in **both** the web console and the harness Qt pane, while routing stays correct — nothing else signals it | No. `test_parse_tree.py` is 8 tests over one default-separator ADT | P1 | | ADR 0054 AC-6 (≥6× multi-core, ~14× single-thread on cp314t) has no executable gate | `test_benchmark_parser.py:207-235` is `skipif`-not-freethreaded and asserts only `ratio > 1.0`; `freethread-smoke.yml` runs only `test_parsing.py` + `test_wiring.py`, so the measure never executes anywhere | The entire justification for reimplementing python-hl7 (ADR 0053/0052) is unverified on any automated surface; only the crude `> 200 msg/s` floor would catch an orders-of-magnitude fall | Partially — the floor exists, the scaling gate does not | P1 | -| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `` selector. Disclosed at `docs/BACKLOG.md:3188` | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 | +| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `` selector. Disclosed at BACKLOG #69 | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 | | No malformed/oversized corpus generator | `generators/_core.generate_message` emits only hl7apy-conformant bodies and raises on an unknown trigger; `messagefoundry generate` exposes only `--type/--triggers/--count/--out/--seed/--list/--json` (`__main__.py:349-362`) | The error/dead-letter path — the count-and-log invariant itself — is never exercised at load or repeatably. Both P0s above are exactly the shapes such a generator would emit. Blocks automating `W25:S4.8` (`WIN2025-TEST-PLAN.md:926,1044,1351`) | No — `W25:S4.8` bad input is GUI-injected manually | P1 | | `ValidationResult.errors` unasserted PHI-redacted | `validate.py:91` appends `str(exc)` from hl7apy verbatim; `wiring_runner.py:3659` does pass it through `safe_text` before persisting, but nothing pins that the **library** result is safe for other callers (dry-run, IDE, harness) | A datatype/table conformance failure can carry the offending field VALUE into any surface that renders `result.errors` directly | Partly — the persisted path is scrubbed; the library return is not asserted | P2 | | No property-based or coverage-guided fuzzing | `pyproject.toml` carries no hypothesis/atheris; both "fuzz" corpora (`test_parsing.py:228-244`, 15 cases; `test_builtin_hl7_hardening.py`) are hand-enumerated. `docs/quality-gates/HANDOFF-mutation-coverage.md:223` proposes mutmut over `parsing/` but is a handoff doc only | Hand-enumerated corpora find only shapes an author imagined — demonstrated by both P0s, each trivially reachable and each missed. Parsing is the largest attacker-influenced surface in the product | No | P2 | @@ -390,6 +390,6 @@ 9. **Does the code-set `UnmappedSink` persistence half get built?** If yes it creates a new PHI-at-rest surface needing encryption, audit and retention decisions, and the `_sink is None` pin (PARSE-48) becomes a pin on the *default*, not on the absence of the feature. **Blocks:** PARSE-48's framing. 10. **Should `parse_tree()` unescape leaf values so the operator's view matches the router's?** Today the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien`. Either answer is defensible; neither is written down or asserted. **Blocks:** PARSE-24, PARSE-51, PARSE-52. 11. **Does `validate(profile=…)` stay a silent no-op (`validate.py:56`), or raise `NotImplementedError` until conformance profiles exist?** A caller passing a profile today gets an `ok=True` that means nothing on a safety-critical strict feed. **Blocks:** PARSE-19. -12. **Is the WSDL `` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at `docs/BACKLOG.md:3188`. **Blocks:** PARSE-31 — a test written now would encode the defect. +12. **Is the WSDL `` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at BACKLOG #69. **Blocks:** PARSE-31 — a test written now would encode the defect. 13. **Should the stale ratings and doc drift be corrected as part of this work?** Concretely: `FEATURE-COVERAGE-PLAN.md` `FCP:PARSE-14` (`:859`) says "partial" (closed by `test_builtin_hl7_hardening.py:595-703`) and `FCP:PARSE-15` (`:860`) says the bench file is "MISSING" (it exists, 235 lines); `docs/HL7-VALIDATION.md:10` still names python-hl7 as the Tier-1 engine though ADR 0054's built-ins parser is the default; ADR 0012 §5 still calls the X12 strict validator deferred though `parsing/x12/validate.py` ships; ADR 0033's header still reads "Proposed … ratified-on-build" though the code sets are built; and `docs/FEATURE-MAP.md` §3 omits ADR 0054, 0122, 0123, 0033, `parsing/xml/`, `consistency.py`, `split.py` and `sniff.py`. **Blocks:** nothing technically — but the acceptance report and this plan both cite those documents as authority. 14. **Is a silent U+FFFD substitution an acceptable ingress outcome?** `normalize()` defaults to `errors="replace"` (`peek.py:152`) precisely so the hot path keeps routing a slightly-off body — but a substitution is an unrecorded, irreversible corruption of every *derived* value while the raw stays byte-faithful, which is count-and-log-clean by construction. Options: keep `replace` and record a warning/`ERROR` disposition when the decoded text differs from a strict decode; switch the ingress decode to `errors="strict"` and let it NAK (PARSE-43's path); or declare it accepted-by-design in `docs/CONNECTIONS.md`. **Blocks:** PARSE-64's unconditional half and the framing of PARSE-43. diff --git a/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md b/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md index 15d9dce1..81785d36 100644 --- a/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md +++ b/docs/testing/master-test-plan/10-auth-rbac-and-active-directory.md @@ -42,7 +42,7 @@ This chapter covers the whole authentication/authorization core and, specificall **Objective.** Move the four directory-facing P0 exposures from "asserted against a mock" to "observed against a real domain controller, an AD FS farm and a real browser", and close the cross-backend and meta-guard holes that let a silent regression through CI. -**Objective's hard constraint — the AD lab has not been stood up, so the gate is split.** Nine of this chapter's eighteen P0 rows can only run against a real Domain Controller / AD FS farm, and **that rig does not exist yet**. Standing it up is the genuine blocker — *not* a missing document. The lab's authority document (`docs/security/AD-FEDERATION-LAB-RUNBOOK.md`, cells L0–L18) exists for the owner; the whole `docs/security/` tree is simply **withheld from the public repo** (`.gitignore:144`; `docs/BACKLOG.md:271` records it as "gitignored post-cutover", and `:6041` states outright that such absences are "a publishing boundary, not evidence of completion"), so it is not readable from this worktree. A single undifferentiated P0 set would nonetheless be **unsatisfiable today**, which makes the release gate meaningless rather than strict. §9.4 splits it into a **P0-automated** gate that blocks every release and a **P0 AD campaign gate** that blocks only a release claiming AD / Kerberos / federated-SSO support; §9.7 carries **standing the lab up** — plus the chapter author's read access to the withheld runbook — as the blocking environment-phase exit item. +**Objective's hard constraint — the AD lab has not been stood up, so the gate is split.** Nine of this chapter's eighteen P0 rows can only run against a real Domain Controller / AD FS farm, and **that rig does not exist yet**. Standing it up is the genuine blocker — *not* a missing document. The lab's authority document (`docs/security/AD-FEDERATION-LAB-RUNBOOK.md`, cells L0–L18) exists for the owner; the whole `docs/security/` tree is simply **withheld from the public repo** (`.gitignore:144`; BACKLOG #185 records it as "gitignored post-cutover" and states outright that such absences are "a publishing boundary, not evidence of completion"), so it is not readable from this worktree. A single undifferentiated P0 set would nonetheless be **unsatisfiable today**, which makes the release gate meaningless rather than strict. §9.4 splits it into a **P0-automated** gate that blocks every release and a **P0 AD campaign gate** that blocks only a release claiming AD / Kerberos / federated-SSO support; §9.7 carries **standing the lab up** — plus the chapter author's read access to the withheld runbook — as the blocking environment-phase exit item. ### 9.2 Already covered — do not re-test @@ -105,7 +105,7 @@ This chapter covers the whole authentication/authorization core and, specificall | Hand-maintained gate exemption sets have no derived drift guard: `_MUST_CHANGE_EXEMPT_PATHS` (`api/security.py:54`) and `_MFA_EXEMPT_ROUTES` (`:72`). `tests/test_mfa_access_gate.py:145` pins the method-keyed **shape**, explicitly not membership against the live route table | Adding a path, or renaming a route so an exemption silently widens (the `DELETE /me/mfa` vs `GET /me/mfa` trap the code comment itself calls out at `:58-61`), lets a half-authenticated session reach a sensitive route | Authenticated-but-unverified privilege escalation | Shape only | **P2** | | AD group → **custom** role mapping is only indirect. `roles_for_ad_groups` can return a `custom:`-prefixed id and `_custom_permissions_for_ids` resolves it (`auth/service.py:1592`, used at `:1103`), but no test drives an AD login whose group maps to a custom role | Silent mis-resolution grants an AD operator nothing (lockout) or more than intended; the audit row records only the opaque role id, so the over-grant is illegible after the fact | Any org using custom roles with AD | **No** | **P2** | | No mutation or diff-coverage signal over the auth suite. `docs/quality-gates/HANDOFF-mutation-coverage.md` is an explicit **DRAFT / ready-to-run**; `.github/workflows/quality-advisory.yml` ships only complexity + clone gates | ~480 auth assertions of unmeasured strength guard the security boundary; a weakened predicate (an `or` where an `and` belongs inside a gate) can survive the whole suite | Any auth gate | **No** | **P2** | -| AD service-account least-privilege and gMSA logon rights unproven. `#99(e)` is the only residual and is provisioning-gated (`docs/BACKLOG.md:177`, `:4172`); nothing tests or documents the minimum directory rights the bind account needs | Deployments over-privilege the bind account; a compromised engine host then holds a directory-**write** credential | Directory-wide, on host compromise | **No** | **P2** | +| AD service-account least-privilege and gMSA logon rights unproven. `#99(e)` is the only residual and is provisioning-gated (BACKLOG #99); nothing tests or documents the minimum directory rights the bind account needs | Deployments over-privilege the bind account; a compromised engine host then holds a directory-**write** credential | Directory-wide, on host compromise | **No** | **P2** | | Hardened-DC interop untested. A DC with `LdapEnforceChannelBinding` / LDAP signing required may refuse the simple binds `auth/ldap.py` issues; the engine passes no `channel_bindings` anywhere (#98 unbuilt, `auth/ldap.py:299-303`) | Every AD login becomes a generic `LdapError` with no engine-side classification distinguishing it from an outage — an unsupportable first-contact failure | Any customer on Microsoft's hardened LDAP defaults | **No** | **P2** | | Reconciler default drift. `config/settings.py:1815` ships `ad_session_recheck_seconds = 300`, but `api/app.py:5620` comments "Default OFF (`ad_session_recheck_seconds = 0`)" and `auth/service.py:1189-1191` repeats the claim | An operator reading the code believes no directory traffic is generated; enabling `ad_enabled` silently starts a 300 s bind loop against their DC | Unexpected DC load; misjudged blast radius during an outage | **No** — the setting default is asserted, the two comments are not | **P2** | | No validator floor on any limiter. `docs/SECURITY.md:1571-1575` records that none of the eleven `*_rate_limit_*` fields nor `lockout_threshold`/`lockout_minutes` carries a Pydantic validator, so `per_key=0` or `window_seconds=0` disables enforcement **while the limiter still reports enabled** | A typo or a copy-pasted profile silently removes brute-force protection with no startup complaint | Password spray against the whole user base | Documented, not asserted | **P2** | @@ -135,7 +135,7 @@ Nine of the eighteen P0 rows sit behind the AD campaign gate, and that gate is * | AUTH-06 | Referral posture pinned statically at both `ldap3.Connection` sites | Negative/Security | pytest | dev-PC | n/a | T | P1 | An AST walk (modelled on `tests/test_ldap_timeouts.py:267`) finds both `Connection(...)` sites in `auth/ldap.py` and asserts each carries an **explicit** `auto_referrals` keyword whose value is not a bare literal `True`; the walker self-test proves it sees the bare-import call form | | AUTH-07 | Referral chase does not redirect the password-verifying bind to an untrusted target | Negative/Security | manual | AD-lab | n/a | T | P1 | With a referral configured from domain A to domain B, a login as an A user either (a) never opens a bind to B carrying the user's password, or (b) is refused; captured by a packet count / connection log on the B side showing **zero** simple binds with the user's DN | | AUTH-08 | Real-domain AD login end to end with RBAC applied and the AD identity on the audit row | Functional | manual | AD-lab | SQLite | T | P0 | Domain user `jdoe@mefor.lab` authenticates; a role-gated route returns 200 and a route outside the role returns 403; an `auth.login_success` row exists with `actor=jdoe`, `detail` containing `{"provider":"ad","roles":[…]}` and a non-NULL `client`. (Closes `W25:S1.AC-AD` / matrix `W25:F1` — record evidence there, not a second time) | -| AUTH-09 | Pin the exact kwargs handed to `spnego.server` at **both** acceptor sites | Negative/Security | pytest | dev-PC | n/a | T | P0 | With `kerberos_spn="HTTP/engine.mefor.lab"`, both `kerberos_principal` (`auth/ldap.py:300`) and `kerberos_acceptor_preflight` (`:360`) call `spnego.server(service="HTTP", hostname="engine.mefor.lab")`; a call passing the whole SPN to `service=` fails the test. Test is written to fail RED against today's code until the SPN fix lands (tracked by the owner as **#275**, above the published #231 baseline — `docs/BACKLOG.md:177` and `:4172` both gate the #99(e) residual behind it) | +| AUTH-09 | Pin the exact kwargs handed to `spnego.server` at **both** acceptor sites | Negative/Security | pytest | dev-PC | n/a | T | P0 | With `kerberos_spn="HTTP/engine.mefor.lab"`, both `kerberos_principal` (`auth/ldap.py:300`) and `kerberos_acceptor_preflight` (`:360`) call `spnego.server(service="HTTP", hostname="engine.mefor.lab")`; a call passing the whole SPN to `service=` fails the test. Test is written to fail RED against today's code until the SPN fix lands (tracked by the owner as **#275**, above the published #231 baseline — BACKLOG #99 gates its (e) residual behind it) | | AUTH-10 | #275 confirmation on a real acceptor (handoff cell **L1**) | Functional | manual | AD-lab | n/a | C | P0 | The acceptor principal is captured verbatim. Defect **confirmed** if it reads `HTTP/engine.mefor.lab/` (or `/unspecified`); **refuted** if it reads `HTTP/engine.mefor.lab@MEFOR.LAB`. Verdict recorded verbatim in the run record; L6 and the #99(e) cells stay blocked until it reports | | AUTH-11 | `POST /auth/negotiate` single-leg SPNEGO against a real KDC | Functional | manual | AD-lab | SQLite | T | P0 | A ticket for `HTTP/engine.mefor.lab` obtained on a domain-joined client is accepted; the response carries a session token; `auth.login_success` records `actor=` with `provider=ad`; **no** `WWW-Authenticate` continuation is sent (single-leg is a hard line) | | AUTH-12 | Browser Kerberos SSO `GET /ui/sso` from a domain-joined client, Chrome **and** Firefox | Functional | browser | AD-lab | SQLite | T | P0 | First navigation returns 401 with `WWW-Authenticate: Negotiate`; the browser re-issues with `Authorization: Negotiate …`; the response is a 303 to `/ui` with exactly one session cookie set; the session has **no** step-up window (`seed_reauth=False`) so the first sensitive action bounces to `/ui/reauth`. **Both** browsers must pass — a failure in either is a FAIL for that browser, recorded per browser and never averaged into one verdict | @@ -502,7 +502,7 @@ This area is signed off for release when **all** of the following hold and are r ### 9.9 Open questions -1. **Read access to the withheld `docs/security/` tree.** `AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18), `KERBEROS-EPA-SPIKE-RUNBOOK.md`, `THREAT-MODEL.md` and the ASVS assessment all **exist**; the whole directory is deliberately **withheld from the public repo** (`.gitignore:144`, ~32 files of posture / assessment / risk-register / runbook detail held back as an attacker roadmap; `docs/BACKLOG.md:271` records it as "gitignored post-cutover"). That is why `docs/SECURITY.md:1556` and `HANDOFF-AD-LAB-aws.md:7-9` link to paths that do not resolve from this tree — a publishing boundary, not a gap, so this chapter cites them as sound evidence. The question is operational: will the chapter author get read access, so the AD-lab rows can cite cell numbers as authoritative instead of restating acceptance criteria inline? **Blocks:** nothing structural — only whether AUTH-10/16/17/18/19/20/21 keep their inline restatements (they currently restate, which is safe either way). +1. **Read access to the withheld `docs/security/` tree.** `AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18), `KERBEROS-EPA-SPIKE-RUNBOOK.md`, `THREAT-MODEL.md` and the ASVS assessment all **exist**; the whole directory is deliberately **withheld from the public repo** (`.gitignore:144`, ~32 files of posture / assessment / risk-register / runbook detail held back as an attacker roadmap; BACKLOG #185 records it as "gitignored post-cutover"). That is why `docs/SECURITY.md:1556` and `HANDOFF-AD-LAB-aws.md:7-9` link to paths that do not resolve from this tree — a publishing boundary, not a gap, so this chapter cites them as sound evidence. The question is operational: will the chapter author get read access, so the AD-lab rows can cite cell numbers as authoritative instead of restating acceptance criteria inline? **Blocks:** nothing structural — only whether AUTH-10/16/17/18/19/20/21 keep their inline restatements (they currently restate, which is safe either way). 2. **Where does the chapter read the fuller backlog ledger?** The committed `docs/BACKLOG.md` is a **published baseline that stops at #231** (`:269`), and `:6041` states the rule outright: an item's absence from it is "a publishing boundary, not evidence of completion". The programme continued past it, so **#274 and #275 are valid, owner-confirmed items** — they surface here only as cross-references (`:177`, `:4170`, `:4172`) because their own entries sit above the baseline. This chapter therefore cites them as sound evidence, annotating "above the published #231 baseline" only where a reader would otherwise go looking for an entry. The open question is access alone: may the author read the fuller ledger to quote #275's own acceptance criteria? **Blocks:** nothing — AUTH-09 and AUTH-10 stand on the code evidence at `auth/ldap.py:300` / `:360` regardless. 3. **May the #275 SPN fix land BEFORE the lab confirms it?** The handoff instructs that L1 runs first and hard-blocks L6 and the #99(e) cells. AUTH-09 is written to fail RED against today's code. **Blocks:** whether AUTH-09 lands now as a failing guard, lands after AUTH-10, or is written to pin today's behaviour and inverted later. 4. **May ADR 0142 / OIDC be described as shipped while its status reads "Proposed — code COMPLETE, awaiting lab validation"?** This chapter labels it *built but unvalidated* throughout. **Blocks:** the wording of AUTH-17..25 pass criteria and whether federated SSO appears in release notes as a feature. diff --git a/docs/testing/master-test-plan/13-steps-editor.md b/docs/testing/master-test-plan/13-steps-editor.md index ad4025d1..2e52b2d5 100644 --- a/docs/testing/master-test-plan/13-steps-editor.md +++ b/docs/testing/master-test-plan/13-steps-editor.md @@ -9,7 +9,7 @@ ### 12.1 Scope & objectives -**In scope.** The BACKLOG #26 carve-out (#222, SHIPPED — `docs/BACKLOG.md:6689`): a structured Steps view over real Python Handlers via a typed action vocabulary, where **plain `.py` stays the only artifact and the only execution path**. Concretely: +**In scope.** The BACKLOG #26 carve-out (#222, SHIPPED): a structured Steps view over real Python Handlers via a typed action vocabulary, where **plain `.py` stays the only artifact and the only execution path**. Concretely: - The **projection contract** — `messagefoundry lens parse [--json]` (`messagefoundry/__main__.py:374-390`, `:2770`) → `messagefoundry/lens.py:270 parse_module` / `:285 parse_source`, and its **coverage-partition invariant** (`lens.py:11-21`). - The **rewrite contract** — `messagefoundry lens rewrite` (`__main__.py:392-404`, `:2793`) → `lens.py:1406 rewrite_module` / `:1423 rewrite_source`, its 11 ops (`lens.py:1382-1396 _SUPPORTED_OPS`), the row-scoped byte-space splice, the F7 stale-coordinate guard (`lens.py:1533 _check_expect_src`), the re-parse gate (`:1582 _assert_reparses`) and the 100-column refusal (`:1403 _MAX_LINE_LENGTH`). diff --git a/docs/testing/master-test-plan/15-alerting-and-observability.md b/docs/testing/master-test-plan/15-alerting-and-observability.md index 608c7607..76f21c5c 100644 --- a/docs/testing/master-test-plan/15-alerting-and-observability.md +++ b/docs/testing/master-test-plan/15-alerting-and-observability.md @@ -14,13 +14,13 @@ off-box, and nothing in CI notices. **In scope — Alerting.** The ADR 0014 rules engine (`AlertRuleSet.decide` matcher, first-match-wins, per-rule severity/transport-subset/cooldown/suppress, `mute`, `recipients`, `id`, `control_action`/ `control_target`, `escalate` tiers, `schedule`, `content_label`) and its five amendments; the -`AlertSink` Protocol ([`pipeline/alerts.py:27`](messagefoundry/pipeline/alerts.py:27) — **20** public +`AlertSink` Protocol ([`pipeline/alerts.py:27`](../../../messagefoundry/pipeline/alerts.py) — **20** public methods) + its `LoggingAlertSink` fallback (same file, `:231`, the same 20) and the **separate** -`NotifierAlertSink` ([`pipeline/alert_sinks.py:578`](messagefoundry/pipeline/alert_sinks.py:578) — a +`NotifierAlertSink` ([`pipeline/alert_sinks.py:578`](../../../messagefoundry/pipeline/alert_sinks.py) — a different class: 35 defs, 27 public = those 20 emit methods + `content_match` + six sink-lifecycle methods; do not conflate its count with the Protocol's); the **18**-member operator-routable set `_ALERT_EVENT_TYPES` -([`config/settings.py:2499-2526`](messagefoundry/config/settings.py:2499) — `lane_stuck` (`:2515`) +([`config/settings.py:2499-2526`](../../../messagefoundry/config/settings.py) — `lane_stuck` (`:2515`) and `rcsi_off_degraded` (`:2516`) **are** members, see 14.2); the webhook and SMTP transports and their failure behaviour; ADR 0127 email templates; ADR 0044 durable `alert_instance` ack/mute(suspend)/resume/resolve lifecycle and auto-resolve inverses; ADR 0128 @@ -60,7 +60,7 @@ has its own `FCP:ALERT-1..FCP:ALERT-24` (different subjects from ours) and the W |---|---| | The 24-row subsystem coverage-gap audit `FCP:ALERT-1..FCP:ALERT-24` (six dimensions, per-feature verdicts) | [`docs/testing/FEATURE-COVERAGE-PLAN.md` §19, lines 1300-1341](../FEATURE-COVERAGE-PLAN.md) — this chapter **supersedes four stale rows** (see 14.2) and otherwise inherits it | | The PHI posture of every log/alert stream (14 rows: format, sink, ACL, retention, PHI class) | [`docs/PHI.md` §7 "Logging inventory (16.1.1 / 16.2.3)", rows 1-14](../../PHI.md) — CI-guarded by `tests/test_phi_logging_inventory.py` | -| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](harness/acceptance/matrix.py:454)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | +| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](../../../harness/acceptance/matrix.py)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | | On-box deployment acceptance (`messagefoundry verify` host/store/smoke/manual/federation sections) | [`docs/testing/VERIFY.md`](../VERIFY.md) — it has no alerting or logging section; nothing here duplicates it | | Load generation mechanics, profiles, governor, SLO verdicts | [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) — this chapter only *consumes* the `connscale` / `sustained-overload` profiles | | Cluster leadership/lease/failover mechanics themselves | the HA/DR chapter — here only the **alerting behaviour across** a leadership move | @@ -120,7 +120,7 @@ tree; carry the correction here, leave that document as a historical artifact): | Stale row | Re-verified status | |---|---| | `FCP:ALERT-4` (line 483: webhook http/SSRF/3xx refusal "partial", P3) | **CLOSED** by `tests/test_asvs_phase0.py:158-190` | -| `FCP:ALERT-12` (lines 431/1319/1341: `lane_stuck` + `rcsi_off_degraded` "absent from `_ALERT_EVENT_TYPES`", P0) | **CLOSED** — both are members at [`settings.py:2515-2516`](messagefoundry/config/settings.py:2515) with rule-targetable tests at `test_alert_rules.py:357/370/395`. The auditor note at line 11 and its `settings.py:1755` line reference are stale (the constant is now at `:2499`) | +| `FCP:ALERT-12` (lines 431/1319/1341: `lane_stuck` + `rcsi_off_degraded` "absent from `_ALERT_EVENT_TYPES`", P0) | **CLOSED** — both are members at [`settings.py:2515-2516`](../../../messagefoundry/config/settings.py) with rule-targetable tests at `test_alert_rules.py:357/370/395`. The auditor note at line 11 and its `settings.py:1755` line reference are stale (the constant is now at `:2499`) | | `FCP:ALERT-19` (line 438: three-backend parity "structural only", P1) | **CLOSED** by live execution on both server backends | | `FCP:ALERT-24` (line 1331) | Cites `tests/test_console_alerts.py`, which **no longer exists**; the PySide6 console is retired. The live surface is `/ui/alerts`, covered by the webconsole suite | @@ -132,27 +132,27 @@ ALERT-13 and ALERT-11. | Risk | Failure mode | Blast radius | Detected today? | Priority | |---|---|---|---|---| -| **G1 — three alert vocabularies bound only by convention** | An emit site's `type` literal, `_ALERT_EVENT_TYPES`, and the `AlertSink` Protocol / `LoggingAlertSink` / `NotifierAlertSink` method sets drift. A type absent from the routable set can never be escalated, routed, suppressed or muted by any operator rule; a method present only on `NotifierAlertSink` raises `AttributeError` on the logging fallback | Every deployment. Drift has fired **twice** already: `lane_stuck` + `rcsi_off_degraded` once shipped unroutable — **both are members today** ([`settings.py:2515-2516`](messagefoundry/config/settings.py:2515)), rule-targetable end to end at `test_alert_rules.py:357/370/395`, so the only live drift is that no guard stops the next one; `content_match` is **still** divergent (only on `NotifierAlertSink` at [`alert_sinks.py:676`](messagefoundry/pipeline/alert_sinks.py:676), absent from the Protocol and the fallback). Note the non-obvious third form: `AlertSink.saturation_rising()` emits type `"saturation"` — method name ≠ event type, so a naive guard would false-fail | **No.** No guard exists in either direction | **P0** | +| **G1 — three alert vocabularies bound only by convention** | An emit site's `type` literal, `_ALERT_EVENT_TYPES`, and the `AlertSink` Protocol / `LoggingAlertSink` / `NotifierAlertSink` method sets drift. A type absent from the routable set can never be escalated, routed, suppressed or muted by any operator rule; a method present only on `NotifierAlertSink` raises `AttributeError` on the logging fallback | Every deployment. Drift has fired **twice** already: `lane_stuck` + `rcsi_off_degraded` once shipped unroutable — **both are members today** ([`settings.py:2515-2516`](../../../messagefoundry/config/settings.py)), rule-targetable end to end at `test_alert_rules.py:357/370/395`, so the only live drift is that no guard stops the next one; `content_match` is **still** divergent (only on `NotifierAlertSink` at [`alert_sinks.py:676`](../../../messagefoundry/pipeline/alert_sinks.py), absent from the Protocol and the fallback). Note the non-obvious third form: `AlertSink.saturation_rising()` emits type `"saturation"` — method name ≠ event type, so a naive guard would false-fail | **No.** No guard exists in either direction | **P0** | | **G2 — `content_match` has no Handler-reachable surface** | ADR 0133 AC-3 says "WHEN a Handler emits a `content_match`" — but there is no export in `messagefoundry/__init__.py` (unlike `db_lookup`/`fhir_lookup` at lines 32-33/150/153), no injected sink on a Handler context, and no dry-run path. Every test calls `sink.content_match(...)` directly | The differentiating Corepoint "Action Point" parity capability is unusable in practice; the PHI-free-by-contract guarantee has never been exercised through real Handler code | **No** — AC-3/AC-4 pass at the sink so nothing fails | **P0** | -| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](messagefoundry/pipeline/alert_sinks.py:100)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](../../../messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | -| **G4 — the alert/security-notify SMTP hop is encrypted but unauthenticated** | `send_plain_email` calls `smtp.starttls()` with **no** `SSLContext` ([`alert_sinks.py:384`](messagefoundry/pipeline/alert_sinks.py:384)), so the stdlib default applies (`check_hostname=False`, `verify_mode=CERT_NONE`). The same transport carries the ASVS 6.3.5/6.3.7 per-user security notifications | A MITM on the SMTP hop reads every alert body (connection names, queue shapes, `safe_exc` detail strings) and every account-security notification (username, source IP, failed-attempt count) — and can suppress them | **No, and the docs disagree with themselves.** `docs/PHI.md` row 11 states the caveat plainly; `docs/BACKLOG.md:5152` claims the sink "uses STARTTLS with a verifying context by design". No test pins either | **P0** | -| **G5 — `[alerts]` is startup-only** | `app.state.alerts_settings` is assigned only at app construction ([`api/app.py:1120`](messagefoundry/api/app.py:1120)) and lifespan startup (`:5485`). `POST /config/reload` (`:2741`) re-runs the `--config` graph, never the service-settings TOML | An operator adds a suppression rule mid-incident via the IDE or `messagefoundry alert add`; the IDE re-lists from the **file** and shows it; `/alerts/rules` still shows the **startup** set; the running notifier keeps paging until a restart. The requirement is documented only in `alerts_edit.py:19-21` and the CLI docstring — nowhere an operator looks | **No** | **P0** | -| **G6 — IDE alert editor offers 4 of the 18 event types and 7 of the 15 fields** | [`ide/src/alertEditor.ts:13-19`](ide/src/alertEditor.ts:13) offers a 5-entry dropdown — `any` plus only `connection_stopped`/`queue_buildup`/`storage_threshold`/`cert_expiry`; `:25-32` supports only `event_type`/`connection`/`min_depth`/`min_oldest_seconds`/`severity`/`transports`/`cooldown_seconds` (7 of the 15 `AlertRule` fields). `ide/src/test/suite/` has **no** alert test file (35 suites, none for the alert editor) | An operator on the supported GUI authoring path cannot rule on **14 of the 18** signals, nor set `id`, `recipients`, `mute`, `escalate`, `schedule`, `content_label`, `control_action`, `control_target` | **No** — the `ide` CI leg runs and tests nothing here | **P1** | +| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](../../../messagefoundry/pipeline/alert_sinks.py)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](../../../messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | +| **G4 — the alert/security-notify SMTP hop is encrypted but unauthenticated** | `send_plain_email` calls `smtp.starttls()` with **no** `SSLContext` ([`alert_sinks.py:384`](../../../messagefoundry/pipeline/alert_sinks.py)), so the stdlib default applies (`check_hostname=False`, `verify_mode=CERT_NONE`). The same transport carries the ASVS 6.3.5/6.3.7 per-user security notifications | A MITM on the SMTP hop reads every alert body (connection names, queue shapes, `safe_exc` detail strings) and every account-security notification (username, source IP, failed-attempt count) — and can suppress them | **No, and the docs disagree with themselves.** `docs/PHI.md` row 11 states the caveat plainly; `docs/BACKLOG.md:5152` claims the sink "uses STARTTLS with a verifying context by design". No test pins either | **P0** | +| **G5 — `[alerts]` is startup-only** | `app.state.alerts_settings` is assigned only at app construction ([`api/app.py:1120`](../../../messagefoundry/api/app.py)) and lifespan startup (`:5485`). `POST /config/reload` (`:2741`) re-runs the `--config` graph, never the service-settings TOML | An operator adds a suppression rule mid-incident via the IDE or `messagefoundry alert add`; the IDE re-lists from the **file** and shows it; `/alerts/rules` still shows the **startup** set; the running notifier keeps paging until a restart. The requirement is documented only in `alerts_edit.py:19-21` and the CLI docstring — nowhere an operator looks | **No** | **P0** | +| **G6 — IDE alert editor offers 4 of the 18 event types and 7 of the 15 fields** | [`ide/src/alertEditor.ts:13-19`](../../../ide/src/alertEditor.ts) offers a 5-entry dropdown — `any` plus only `connection_stopped`/`queue_buildup`/`storage_threshold`/`cert_expiry`; `:25-32` supports only `event_type`/`connection`/`min_depth`/`min_oldest_seconds`/`severity`/`transports`/`cooldown_seconds` (7 of the 15 `AlertRule` fields). `ide/src/test/suite/` has **no** alert test file (35 suites, none for the alert editor) | An operator on the supported GUI authoring path cannot rule on **14 of the 18** signals, nor set `id`, `recipients`, `mute`, `escalate`, `schedule`, `content_label`, `control_action`, `control_target` | **No** — the `ide` CI leg runs and tests nothing here | **P1** | | **G7 — no end-to-end drive of the runner's buildup / stall / saturation emit sites** | `_maybe_alert_buildup` (`:5401`), `_maybe_alert_saturation` (`:5444`), `_maybe_alert_stall` (`:5511`) in `wiring_runner.py`. Only the pure `SaturationDetector` and the **engine-shard** non-owned-lane watchdog are tested | These are the three alerts an operator relies on to notice a stalled or drowning feed. Threshold resolution, the `_outbound_paused` suppression guard, the per-`(stage,lane)` `_BUILDUP_REALERT_SECONDS` throttle and the `pending_depth` read could all break silently | **No** — FEATURE-COVERAGE-PLAN `FCP:ALERT-10` flagged it and it is still open | **P1** | -| **G8 — the alert-storm bound is untested** | `_MAX_QUEUE = 1000` with drop-with-warning ([`alert_sinks.py:126-131`](messagefoundry/pipeline/alert_sinks.py:126)). With a wedged webhook (a hung POST inside the 10 s timeout) and a large estate, a burst silently exceeds the bound and the excess is dropped with only a `WARNING` — there is **no dropped-alert counter or metric** | The operator sees neither the alerts nor a countable drop signal. A regression that lowers the bound, blocks the drain loop, or turns the drop into a *stall* on the emitting delivery worker is invisible | **No** — `tests/test_communications_inventory.py:372` pins `_MAX_QUEUE` as documentation only | **P1** | +| **G8 — the alert-storm bound is untested** | `_MAX_QUEUE = 1000` with drop-with-warning ([`alert_sinks.py:126-131`](../../../messagefoundry/pipeline/alert_sinks.py)). With a wedged webhook (a hung POST inside the 10 s timeout) and a large estate, a burst silently exceeds the bound and the excess is dropped with only a `WARNING` — there is **no dropped-alert counter or metric** | The operator sees neither the alerts nor a countable drop signal. A regression that lowers the bound, blocks the drain loop, or turns the drop into a *stall* on the emitting delivery worker is invisible | **No** — `tests/test_communications_inventory.py:372` pins `_MAX_QUEUE` as documentation only | **P1** | | **G9 — post-failover / post-restart re-page volume is unbounded and untested** | `_last_sent` (throttle) and `_occurrences` (escalation counter) are per-node, in-memory and **not** primed from the store — only `_suspended` is (`prime_suspensions`, `:873`) | On an HA leadership move, a DR promotion, or an ordinary restart, the new node re-pages **every** open condition at once and every escalation tier resets to base, so a long-running critical drops back to warning. ADR 0014 §4 (lines 91-92, 117) records this as accepted — but nothing bounds it and nothing tests it | **No** | **P1** | | **G10 — no boundary assertion that every emit site's `detail`/`reason` is PHI-scrubbed** | `docs/PHI.md` row 10: the webhook carries this free text "`safe_exc()`-scrubbed at the emit sites, but **not** re-run through `safe_text` on this path". The scrub is a convention across ~20 scattered call sites | One new emit site passing `str(exc)` instead of `safe_exc(exc)` sends an HL7 fragment straight off-box to a third-party webhook (Slack/Teams/PagerDuty) — no store, no ACL, no audit. **Silent**: the `alert_instance.reason` IS `safe_text`'d at the store, so an at-rest test passes while the wire payload leaks | **No** | **P1** | | **G11 — outbound saturation blind spot** | `_maybe_alert_saturation` is reached only from `_maybe_alert_buildup`, which runs on the delivery-failure/retry tick — a **healthy-but-behind** outbound (delivering successfully while its backlog climbs) is never sampled. ADR 0014 amendment records it as the "BACKLOG #93 residual" | The single most common real overload — a downstream that is slow but not failing — produces no saturation page, while an operator who opted in believes they are covered | **No** — the known non-coverage is documented but not pinned, so a refactor could widen or narrow it unnoticed | **P1** | | **G12 — four periodic alert runners are not leader-gated** | `cert_expiry.py`, `secret_rotation.py`, `update_check.py`, `gcm_invocations.py` have no `is_leader()` call; only `retention.py:357/:380` does. `gcm_invocations.py:28-29` documents its non-gating *for block refill*, which is a different question from the alert | In an N-node active-passive cluster each node independently observes the **same cluster-wide facts** (a cert file, the DEK rotation age, the installed version, the shared DEK invocation count) and pages every cooldown — N duplicate pages per condition, plus N `alert_instance` upserts contending on `ux_alert_instance_open`. ADR 0014 §4's per-node reasoning covers *lane* events, not these | **No** | **P1** | | **G13 — `support/redact.py` has no adversarial corpus test** | `tests/test_support_bundle.py:93-234` asserts ~7 shapes. The module is a small `re` pass plus the shared `messagefoundry.redaction.redact`; its stated residual is the single-token identifier | The support bundle is the **one** stream designed to leave the box, and `docs/PHI.md` row 14 records it carries **no RBAC, no audit row, no retention** — the operator emails it to a third party. A missed residual is an unlogged, uncontrolled PHI disclosure | **No** — the residual is stated but never measured | **P1** | | **G14 — no cross-surface alert-rule parity check** | Three views of one rule set: `/alerts/rules` (startup `app.state`), the IDE editor list (the TOML **file**), and `/ui/alerts` (rendering `/alerts/rules`). During the restart window they disagree **by construction** (G5) | An operator confirming "the suppression rule is in place" in the IDE while the engine still pages is a real incident-time failure. Nothing detects or signals the divergence | **No** | **P1** | -| **G15 — an alert-triggered `control_action` writes no audit row** | `_dispatch_control` ([`alert_sinks.py:1039`](messagefoundry/pipeline/alert_sinks.py:1039)) fires the injected callback and only logs on failure; `_alert_control` at [`api/app.py:5469`](messagefoundry/api/app.py:5469) has no `record_audit` | A rule can restart a production inbound or outbound automatically with **no** entry in the tamper-evident audit chain. Every other connection-control path in the API is audited | **No** | **P2** | +| **G15 — an alert-triggered `control_action` writes no audit row** | `_dispatch_control` ([`alert_sinks.py:1039`](../../../messagefoundry/pipeline/alert_sinks.py)) fires the injected callback and only logs on failure; `_alert_control` at [`api/app.py:5469`](../../../messagefoundry/api/app.py) has no `record_audit` | A rule can restart a production inbound or outbound automatically with **no** entry in the tamper-evident audit chain. Every other connection-control path in the API is audited | **No** | **P2** | | **G16 — never-resolved `alert_instance` rows grow without bound** | Retention purges **RESOLVED** rows only (`docs/PHI.md` row 8: "an open or acknowledged condition is never aged out from under an operator"). With G3, open `connection_stopped` rows accumulate permanently | The `ux_alert_instance_open` partial unique index and every `/alerts/active` read carry them forever; the nav bell caps at 200 so the count silently understates | **No** — no growth or soak test | **P2** | -| **G17 — OpenTelemetry export is smoke-only** | `tests/test_metrics_exporter.py:345` records against an in-process provider. No OTLP collector round-trip; no CI leg exercises the `[otel]` extra present **or** absent (the guarded-import `RuntimeError` at [`api/metrics.py:463-465`](messagefoundry/api/metrics.py:463)) | `[otel]` is an advertised capability (`docs/FEATURE-MAP.md:159`). A broken export or a wrong resource/attribute set regresses silently while the default Prometheus path keeps passing | **No** | **P2** | +| **G17 — OpenTelemetry export is smoke-only** | `tests/test_metrics_exporter.py:345` records against an in-process provider. No OTLP collector round-trip; no CI leg exercises the `[otel]` extra present **or** absent (the guarded-import `RuntimeError` at [`api/metrics.py:463-465`](../../../messagefoundry/api/metrics.py)) | `[otel]` is an advertised capability (`docs/FEATURE-MAP.md:159`). A broken export or a wrong resource/attribute set regresses silently while the default Prometheus path keeps passing | **No** | **P2** | | **G18 — NSSM log rotation is never asserted** | `scripts/service/install-service.ps1:456-458` sets `AppRotateFiles 1` / `AppRotateOnline 1` / `AppRotateBytes 10485760`; the `windows-service-smoke` leg only captures and uploads the logs (`ci.yml:1239-1259`) | The app log is the **sole** sink for PHI.md stream 1 (the engine installs no file handler) and the source of `/logs/tail` and the support bundle. If rotation stops, the volume fills or the tail becomes unusable — precisely during an incident | **No** — `[retention].app_log_days` deletion and `app_log_compress_days` gzip are unit-tested but never on a real Windows service | **P2** | | **G19 — `/metrics` has no cardinality or scrape-cost bound** | `gather_snapshot` does per-scrape store reads; exporter tests use a handful of connections | A 200+ connection estate on a 15 s Prometheus scrape adds measurable store load, and an unbounded label set (connection × destination × status) can blow up a TSDB | **No** | **P2** | | **G20 — off-box log forwarding has no live-collector round-trip** | `tests/test_logging.py:371-660` covers handler construction, TLS context, CA anchoring, client cert and unreachable-collector tolerance — nothing asserts a record **arrives**, correctly framed, at a collector | This is the SIEM evidence path (`docs/PHI.md` rows 3/4). An RFC 5425 length-prefix or framing regression means audit rows silently never reach the SIEM while the engine reports nothing wrong | **No** | **P2** | -| **G21 — the notifier is constructed only on the `serve` path** | `notifier_from_settings` is called once, in the `create_serve_app` lifespan ([`api/app.py:5316-5318`](messagefoundry/api/app.py:5316)). Any other engine path (embedded `Engine`, `messagefoundry check`, dryrun, most tests) silently runs on `LoggingAlertSink` | A future deployment mode or a lifespan reorder would leave a production engine with no webhook/email notifier and only `WARNING` log lines | **No** — nothing asserts the runner's sink identity after a `serve` lifespan | **P2** | +| **G21 — the notifier is constructed only on the `serve` path** | `notifier_from_settings` is called once, in the `create_serve_app` lifespan ([`api/app.py:5316-5318`](../../../messagefoundry/api/app.py)). Any other engine path (embedded `Engine`, `messagefoundry check`, dryrun, most tests) silently runs on `LoggingAlertSink` | A future deployment mode or a lifespan reorder would leave a production engine with no webhook/email notifier and only `WARNING` log lines | **No** — nothing asserts the runner's sink identity after a `serve` lifespan | **P2** | | **G22 — no bounded-latency assertion that a new alert reaches an operator surface** | The nav bell + `/ui/alerts` are poll-driven (`/ui/nav-status` ~every 15 s, [`webconsole/routes/status.py:128`](../../../messagefoundry_webconsole/routes/status.py)); `/ws/stats` pushes queue counts, **not** alerts | An operator watching the console can be a poll interval behind a stopped connection with no indication the view is stale | **No** — the webconsole tests assert rendering and RBAC, not freshness | **P2** | | **G23 — three ledger/catalog documents lie about this area** | `docs/FEATURE-MAP.md` §9 lists 8 rows and omits ADR 0044 alert state, escalation, templates, per-rule recipients, control actions, **16 of the 18** event types (§9 names only `connection_stopped` and `queue_buildup`), the support bundle, crashdump suppression, the `connection_event` log, `/logs/tail`, `/metrics/history` and host metrics; §10 is still titled "Surfaces — Admin Console (PySide6)" with an Alerts page row at `:172`, and `:131` asserts "The PySide6 desktop console stays (additive)" though `messagefoundry/console/` does not exist. `docs/BACKLOG.md` #171 (lines 5768-5780) still banners DEMAND-GATE and states "there is no runtime/per-area verbosity control and no interactive in-console log viewer" — both are BUILT | FEATURE-MAP reaches the **public mirror** (`tests/test_feature_map_claims.py:3`). It understates the shipped alerting surface and overstates a retired one; the BACKLOG lies about build state — exactly what `backlog-hygiene.yml` exists to prevent | **No** — the existing guard tests links and ASVS score claims, not row currency | **P2** | @@ -172,9 +172,9 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ID | Test | Type | Method | Env | Backend | Cls | Pri | Pass criteria | |---|---|---|---|---|---|---|---|---| -| ALERT-01 | Alert-vocabulary mirror invariant: every `type` literal emitted by a `NotifierAlertSink` method is in `_ALERT_EVENT_TYPES` or in `_AUTO_RESOLVE` | Functional | pytest | dev-PC | n/a | T | P0 | AST-walk `alert_sinks.py`; collect every `"type": ` in a `self._emit({...})` / `self._record_state({...})` call. Each literal is in `_ALERT_EVENT_TYPES ∪ set(_AUTO_RESOLVE)`. The set of emitted-and-routable literals equals `_ALERT_EVENT_TYPES` exactly (currently **18** members, [`config/settings.py:2499-2526`](messagefoundry/config/settings.py:2499)). `saturation_rising` → `"saturation"` is asserted by name so the method-name≠type case is pinned, not a false failure. A planted extra emit type fails the test | +| ALERT-01 | Alert-vocabulary mirror invariant: every `type` literal emitted by a `NotifierAlertSink` method is in `_ALERT_EVENT_TYPES` or in `_AUTO_RESOLVE` | Functional | pytest | dev-PC | n/a | T | P0 | AST-walk `alert_sinks.py`; collect every `"type": ` in a `self._emit({...})` / `self._record_state({...})` call. Each literal is in `_ALERT_EVENT_TYPES ∪ set(_AUTO_RESOLVE)`. The set of emitted-and-routable literals equals `_ALERT_EVENT_TYPES` exactly (currently **18** members, [`config/settings.py:2499-2526`](../../../messagefoundry/config/settings.py)). `saturation_rising` → `"saturation"` is asserted by name so the method-name≠type case is pinned, not a false failure. A planted extra emit type fails the test | | ALERT-02 | Alert-vocabulary mirror invariant: the three classes expose identical public method sets | Functional | pytest | dev-PC | n/a | T | P0 | `{m for m in dir(AlertSink) if not m.startswith("_")}` == the same for `LoggingAlertSink` == the emit-method subset of `NotifierAlertSink` (excluding the sink-lifecycle methods `start`/`aclose`/`set_store`/`set_control_callback`/`suspend`/`resume`/`forget`/`prime_suspensions`, named explicitly as an allowlist). Test **currently FAILS** on `content_match` — record the failure as the acceptance evidence for the G2 decision, then make it pass by whichever way OQ-1 resolves | -| ALERT-03 | `LoggingAlertSink` fallback survives every emit the runner can make | Negative/Security | pytest | dev-PC | n/a | T | P0 | For each of the **20** methods on the `AlertSink` Protocol ([`pipeline/alerts.py:27`](messagefoundry/pipeline/alerts.py:27) — the Protocol lives in `alerts.py`, **not** `alert_sinks.py`; the 35-def `NotifierAlertSink` at [`alert_sinks.py:578`](messagefoundry/pipeline/alert_sinks.py:578) is a different class and its count is not the Protocol's), call it on `LoggingAlertSink` ([`alerts.py:231`](messagefoundry/pipeline/alerts.py:231), the same 20) with representative PHI-free args; no `AttributeError`, no exception, and exactly one log record at the documented level (`WARNING`, except `leadership_lost`/`dr_released` at `INFO` and `connection_restored` a no-op). Calling `content_match` on the fallback raises `AttributeError` today — assert the *current* behaviour explicitly so the divergence is a pinned fact | +| ALERT-03 | `LoggingAlertSink` fallback survives every emit the runner can make | Negative/Security | pytest | dev-PC | n/a | T | P0 | For each of the **20** methods on the `AlertSink` Protocol ([`pipeline/alerts.py:27`](../../../messagefoundry/pipeline/alerts.py) — the Protocol lives in `alerts.py`, **not** `alert_sinks.py`; the 35-def `NotifierAlertSink` at [`alert_sinks.py:578`](../../../messagefoundry/pipeline/alert_sinks.py) is a different class and its count is not the Protocol's), call it on `LoggingAlertSink` ([`alerts.py:231`](../../../messagefoundry/pipeline/alerts.py), the same 20) with representative PHI-free args; no `AttributeError`, no exception, and exactly one log record at the documented level (`WARNING`, except `leadership_lost`/`dr_released` at `INFO` and `connection_restored` a no-op). Calling `content_match` on the fallback raises `AttributeError` today — assert the *current* behaviour explicitly so the divergence is a pinned fact | | ALERT-04 | `content_match` reachability from a real Handler | Functional | pytest | dev-PC | SQLite | T | P0 | Author a sample Handler under a temp config dir that attempts to raise a content alert using only the public `messagefoundry` surface. Assert either (a) a working export (`messagefoundry.alert_content` or an injected sink) produces exactly one `content_match` event whose payload keys are `{type, connection, label}` (+ optional `rule_id`) and **no** other key, **or** (b) an explicit, documented refusal. Assert `"content_match"` in `messagefoundry.__all__` iff (a) | | ALERT-05 | `content_match` unavailable in dryrun and on a Router | Negative/Security | pytest | dev-PC | n/a | T | P0 | If OQ-1 resolves to a Handler export: calling it from a Router or under `messagefoundry dryrun` raises, matching the `db_lookup`/`fhir_lookup` carve-out posture (CLAUDE.md §2). No event is enqueued in either case | | ALERT-06 | A restarted stopped lane clears its open `connection_stopped` instance | HA/Resilience | pytest | dev-PC | SQLite | T | P0 | Drive an outbound to `connection_stopped` through `RegistryRunner`; assert one `alert_instance` row `status='open'`. Call `RegistryRunner.restart_outbound(name)`; assert the instance reaches `status='resolved'` **and** `count_open_alerts_by_connection` drops that connection to 0 within one poll. Fails today (nothing emits `connection_started`) — see 14.5 §S1 | @@ -200,7 +200,7 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ALERT-26 | `_suspended` survives a restart but `_last_sent`/`_occurrences` do not — pinned | HA/Resilience | pytest | dev-PC | SQLite | T | P1 | Suspend `(type, connection)` until `now+3600`, restart the sink, call `prime_suspensions()`; assert the next emit is muted. In the same test assert `_last_sent` and `_occurrences` are empty after the restart (the accepted ADR 0014 §4 asymmetry is a pinned fact, not an accident) | | ALERT-27 | Cluster duplicate-page behaviour for the four ungated periodic runners | HA/Resilience | pytest | dev-PC | n/a | T | P1 | Under a fixture with three coordinators where only one is leader, run one pass each of `CertExpiryRunner`, `SecretRotationRunner`, `UpdateCheckRunner`, `GcmInvocationRunner`. Assert the observed emit count per condition (3 today) against the decided expectation (OQ-5). `RetentionRunner` in the same fixture emits 1 (leader-gated) — the contrast is the point | | ALERT-28 | Duplicate `alert_instance` upserts under concurrent nodes do not violate the open-unique index | Cross-backend | pytest | container-CI | x2 | T | P1 | Three concurrent `upsert_alert_instance` calls for the same `(event_type, connection)` against a live Postgres and a live SQL Server produce exactly one `open` row with `count == 3`; no `IntegrityError` escapes; `ux_alert_instance_open` holds | -| ALERT-29 | IDE alert editor event-type parity | Compat | ide-mocha | container-CI | n/a | T | P1 | `EVENT_TYPES` in `ide/src/alertEditor.ts` equals `["any", ...sorted(_ALERT_EVENT_TYPES)]` read from a generated constant. New file `ide/src/test/suite/alert-editor.test.ts`. Fails today: the dropdown holds **5** entries (`any` + 4 real types, [`alertEditor.ts:13-19`](ide/src/alertEditor.ts:13)) against the required **19** (`any` + the 18 `_ALERT_EVENT_TYPES` members) | +| ALERT-29 | IDE alert editor event-type parity | Compat | ide-mocha | container-CI | n/a | T | P1 | `EVENT_TYPES` in `ide/src/alertEditor.ts` equals `["any", ...sorted(_ALERT_EVENT_TYPES)]` read from a generated constant. New file `ide/src/test/suite/alert-editor.test.ts`. Fails today: the dropdown holds **5** entries (`any` + 4 real types, [`alertEditor.ts:13-19`](../../../ide/src/alertEditor.ts)) against the required **19** (`any` + the 18 `_ALERT_EVENT_TYPES` members) | | ALERT-30 | IDE alert editor field parity | Compat | ide-mocha | container-CI | n/a | T | P1 | The `NewRule` field set equals `AlertRule.model_fields` from the same generated constant, or a decided, explicitly-listed subset with the omitted fields named. Mirrors the `tests/test_alerts_edit.py:153` `_RULE_FIELDS` pattern | | ALERT-31 | The generated IDE constant is regenerated from the engine and drift fails CI | Compat | pytest | container-CI | n/a | T | P1 | An engine-side test regenerates the constant from `_ALERT_EVENT_TYPES` + `AlertRule.model_fields` and asserts the committed file is byte-identical. Adding an event type without regenerating fails the `test` leg (not only the `ide` leg) | | ALERT-32 | Cross-surface rule parity after a restart | Functional | pytest | dev-PC | SQLite | T | P1 | Add a rule via `messagefoundry alert add`, restart the serve app, then assert `messagefoundry alert list --json`, `GET /alerts/rules`, and the `/ui/alerts` rendered rule table all contain the same rule count, ids and event types | @@ -211,7 +211,7 @@ observations whose falsifiable halves are carried by T rows (ALERT-18/40/41 and | ALERT-37 | A newly-opened alert reaches the console within a bounded number of polls | Usability | pytest | dev-PC | SQLite | T | P2 | Upsert an `alert_instance`, then poll `/ui/nav-status`; the alerts `count` reflects it on the first poll after the write and the `severity` field equals the worst open severity. Assert the documented ~15 s poll cadence is what the page's script uses | | ALERT-38 | `/ui/alerts` ack / resolve / suspend / resume round-trip and CSRF posture | Functional | pytest | dev-PC | SQLite | T | P2 | Each of the four POST routes in `messagefoundry_webconsole/routes/monitoring_writes.py` mutates the instance and 303-redirects to `/ui/alerts`; a cross-site POST is refused; an operator without `monitoring:diagnose` gets no forms rendered | | ALERT-39 | `POST /alerts/test-email` exercises the identical send path a real alert uses | Functional | pytest | dev-PC | SQLite | T | P2 | With a capturing SMTP stub, the frames produced by `/alerts/test-email` and by a real fired alert differ only in subject/body content — the same `EmailTransport` → `send_plain_email` → `starttls` → `send_message` sequence, the same allowlist check, the same timeout. Guards against the test path drifting into a false-confidence stub | -| ALERT-40 | `/metrics` cardinality and scrape cost on a large estate — **the owning row** (the API chapter's `API-54` points here) | Performance | load-harness | dev-PC | SQLite | T | P2 | Derive the expected series count **from the registry**, don't guess it: `_MetricsCollector.collect` ([`api/metrics.py:246-441`](messagefoundry/api/metrics.py:246)) yields a fixed block (`build_info`, `in_pipeline`, `store_committed_txns`, `store_body_copies`, + 4 host gauges when psutil reads, + the 9-series pool block on a server backend), `outbox_status` at one series per outbox status, **2 per inbound connection** (`messages_received`, `messages_errored`), **4 per (connection, destination) pair** (`deliveries`, `deliveries_dead`, `queue_depth`, `oldest_pending_age_seconds`) and **15 per pair** for `delivery_latency_seconds` (12 `DEFAULT_LATENCY_BUCKETS` + `+Inf` + `_sum` + `_count`). On the `harness/load/profiles/connscale.toml` estate assert total series `== K + 2·I + 19·D` **exactly** — i.e. growth is linear in inbound connections and in *existing* destination pairs, never the connection×destination×status product — that the scrape stays under a committed latency ceiling at that estate size, and that the label-**name** set is exactly the strict allowlist `{connection, destination, status, version, le}` (extending `tests/test_metrics_exporter.py:135`) with no PHI-shaped label value | +| ALERT-40 | `/metrics` cardinality and scrape cost on a large estate — **the owning row** (the API chapter's `API-54` points here) | Performance | load-harness | dev-PC | SQLite | T | P2 | Derive the expected series count **from the registry**, don't guess it: `_MetricsCollector.collect` ([`api/metrics.py:246-441`](../../../messagefoundry/api/metrics.py)) yields a fixed block (`build_info`, `in_pipeline`, `store_committed_txns`, `store_body_copies`, + 4 host gauges when psutil reads, + the 9-series pool block on a server backend), `outbox_status` at one series per outbox status, **2 per inbound connection** (`messages_received`, `messages_errored`), **4 per (connection, destination) pair** (`deliveries`, `deliveries_dead`, `queue_depth`, `oldest_pending_age_seconds`) and **15 per pair** for `delivery_latency_seconds` (12 `DEFAULT_LATENCY_BUCKETS` + `+Inf` + `_sum` + `_count`). On the `harness/load/profiles/connscale.toml` estate assert total series `== K + 2·I + 19·D` **exactly** — i.e. growth is linear in inbound connections and in *existing* destination pairs, never the connection×destination×status product — that the scrape stays under a committed latency ceiling at that estate size, and that the label-**name** set is exactly the strict allowlist `{connection, destination, status, version, le}` (extending `tests/test_metrics_exporter.py:135`) with no PHI-shaped label value | | ALERT-41 | `/metrics` counter semantics survive a service restart | Compat | pytest | dev-PC | SQLite | T | P2 | Counters that are store-derived resume from the store value after a restart; process-local counters reset to 0 and are documented as such in the exporter's HELP text. No counter silently goes backwards without a documented reason | | ALERT-42 | `/metrics/history` is in-process and lost on restart, by design | Functional | pytest | dev-PC | n/a | T | P2 | Record samples, restart the app, assert `GET /metrics/history` returns an empty ring; assert `capacity == 900` and `min_interval == 0.9`; assert two concurrent `/ws/stats` samplers do not double-append the same instant | | ALERT-43 | OTel guarded-import failure message when `[otel]` is absent | Negative/Security | pytest | container-CI | n/a | T | P2 | With `opentelemetry` uninstalled, `build_otel_meter_provider()` and `OtelMetricsExporter(...)` each raise `RuntimeError` whose message names `pip install messagefoundry[otel]`. Currently marked `pragma: no cover` — this row removes the blind spot | diff --git a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md index edb5d2af..65c8a4bb 100644 --- a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md +++ b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md @@ -888,7 +888,7 @@ This area is signed off for release when **all** of the following hold: campaign? Which dependency (`hypothesis` / `atheris` / `schemathesis`) is approved?** *Blocks:* SEC-67. The HL7, X12, DICOM and base64-carriage parsers are the most attacker-exposed code in the product and today have only hand-written adversarial cases — and two - of them sit on single-maintainer upstreams with no dormancy contingency (`docs/BACKLOG.md:394`). + of them sit on single-maintainer upstreams with no dormancy contingency (BACKLOG #89). 13. **Which licence-resolution tool becomes the SEC-77 gate, and who ratifies the allow-list — specifically the rule that a term acceptable for AGPL distribution but not sublicensable under diff --git a/docs/testing/master-test-plan/18-interop-migration-and-uat.md b/docs/testing/master-test-plan/18-interop-migration-and-uat.md index 4a02a42c..fdae2560 100644 --- a/docs/testing/master-test-plan/18-interop-migration-and-uat.md +++ b/docs/testing/master-test-plan/18-interop-migration-and-uat.md @@ -164,7 +164,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-32 | The import `--json` ledger exposes its own denominator | Functional | pytest | dev-PC | n/a | T | P1 | `ImportResult.to_json()` (`corepoint_import.py:252-288`) emits a `total_statements` alongside mapped/unmapped/disabled, and `mapped + unmapped + disabled == total_statements` for every fixture. Today the CLI reports three numerators and no denominator, so a migrator cannot see the accounting invariant the tests assert internally | | MIG-33 | `import corepoint` CLI wrapper — end-to-end | Functional | pytest | dev-PC | n/a | C | P2 | **Owned as FCP:CLI-17** (FEATURE-COVERAGE-PLAN :185, :478, :1442). Close it there: drive `main(["import","corepoint",...])` for text and `--json`, assert exit codes and the summary shape. Two cases already exist (`test_corepoint_import.py:945`, `:968`) — the row is the argparse-surface and flag coverage, not a rewrite. **C**, because the falsifiable assertion lives in another document's ID space: this row records that FCP:CLI-17 moved off `partial`, so it cannot fail here and must not gate a MessageFoundry release. It becomes **T** the day the argparse-surface coverage is scoped into this plan instead | | MIG-34 | A `SPEC-105` specified-but-unbuilt mapping is reported as unmapped **with its cause**, never claimed | Functional | pytest | dev-PC | n/a | T | P2 | `EnvLogText` has a built vocabulary target (`messagefoundry/diagnostics.py:36` `log_note`) but no importer mapping (`grep EnvLogText corepoint_import.py` → none). Assert an `EnvLogText` statement lands in the unmapped bucket with a named `_decline_reason`, and that the emitted `# TODO: Corepoint …` marker names `log_note` as the hand-finish target. The same shape for the three-part `MsgSend` fix | -| MIG-35 | Every BACKLOG cross-reference resolves or is classified above the published baseline → **pointer** | Compat | — | dev-PC | n/a | T | P1 | **Covered by MIG-74; no separate work scoped.** The BACKLOG-reference classifier folds into MIG-74's doc-path linter so the publishing boundary is declared in exactly one allowlist rather than a second time inside `scripts/docs/backlog_status_check.py`. Context retained for this chapter's readers: a `#NNN` reference must resolve to a `## NNN.` section **or** be classified above-baseline — the published index ends at `## 231.` (`docs/BACKLOG.md:6883`, boundary prose at `:6041`) — and an above-baseline number such as #313 (cited at `:269` and `:4394`) is **valid evidence**, so the classifier must assert `NNN > 231` rather than tolerating any unresolved reference | +| MIG-35 | Every BACKLOG cross-reference resolves or is classified above the published baseline → **pointer** | Compat | — | dev-PC | n/a | T | P1 | **Covered by MIG-74; no separate work scoped.** The BACKLOG-reference classifier folds into MIG-74's doc-path linter so the publishing boundary is declared in exactly one allowlist rather than a second time inside `scripts/docs/backlog_status_check.py`. Context retained for this chapter's readers: a `#NNN` reference must resolve to a `## NNN.` section **or** be classified above-baseline — the published index ends at `## 231.` (boundary prose in BACKLOG #185) — and an above-baseline number such as #313 (cited at `:269` and `:4394`) is **valid evidence**, so the classifier must assert `NNN > 231` rather than tolerating any unresolved reference | | MIG-36 | `reconcile compare` default output is PHI-safe | PHI | pytest | dev-PC | n/a | T | P0 | Build a needle-bearing synthetic corpus (unique tokens in PID-5, PID-3, PID-7, OBX-5) with a deliberate mismatch. With no diff flag, assert **no needle** appears in stdout, stderr, or the `--report-json` artifact; only counts and match keys. Fails today: `report.py` `render_text` prints field values at :63-66 and `render_json` always emits `left`/`right` at :31-42 | | MIG-37 | `reconcile compare` gates diff disclosure behind an explicit, warned flag | PHI | pytest | dev-PC | n/a | T | P0 | A `--show-diffs` flag (mirroring `tee/__main__.py:269`) is required to reveal field values, and using it prints a PHI/TEST-DATA-ONLY warning to stderr before any diff (mirroring `tee/__main__.py:451-453`). Without it the exit code and counts are unchanged | | MIG-38 | The reconcile capture CLI makes de-identification the reachable path | PHI | pytest | dev-PC | n/a | T | P0 | The `capture` subparser exposes `--anonymize --salt-env ` wiring `CaptureSink(anonymizer=…)` (the kwarg already exists at `capture.py:46`); with it set, no un-anonymized body reaches the JSONL, verified with the ADR 0030 fail-closed leak gate. Without it, a stderr banner states that bodies are written verbatim. Fails today — `_run_capture` never passes the kwarg and no flag exists | @@ -203,7 +203,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-71 | Naive-user INSTALL-GUIDE walkthrough executed verbatim | Usability | manual | W2025-box | SQLite | T | P1 | A first-time operator with no prior Python or engine install follows `docs/INSTALL-GUIDE.md` top to bottom on a clean Windows Server 2025 box, typing only what the document prints. Every deviation, guess, or dead end is recorded as a finding with the line number. PASS = a serving engine, a reachable `/ui`, and one synthetic message through to `PROCESSED`, with **zero** undocumented steps. Repeat from a clean VM snapshot after each doc change | | MIG-72 | Naive-user USER-GUIDE + EARLY-ADOPTER-GUIDE Stage 0 walkthrough | Usability | manual | W2025-box | SQLite | C | P2 | Same discipline over `docs/USER-GUIDE.md` and `EARLY-ADOPTER-GUIDE.md` §11 Stage 0's four exit criteria. Findings are filed against line numbers. Synthetic corpus only, produced by `messagefoundry generate` | | MIG-73 | Area go/no-go roll-up is a signed artifact | Compat | manual | any | n/a | C | P1 | A per-release MIG sign-off sheet: every P0 row PASS; MIG-71 executed on the release candidate with zero undocumented steps; MIG-08/MIG-47's downgrade answer recorded in EARLY-ADOPTER-GUIDE §13; the MIG-60 partner checklist filled for every named partner in scope; a named approver accepting residual risk (mirroring **W25:S6.3**'s shape, not duplicating its content) | -| MIG-74 | **Every doc path named in shipped text or a docstring resolves** — one linter, one boundary allowlist | Compat | pytest | dev-PC | n/a | T | P1 | **Owner of the plan-wide "doc paths resolve" deliverable** — WEB-56 and SEC-09 point here and MIG-35 folds in; no separate work is scoped there. One new module walks every repo-relative path reference and markdown link target in `docs/**/*.md`, `README.md`, `CLAUDE.md` and the docstrings/comments of `messagefoundry/**`, `harness/**`, `tee/**`, `scripts/**`, `ide/src/**`, and resolves each on disk. Three outcomes, none of them silence: **RESOLVED**; **WITHHELD** — a path under `docs/security/`, `docs/reviews/` or `docs/marketing/` (gitignored post-cutover, `.gitignore:144-146`) matched against a checked-in manifest of expected names, reported as withheld and never a FAIL in a public checkout — and the manifest cannot rot vacuous (an empty manifest, or a WITHHELD claim for a path *outside* those three directories, FAILS); **BROKEN** — anything else reds the gate (`TEST-ENVIRONMENT-PLAN.md §5`, MIG-42, is today's example). Folds in MIG-35: a `#NNN` BACKLOG reference must resolve to a `## NNN.` section **or** classify as above-baseline, where `NNN > 231` (`docs/BACKLOG.md:6883`) is valid evidence and an unresolved `NNN ≤ 231` is a FAIL. A `file:line` citation is checked for file existence, and where the anchor names a symbol, that the symbol is present | +| MIG-74 | **Every doc path named in shipped text or a docstring resolves** — one linter, one boundary allowlist | Compat | pytest | dev-PC | n/a | T | P1 | **Owner of the plan-wide "doc paths resolve" deliverable** — WEB-56 and SEC-09 point here and MIG-35 folds in; no separate work is scoped there. One new module walks every repo-relative path reference and markdown link target in `docs/**/*.md`, `README.md`, `CLAUDE.md` and the docstrings/comments of `messagefoundry/**`, `harness/**`, `tee/**`, `scripts/**`, `ide/src/**`, and resolves each on disk. Three outcomes, none of them silence: **RESOLVED**; **WITHHELD** — a path under `docs/security/`, `docs/reviews/` or `docs/marketing/` (gitignored post-cutover, `.gitignore:144-146`) matched against a checked-in manifest of expected names, reported as withheld and never a FAIL in a public checkout — and the manifest cannot rot vacuous (an empty manifest, or a WITHHELD claim for a path *outside* those three directories, FAILS); **BROKEN** — anything else reds the gate (`TEST-ENVIRONMENT-PLAN.md §5`, MIG-42, is today's example). Folds in MIG-35: a `#NNN` BACKLOG reference must resolve to a `## NNN.` section **or** classify as above-baseline, where `NNN > 231` is valid evidence and an unresolved `NNN ≤ 231` is a FAIL. A `file:line` citation is checked for file existence, and where the anchor names a symbol, that the symbol is present | | MIG-75 | **Non-ASCII patient data survives ingest → store → console → export byte-for-byte** | Functional | pytest | dev-PC | x3 | T | P0 | A synthetic corpus whose PID-5, PID-3 and OBX-5 carry accented Latin (`Zoë Ångström-Muñoz`), CJK (`山田 太郎`) and a declared `MSH-18` character set (one `UNICODE UTF-8` case, one `8859/1` case), driven through every hop and byte-compared at each. **(1) Ingest:** the inbound decodes with the connection's declared `encoding` at `errors="strict"` (`pipeline/wiring_runner.py:3485-3494`) and `normalize()` collapses line endings only (`parsing/peek.py:152-163`); on an `MSH-18`-vs-configured-`encoding` conflict the engine must either honour `MSH-18` or record a **named `ERROR` disposition** — silently decoding under the other charset, or substituting U+FFFD into the stored raw, is the FAIL. **(2) Store:** the raw round-trips identical on all three backends and the at-rest-encrypted body decrypts to the same bytes — SQL Server text columns must be `NVARCHAR`, never `VARCHAR`, since a code-page collation silently mangling CJK is the classic version of this defect. **(3) Console:** `GET /messages/{id}` and the `/ui` render return the same code points (rides the console suite's seam, `packaging/messagefoundry-webconsole/tests/test_webui.py`). **(4) Export/attachment:** the NDJSON from `GET /messages/export` (`api/app.py:3035`, #124 / ADR 0131) and the reconstructed bytes from `GET /messages/{id}/attachments/{attachment_id}` (`:3225`, #149 / ADR 0105) are byte-identical to what arrived, base64 attachment path included (ADR 0028 `mfb64:v1:` — **never** latin-1). Finally the outbound re-encode emits the same bytes. **Gap being closed:** `MSH-18` occurs **nowhere** in the tree (0 hits repo-wide), so the declared character set is never consulted; the only adjacent coverage is `tests/test_mllp_encoding_override.py` (17 tests, delimiter re-encode, accented names incidental) and the IDE hex-pane row (ADR 0119). Synthetic names only | | MIG-76 | Wall clock stepped **backwards** while lanes drain — no reordering, no duplicate delivery, no stalled lane | HA/Resilience | pytest | dev-PC | x3 | T | P1 | Under a steady synthetic send with all three stages draining, step the wall clock back 90 s (monkeypatched `time.time` in-process; a container clock step on the CI leg) and assert four things. **(a) No reordering:** per-lane delivery order stays strict FIFO. ADR 0059 makes ordering `seq`-only precisely so a backward step cannot reorder — this row is the regression pin for that decision and for the **#285** per-lane-FIFO no-skip conformance gate (above the published #231 baseline); a claim ordered by `created_at, seq` would fail here, which is what makes the row falsifiable. **(b) No duplicate delivery:** each message is delivered exactly once — no in-flight row is re-claimed because a lease or due-time comparison went backwards (`queue.next_attempt_at`, `store/store.py:1183`; `ix_queue_ready`, `:1189`; `reset_stale_inflight`, `:5762`), and the leader lease neither flaps nor is stolen (`pipeline/cluster.py:94`, `lease_expires_at`). **(c) No stalled lane:** a retry stamped before the step still becomes due and every lane reaches `in_pipeline_depth() == 0` within the retry budget instead of waiting out the skew. **(d)** The forward-step direction fires no retry storm and no duplicate delivery. **Leap-second note:** a positive leap second is a repeated second, not a step, so (a)–(c) subsume it — but no assertion in this row (or in the code it guards) may depend on `time.time()` being *strictly* increasing: equal consecutive timestamps are legal and must not be treated as an ordering error | | MIG-77 | `connections.toml` forward/backward schema compatibility across engine versions | Compat | pytest | dev-PC | n/a | T | P1 | The existing rows test parity (code-first vs data-authored desugaring to identical `Registry` entries); this row tests **versioning**, which nothing does. `connections.toml` carries **no version marker** and its reader is a strict allowlist (`config/connections_file.py` `_INBOUND_KEYS`/`_OUTBOUND_KEYS`, `_reject_unknown` at :300-306) — unlike `connection schema --json`, which does emit `schemaVersion` (`config/connection_schema.py:45`) and which the IDE refuses forward-incompatibly. Pin both directions against a per-vintage fixture set built with MIG-06's generator discipline: **newer file → older engine** — a table carrying a key added after that engine (`flagged`, `deployed`, `auto_start` are the worked examples, `connections_file.py:110-119`) fails loud as a `WiringError` naming the unknown key **and** the file, never a silent drop of the key's effect and never a partial `Registry`; **older file → newer engine** — a table written before those keys existed loads unchanged and desugars into `Registry` entries byte-identical to today's, with the documented defaults (`auto_start`/`deployed` TRUE, `flagged` FALSE), so an adopter's committed file survives an engine upgrade untouched. Additionally assert that every key added to either allowlist since the previous release is named in the upgrade notes MIG-16 checks. If the owner later rules that a version marker is wanted, this row becomes its conformance test; either way the *behaviour* is pinned now | @@ -498,8 +498,8 @@ This area is signed off for a release when **all** of the following hold and are ### 17.9 Open questions -1. **Is `messagefoundry import corepoint` an adopter-facing capability or an internal migration aid?** `docs/AI-OFF-MATRIX.md:21` marks it "✅ shipped"; `docs/BACKLOG.md:4390` keeps #105 OPEN at P3 demand-gated; `docs/FEATURE-MAP.md` has **no row for it at all**. *Blocks:* the testing bar for MIG-31 through MIG-34, which differs by an order of magnitude between the two answers, and whether the importer appears in FEATURE-MAP at all (MIG-28). -2. **What is the decision on BACKLOG #313 (the multi-message Handler model), and is Corepoint verb coverage in scope for any release?** ~2,032 statements are refused for want of it (`HANDOFF-105:24-28`); #313 sits above the published #231 baseline (`docs/BACKLOG.md:6041`), which makes it sound evidence a public-only reader cannot open, not a doubtful citation. *Blocks:* whether MIG-31's pinned ledger is a *floor that must rise* or a *permanent cap*, and whether the SPEC-105 "specified but unbuilt" items (MIG-34) are scheduled or declined. +1. **Is `messagefoundry import corepoint` an adopter-facing capability or an internal migration aid?** `docs/AI-OFF-MATRIX.md:21` marks it "✅ shipped"; BACKLOG #105 is OPEN at P3, demand-gated; `docs/FEATURE-MAP.md` has **no row for it at all**. *Blocks:* the testing bar for MIG-31 through MIG-34, which differs by an order of magnitude between the two answers, and whether the importer appears in FEATURE-MAP at all (MIG-28). +2. **What is the decision on BACKLOG #313 (the multi-message Handler model), and is Corepoint verb coverage in scope for any release?** ~2,032 statements are refused for want of it (`HANDOFF-105:24-28`); #313 sits above the published #231 baseline (stated in BACKLOG #185), which makes it sound evidence a public-only reader cannot open, not a doubtful citation. *Blocks:* whether MIG-31's pinned ledger is a *floor that must rise* or a *permanent cap*, and whether the SPEC-105 "specified but unbuilt" items (MIG-34) are scheduled or declined. 3. **Is Mirth import ever in scope, or is the AI chat participant (`ide/src/chat.ts:40`, `ide/package.json:584-585`) the permanent answer?** This chapter's scope names "Corepoint/Mirth import"; only Corepoint exists as a deterministic path. *Blocks:* whether a MIG row for a Mirth importer belongs in the matrix at all, and whether shipped text may say "Mirth migration" without qualification. 4. **Should the greenfield-only SQLite→server-DB constraint be ENFORCED — refuse or loudly warn on a backend switch with a populated prior store — or remain documentation-only?** *Blocks:* MIG-01 and MIG-02 (a code change plus a test) vs. MIG-03 alone (a doc-drift test plus a recorded risk acceptance). This is the single highest-value decision in the chapter. 5. **What is the supported engine↔console version relationship: strict lockstep as coded (`SUPPORTED_ENGINE_SEAMS == frozenset({15})`), or an N-1 range?** The constant's own comment says re-widening requires "the CI matrix that installs the MIN and MAX supported engine builds — the claim and its test land together, or not at all." *Blocks:* whether MIG-15's PEP 508 range is `>=X,