diff --git a/docs/testing/master-test-plan/00-strategy-and-governance.md b/docs/testing/master-test-plan/00-strategy-and-governance.md index 3d11eb32..cfa0e207 100644 --- a/docs/testing/master-test-plan/00-strategy-and-governance.md +++ b/docs/testing/master-test-plan/00-strategy-and-governance.md @@ -10,13 +10,13 @@ single index that the project's existing, deeper test artifacts plug into — it them (§0.5). It exists because the estate is now large enough to lose things between artifacts: 535 `test_*.py` modules -under [`tests/`](tests/), 35 mocha suites under [`ide/src/test/suite/`](ide/src/test/suite/), 16 GitHub -Actions workflows under [`.github/workflows/`](.github/workflows/), a load/failover rig -([`harness/load/`](harness/load/)), an on-box acceptance runner -([`harness/acceptance/`](harness/acceptance/)), a parallel-run reconciliation harness -([`harness/reconcile/`](harness/reconcile/)), an in-product deployment-acceptance command -([`messagefoundry/verify/`](messagefoundry/verify/)), and a commit/CI gate -([`messagefoundry/checks.py`](messagefoundry/checks.py)). Each is well-run in isolation. Nothing until +under [`tests/`](../../../tests), 35 mocha suites under [`ide/src/test/suite/`](../../../ide/src/test/suite), 16 GitHub +Actions workflows under [`.github/workflows/`](../../../.github/workflows), a load/failover rig +([`harness/load/`](../../../harness/load)), an on-box acceptance runner +([`harness/acceptance/`](../../../harness/acceptance)), a parallel-run reconciliation harness +([`harness/reconcile/`](../../../harness/reconcile)), an in-product deployment-acceptance command +([`messagefoundry/verify/`](../../../messagefoundry/verify)), and a commit/CI gate +([`messagefoundry/checks.py`](../../../messagefoundry/checks.py)). Each is well-run in isolation. Nothing until now said, in one place, how they compose into a release decision. **Audience.** @@ -45,7 +45,7 @@ Rules: 2. **The ID is the citation key.** Commit messages, PR bodies, ADRs and BACKLOG items reference tests by ID, never by test-function name (function names get refactored; IDs do not). 3. **Disambiguation against the coverage plan (important).** - [`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) uses its own + [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) uses its own *gap*-ID space with several colliding prefixes — it has `PIPE-14`, `STORE-4`, `API-13`, `ALERT-12`, `CFG-19`, `PARSE-14`, `HA-*` and more. **In this document an unqualified `PIPE-14` always means this plan's test ID.** A coverage-plan gap is always written **`FCP:PIPE-14`**. A WIN2025 matrix row is @@ -78,31 +78,31 @@ table. | Method | Rig | Invocation | |---|---|---| -| `pytest-unit` | [`tests/`](tests/) | `pytest -q` (Qt tests need `QT_QPA_PLATFORM=offscreen`) | -| `pytest-int` | [`tests/`](tests/) with a live store / socket / service container | `pytest -q` under `MEFOR_TEST_SQLSERVER` / `MEFOR_TEST_POSTGRES` | -| `pytest-e2e` | [`tests/`](tests/) driving a served engine end to end | `pytest -q` | -| `mocha` | [`ide/src/test/suite/`](ide/src/test/suite/), host-free suites | `npm run test:unit` (ide/) | +| `pytest-unit` | [`tests/`](../../../tests) | `pytest -q` (Qt tests need `QT_QPA_PLATFORM=offscreen`) | +| `pytest-int` | [`tests/`](../../../tests) with a live store / socket / service container | `pytest -q` under `MEFOR_TEST_SQLSERVER` / `MEFOR_TEST_POSTGRES` | +| `pytest-e2e` | [`tests/`](../../../tests) driving a served engine end to end | `pytest -q` | +| `mocha` | [`ide/src/test/suite/`](../../../ide/src/test/suite), host-free suites | `npm run test:unit` (ide/) | | `electron` | Same suites under a real VS Code host | `npm test` (ide/, `@vscode/test-electron`) | -| `harness-scenario` | [`harness/`](harness/) headless scenarios | `python -m harness --scenario ` | -| `harness-load` | [`harness/load/`](harness/load/) | `python -m harness --load ` | -| `harness-probe` | [`harness/acceptance/`](harness/acceptance/) | `python -m harness.acceptance [--section A,B,C]` | -| `harness-reconcile` | [`harness/reconcile/`](harness/reconcile/) | `python -m harness.reconcile capture` / `compare` | -| `verify` | [`messagefoundry/verify/`](messagefoundry/verify/) | `messagefoundry verify --section host,store,smoke,manual,federation` | -| `check` | [`messagefoundry/checks.py`](messagefoundry/checks.py) | `messagefoundry check` (git hook + CI + IDE) | -| `manual` | A documented human step | Recorded in [`docs/testing/WIN2025-TEST-MATRIX.md`](docs/testing/WIN2025-TEST-MATRIX.md) | +| `harness-scenario` | [`harness/`](../../../harness) headless scenarios | `python -m harness --scenario ` | +| `harness-load` | [`harness/load/`](../../../harness/load) | `python -m harness --load ` | +| `harness-probe` | [`harness/acceptance/`](../../../harness/acceptance) | `python -m harness.acceptance [--section A,B,C]` | +| `harness-reconcile` | [`harness/reconcile/`](../../../harness/reconcile) | `python -m harness.reconcile capture` / `compare` | +| `verify` | [`messagefoundry/verify/`](../../../messagefoundry/verify) | `messagefoundry verify --section host,store,smoke,manual,federation` | +| `check` | [`messagefoundry/checks.py`](../../../messagefoundry/checks.py) | `messagefoundry check` (git hook + CI + IDE) | +| `manual` | A documented human step | Recorded in [`docs/testing/WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md) | **Env** — where it runs. (The self-hosted NucBox Windows runners are retired; the `test` matrix is -GitHub-hosted, per the comment at [`.github/workflows/ci.yml`](.github/workflows/ci.yml) job `test`.) +GitHub-hosted, per the comment at [`.github/workflows/ci.yml`](../../../.github/workflows/ci.yml) job `test`.) | Env | Meaning | |---|---| | `dev` | Developer workstation / worktree | | `ci-hosted` | GitHub-hosted runners — `ubuntu-latest`, `windows-2022`, `windows-2025` (`ci.yml` job `test`) | | `ci-service` | GitHub-hosted with a service container — `sqlserver-store`, `postgres-store`, `docker-smoke` | -| `ci-selfhosted` | [`selfhosted-win2025-sql.yml`](.github/workflows/selfhosted-win2025-sql.yml), job `sqlserver-real` | -| `win2025-box` | The Windows Server 2025 acceptance box ([`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md)) | +| `ci-selfhosted` | [`selfhosted-win2025-sql.yml`](../../../.github/workflows/selfhosted-win2025-sql.yml), job `sqlserver-real` | +| `win2025-box` | The Windows Server 2025 acceptance box ([`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md)) | | `ad-lab` | The AD / Kerberos lab (WIN2025 plan, AD-lab rows) | -| `perf-rig` | The dedicated throughput rig used by [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md) and `benchmark.yml` | +| `perf-rig` | The dedicated throughput rig used by [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) and `benchmark.yml` | **Backend** — `sqlite` · `sqlserver` · `postgres` · `all3` · `n/a`. `all3` means the assertion **executes** on each backend (gated CI legs are acceptable; a structural DDL/`dir()` check is not). @@ -117,22 +117,22 @@ toward the countable release gate. | # | Prefix | Chapter | Primary code under test | |---|---|---|---| -| 1 | `PIPE` | Staged pipeline, workers, dispositions, at-least-once | [`messagefoundry/pipeline/`](messagefoundry/pipeline/) | -| 2 | `STORE` | Store backends, staged queue, dead-letter, retention, at-rest crypto | [`messagefoundry/store/`](messagefoundry/store/) | +| 1 | `PIPE` | Staged pipeline, workers, dispositions, at-least-once | [`messagefoundry/pipeline/`](../../../messagefoundry/pipeline) | +| 2 | `STORE` | Store backends, staged queue, dead-letter, retention, at-rest crypto | [`messagefoundry/store/`](../../../messagefoundry/store) | | 3 | `HA` | Active-passive failover, engine shards, DR backup/restore | `pipeline/`, `store/`, `backup` / `restore-verify` | -| 4 | `CONN` | Connections / transports (MLLP, File, DB, REST/SOAP/FHIR, DICOM, X12, email) | [`messagefoundry/transports/`](messagefoundry/transports/) | -| 5 | `PARSE` | HL7 peek/strict/tree, payload-agnostic ingress, X12 + DICOM codecs, binary carriage | [`messagefoundry/parsing/`](messagefoundry/parsing/) | -| 6 | `CFG` | Wiring, `connections.toml`, environments, code sets, service settings | [`messagefoundry/config/`](messagefoundry/config/) | +| 4 | `CONN` | Connections / transports (MLLP, File, DB, REST/SOAP/FHIR, DICOM, X12, email) | [`messagefoundry/transports/`](../../../messagefoundry/transports) | +| 5 | `PARSE` | HL7 peek/strict/tree, payload-agnostic ingress, X12 + DICOM codecs, binary carriage | [`messagefoundry/parsing/`](../../../messagefoundry/parsing) | +| 6 | `CFG` | Wiring, `connections.toml`, environments, code sets, service settings | [`messagefoundry/config/`](../../../messagefoundry/config) | | 7 | `PUB` | The publish/promote path: `check` → dry-run pre-flight → `POST /config/reload` (incl. dual control) | `checks.py`, `api/app.py:2741`, `ide/src/promote.ts` | -| 8 | `API` | The FastAPI engine API surface | [`messagefoundry/api/`](messagefoundry/api/) | -| 9 | `AUTH` | Authentication (local / AD / Kerberos, TOTP, passkeys, sessions), RBAC, audit | [`messagefoundry/auth/`](messagefoundry/auth/) | -| 10 | `WEB` | Web console at `/ui` | [`messagefoundry_webconsole/`](messagefoundry_webconsole/) | -| 11 | `IDE` | VS Code extension (setup, wizards, engine link, test bench, AI commands) | [`ide/`](ide/) | -| 12 | `STEPS` | Steps view — the typed step vocabulary and its Python codegen/round-trip | [`messagefoundry/lens.py`](messagefoundry/lens.py), `ide/src/stepsModel.ts` | -| 13 | `TRAY` | Tray app + its frozen boundary (**13a**), the NSSM Windows service (**13b**), distribution & install (**13c**), and the standalone PySide6 **test harness GUI** (**13d**) | [`messagefoundry/tray/`](messagefoundry/tray/), [`scripts/service/`](scripts/service/), [`harness/`](harness/) | -| 14 | `ALERT` | Alerting, alert state, health/stats, observability | [`messagefoundry/pipeline/alerts.py`](messagefoundry/pipeline/alerts.py) (the `AlertSink` Protocol, `:27`), [`alert_sinks.py`](messagefoundry/pipeline/alert_sinks.py) (`NotifierAlertSink`) | -| 15 | `SEC` | TLS/posture, egress allowlists, redaction, key lifecycle, supply chain | [`messagefoundry/security/`](messagefoundry/security/), `pki.py`, `redaction.py`, `security.yml` | -| 16 | `PERF` | Throughput, latency, engine-shard scaling, soak — and the honesty of the instruments | [`harness/load/`](harness/load/) | +| 8 | `API` | The FastAPI engine API surface | [`messagefoundry/api/`](../../../messagefoundry/api) | +| 9 | `AUTH` | Authentication (local / AD / Kerberos, TOTP, passkeys, sessions), RBAC, audit | [`messagefoundry/auth/`](../../../messagefoundry/auth) | +| 10 | `WEB` | Web console at `/ui` | [`messagefoundry_webconsole/`](../../../messagefoundry_webconsole) | +| 11 | `IDE` | VS Code extension (setup, wizards, engine link, test bench, AI commands) | [`ide/`](../../../ide) | +| 12 | `STEPS` | Steps view — the typed step vocabulary and its Python codegen/round-trip | [`messagefoundry/lens.py`](../../../messagefoundry/lens.py), `ide/src/stepsModel.ts` | +| 13 | `TRAY` | Tray app + its frozen boundary (**13a**), the NSSM Windows service (**13b**), distribution & install (**13c**), and the standalone PySide6 **test harness GUI** (**13d**) | [`messagefoundry/tray/`](../../../messagefoundry/tray), [`scripts/service/`](../../../scripts/service), [`harness/`](../../../harness) | +| 14 | `ALERT` | Alerting, alert state, health/stats, observability | [`messagefoundry/pipeline/alerts.py`](../../../messagefoundry/pipeline/alerts.py) (the `AlertSink` Protocol, `:27`), [`alert_sinks.py`](../../../messagefoundry/pipeline/alert_sinks.py) (`NotifierAlertSink`) | +| 15 | `SEC` | TLS/posture, egress allowlists, redaction, key lifecycle, supply chain | [`messagefoundry/security/`](../../../messagefoundry/security), `pki.py`, `redaction.py`, `security.yml` | +| 16 | `PERF` | Throughput, latency, engine-shard scaling, soak — and the honesty of the instruments | [`harness/load/`](../../../harness/load) | | 17 | `MIG` | Install / upgrade / packaging, store schema migration, Corepoint import, parallel-run cutover | `packaging/`, `docker/`, `scripts/service/`, `corepoint_import.py`, `harness/reconcile/` | --- @@ -145,20 +145,20 @@ names the chapter that owns it. | Surface | What it is, concretely | Owning chapter(s) | |---|---|---| -| **Engine core** | Headless **asyncio** service under uvicorn/FastAPI. One listener + a **router worker** + a **transform worker** per inbound Connection, one delivery worker per outbound Connection, supervised by `RegistryRunner` ([`pipeline/wiring_runner.py`](messagefoundry/pipeline/wiring_runner.py)) | `PIPE` | +| **Engine core** | Headless **asyncio** service under uvicorn/FastAPI. One listener + a **router worker** + a **transform worker** per inbound Connection, one delivery worker per outbound Connection, supervised by `RegistryRunner` ([`pipeline/wiring_runner.py`](../../../messagefoundry/pipeline/wiring_runner.py)) | `PIPE` | | **Staged queue** | `ingress → routed → outbound` stages on a transactional store; dispositions `RECEIVED / ROUTED / UNROUTED / PROCESSED / FILTERED / ERROR`; ACK-on-receipt; `reset_stale_inflight` recovery | `PIPE`, `STORE` | -| **Store backends (3)** | SQLite/WAL ([`store/store.py`](messagefoundry/store/store.py)), SQL Server ([`store/sqlserver.py`](messagefoundry/store/sqlserver.py)), PostgreSQL ([`store/postgres.py`](messagefoundry/store/postgres.py)) behind the `Store` protocol + `open_store` ([`store/base.py`](messagefoundry/store/base.py)) | `STORE` (+ `XBE`-typed rows everywhere) | -| **Connections** | MLLP/TCP, File + RemoteFile, DB source/destination, REST/SOAP/FHIR (+ SMART token provider), DICOM C-STORE SCP/SCU + C-ECHO, DICOMweb STOW-RS, X12 raw-TCP, email — all resolved through the connector registry ([`transports/base.py`](messagefoundry/transports/base.py)) | `CONN` | +| **Store backends (3)** | SQLite/WAL ([`store/store.py`](../../../messagefoundry/store/store.py)), SQL Server ([`store/sqlserver.py`](../../../messagefoundry/store/sqlserver.py)), PostgreSQL ([`store/postgres.py`](../../../messagefoundry/store/postgres.py)) behind the `Store` protocol + `open_store` ([`store/base.py`](../../../messagefoundry/store/base.py)) | `STORE` (+ `XBE`-typed rows everywhere) | +| **Connections** | MLLP/TCP, File + RemoteFile, DB source/destination, REST/SOAP/FHIR (+ SMART token provider), DICOM C-STORE SCP/SCU + C-ECHO, DICOMweb STOW-RS, X12 raw-TCP, email — all resolved through the connector registry ([`transports/base.py`](../../../messagefoundry/transports/base.py)) | `CONN` | | **Routers & Handlers** | Code-first `@router` / `@handler` Python, wired by name; purity contract; the sanctioned read-only `db_lookup` / `fhir_lookup` carve-outs | `PIPE` (execution), `CFG` (loading), `STEPS` (authoring) | | **Parsing** | python-hl7 tolerant peek on the hot path, hl7apy opt-in strict validation, tree/message model, X12 + DICOM codecs, `mfb64:v1:` binary carriage | `PARSE` | | **HA / scale** | Active-passive engine failover over one unified store; **engine shards** (`serve --shard`, N processes over ONE store); DR `backup` / `restore-verify`. (**Database** sharding is shelved and out of scope — §0.10.) | `HA` | -| **Engine API** | The engine's only external surface: [`api/app.py`](messagefoundry/api/app.py) + `api/security.py` + `auth_routes.py`, bound to `127.0.0.1` by default, deny-by-default per-route permissions | `API`, `AUTH` | -| **Web console `/ui`** | The **sole operator console** — a browser SPA served **same-origin** by the engine's own app ([`messagefoundry_webconsole/mount.py`](messagefoundry_webconsole/mount.py)); talks HTTP/WS only, never imports the engine or touches the DB. Ships as its own distribution (`packaging/messagefoundry-webconsole`, released by `release.yml` job `release-webconsole`) | `WEB` | -| **VS Code IDE extension** | TypeScript authoring surface ([`ide/`](ide/)) — setup, connection wizard/editors, graph tree, code sets, live debug, trace view, test bench, engine link doctor, promote | `IDE`, `PUB`, `STEPS` | -| **Windows tray app** | Unprivileged notification-area service manager ([`messagefoundry/tray/`](messagefoundry/tray/), stdlib `ctypes`, no Qt). Reads exactly two credential-free signals — SCM service state and tokenless `GET /health` — and deep-links to `/ui`. Boundary frozen by [`tests/test_tray_boundary.py`](tests/test_tray_boundary.py) | `TRAY` | -| **PySide6 test harness** | Standalone send/receive/compose/monitor GUI + headless scenarios + the load, acceptance and reconcile rigs ([`harness/`](harness/)). A **test instrument first**, but also a shipped distribution (`packaging/messagefoundry-harness`, `release.yml` job `release-harness`). **The retired PySide6 desktop console is not a surface** — PySide6 backs only this harness | **`TRAY` (part 13d)** owns the harness GUI — launch path, sign-in dialog, panels, and the `messagefoundry-harness` wheel; `PERF` owns the load rig's instrument correctness. *(No longer "the chapter using it": the plan's only hostile-input and fault-injection instrument has a named owner.)* | -| **CLI** | ~30 subcommands in [`messagefoundry/__main__.py`](messagefoundry/__main__.py) — see the split below | distributed (table below) | -| **Deployment substrate** | NSSM Windows service ([`scripts/service/install-service.ps1`](scripts/service/install-service.ps1)), the wheel, `docker/` (Dockerfile, compose, k8s), `environments/` value files (`dev.toml`, `prod.toml` today), signed release pipeline (`release.yml`) | `MIG` (+ `W25:` for on-box acceptance) | +| **Engine API** | The engine's only external surface: [`api/app.py`](../../../messagefoundry/api/app.py) + `api/security.py` + `auth_routes.py`, bound to `127.0.0.1` by default, deny-by-default per-route permissions | `API`, `AUTH` | +| **Web console `/ui`** | The **sole operator console** — a browser SPA served **same-origin** by the engine's own app ([`messagefoundry_webconsole/mount.py`](../../../messagefoundry_webconsole/mount.py)); talks HTTP/WS only, never imports the engine or touches the DB. Ships as its own distribution (`packaging/messagefoundry-webconsole`, released by `release.yml` job `release-webconsole`) | `WEB` | +| **VS Code IDE extension** | TypeScript authoring surface ([`ide/`](../../../ide)) — setup, connection wizard/editors, graph tree, code sets, live debug, trace view, test bench, engine link doctor, promote | `IDE`, `PUB`, `STEPS` | +| **Windows tray app** | Unprivileged notification-area service manager ([`messagefoundry/tray/`](../../../messagefoundry/tray), stdlib `ctypes`, no Qt). Reads exactly two credential-free signals — SCM service state and tokenless `GET /health` — and deep-links to `/ui`. Boundary frozen by [`tests/test_tray_boundary.py`](../../../tests/test_tray_boundary.py) | `TRAY` | +| **PySide6 test harness** | Standalone send/receive/compose/monitor GUI + headless scenarios + the load, acceptance and reconcile rigs ([`harness/`](../../../harness)). A **test instrument first**, but also a shipped distribution (`packaging/messagefoundry-harness`, `release.yml` job `release-harness`). **The retired PySide6 desktop console is not a surface** — PySide6 backs only this harness | **`TRAY` (part 13d)** owns the harness GUI — launch path, sign-in dialog, panels, and the `messagefoundry-harness` wheel; `PERF` owns the load rig's instrument correctness. *(No longer "the chapter using it": the plan's only hostile-input and fault-injection instrument has a named owner.)* | +| **CLI** | ~30 subcommands in [`messagefoundry/__main__.py`](../../../messagefoundry/__main__.py) — see the split below | distributed (table below) | +| **Deployment substrate** | NSSM Windows service ([`scripts/service/install-service.ps1`](../../../scripts/service/install-service.ps1)), the wheel, `docker/` (Dockerfile, compose, k8s), `environments/` value files (`dev.toml`, `prod.toml` today), signed release pipeline (`release.yml`) | `MIG` (+ `W25:` for on-box acceptance) | **CLI ownership** (the CLI has no chapter of its own; each subcommand is owned by the subsystem it drives): @@ -184,7 +184,7 @@ drives): |---|---|---| | **Unit** | Does this function/class behave, including its error branches? | `tests/` (`pytest-unit`), `ide/src/test/suite/` (`mocha`) | | **Component / integration** | Do two or more real components work against a real store / socket / service container? | `tests/` under `MEFOR_TEST_SQLSERVER` / `MEFOR_TEST_POSTGRES`; `ci.yml` jobs `sqlserver-store`, `postgres-store` | -| **Contract** | Does a boundary hold its shape — API ↔ client, matrix ↔ code, web console seam, tray boundary? | [`tests/test_webconsole_seam_snapshot.py`](tests/test_webconsole_seam_snapshot.py), [`tests/test_win2025_acceptance.py`](tests/test_win2025_acceptance.py), [`tests/test_tray_boundary.py`](tests/test_tray_boundary.py), [`tests/test_api_health_tokenless.py`](tests/test_api_health_tokenless.py) | +| **Contract** | Does a boundary hold its shape — API ↔ client, matrix ↔ code, web console seam, tray boundary? | [`tests/test_webconsole_seam_snapshot.py`](../../../tests/test_webconsole_seam_snapshot.py), [`tests/test_win2025_acceptance.py`](../../../tests/test_win2025_acceptance.py), [`tests/test_tray_boundary.py`](../../../tests/test_tray_boundary.py), [`tests/test_api_health_tokenless.py`](../../../tests/test_api_health_tokenless.py) | | **System / end-to-end** | Does a synthetic message traverse a served engine and land with the right disposition? | `harness/config` disposition graph + `python -m harness --scenario …`; `messagefoundry verify --smoke live --check-disposition` | | **Host acceptance** | Does it install, run, survive reboot, and behave **under the service identity** on a real Windows Server 2025 box? | `harness/acceptance` probes + `WIN2025-TEST-MATRIX.md` (owned there, not here) | | **Performance** | Throughput, latency, engine-shard scaling, failover-under-load, soak | `harness/load/` (`--load`, `--failover`, `--estate`), `benchmark.yml` (`baseline-sqlite` / `-postgres` / `-sqlserver`) | @@ -201,7 +201,7 @@ drives): | `ide` mocha (`npm run test:unit`) | Unit | Deliberately ignores 8 suites that need a VS Code host (`package.json` `test:unit`) | | `ide` electron (`npm test`) | Component / UX | Runs the full suite under a real VS Code host; `ci.yml` job `ide` is **path-gated** (`needs.changes.outputs.ide`) | | `harness` scenarios (`--scenario`) | System / e2e | Needs a served engine; asserts disposition, not throughput | -| `harness/load` (`--load` / `--failover` / `--estate`) | Performance, resilience | Reports metrics only; SLO verdicts and exit codes per [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md) §"Exit codes" | +| `harness/load` (`--load` / `--failover` / `--estate`) | Performance, resilience | Reports metrics only; SLO verdicts and exit codes per [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) §"Exit codes" | | `harness/acceptance` (`python -m harness.acceptance`) | Host acceptance | Executes probes + re-runs backing pytest suites; **never fakes green** — unautomatable rows report `MANUAL` | | `harness/reconcile` (`capture` / `compare`) | Migration / parallel run | Offline comparison core; normalizes engine-non-deterministic fields; exits non-zero on a real diff so it can gate a per-Connection sign-off | | `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](messagefoundry/verify/runner.py:22)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message | @@ -210,8 +210,8 @@ drives): | `quality-advisory.yml` | Meta / test-quality | `complexity`, `clone`, `coverage` (diff-cover), `mutation` — all **advisory, never required**; the `liveness` job is the meta-gate that demands proof a signal actually executed | **Anti-metric rule (inherited, restated).** Coverage % and mutation score are **surfaced, never gated** -— per [`docs/Code_Quality_Standards.md`](docs/Code_Quality_Standards.md) §4.1 and the handoff at -[`docs/quality-gates/HANDOFF-mutation-coverage.md`](docs/quality-gates/HANDOFF-mutation-coverage.md). +— per [`docs/Code_Quality_Standards.md`](../../Code_Quality_Standards.md) §4.1 and the handoff at +[`docs/quality-gates/HANDOFF-mutation-coverage.md`](../../quality-gates/HANDOFF-mutation-coverage.md). This plan does not introduce a coverage-percentage exit criterion. --- @@ -238,7 +238,7 @@ hospital network. Seven failure classes define priority; everything else is down | Pri | Definition | Consequence | |---|---|---| | **P0** | Proves an **R1–R4** property, or the **absence of silence** for R5, on a path that ships by default. A regression here is unacceptable at any release | Blocks a release. Must be automated and required in CI, or — where only a box can prove it (`win2025-box`, `ad-lab`) — must be an explicitly signed-off row before the tag | -| **P1** | Proves R5–R7, or an R1–R4 property on a **non-default / opt-in** path (a connector behind an extra, a backend not promoted for prod, a feature flagged off). Also: cross-backend (`XBE`) execution of an already-P0-proven behaviour | Blocks a release **for the affected feature**; a documented, dated deferral by the owner is permissible if the feature's status is downgraded accordingly in [`docs/FEATURE-MAP.md`](docs/FEATURE-MAP.md) | +| **P1** | Proves R5–R7, or an R1–R4 property on a **non-default / opt-in** path (a connector behind an extra, a backend not promoted for prod, a feature flagged off). Also: cross-backend (`XBE`) execution of an already-P0-proven behaviour | Blocks a release **for the affected feature**; a documented, dated deferral by the owner is permissible if the feature's status is downgraded accordingly in [`docs/FEATURE-MAP.md`](../../FEATURE-MAP.md) | | **P2** | Breadth, ergonomics, diagnostics quality, and hardening beyond the shipped default posture | Never blocks a release. Scheduled opportunistically or when its area is being touched anyway | ### 0.4.3 The selection rule: **silent AND material** @@ -329,13 +329,13 @@ COVERAGE-PLAN (PLAN/MATRIX/ (`verify` .md .md HAND | Artifact | It owns | This plan's relationship | |---|---|---| -| [`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) | The **gap audit**: 128 gaps across 24 subsystems and six dimensions, phased P0–P7, plus the "what NOT to re-test" list and the verification errata | **The backlog of work.** Part II chapters cite its gap IDs as `FCP:` and inherit its dimension vocabulary. This plan never re-derives a gap it already found, and never re-tests anything on its `covered` / "leave alone" list | -| [`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](docs/testing/WIN2025-TEST-MATRIX.md) + [`WIN2025-ACCEPTANCE.md`](docs/testing/WIN2025-ACCEPTANCE.md) | **Everything only a real box can prove**: the CI-OWNED / BOX-OWNED split, the 54-row matrix (sections A–H), the eight "false-green" gaps (`W25:S2.1`–`W25:S2.8`), the manual `S1.AC-*` rows, and the executable runner `python -m harness.acceptance` | **Delegated wholesale.** Any `Env: win2025-box` or `ad-lab` row in Part II is a **pointer** (`W25:`), never a restatement. This plan's release exit criteria (§0.6) *require* a WIN2025 pass; it does not describe how to perform one | -| [`docs/testing/VERIFY.md`](docs/testing/VERIFY.md) | The `messagefoundry verify` command: its five sections, `self` vs `live` smoke, per-DB validation, and — critically — **what a green run does not prove** | **Cited as the deployment-acceptance instrument.** Part II `Method: verify` rows name the section and flags; the semantics live in VERIFY.md | -| [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md) | The load rig: profiles, engine load-config knobs, the report schema and exit codes, backend comparison, `--failover`, the `--estate` 1,500-Connection demo, known limitations | **Cited as the performance instrument.** `PERF` chapter rows specify profile + shape + falsifier; they do not re-document the rig | -| [`docs/CI-QUALITY.md`](docs/CI-QUALITY.md) | The **plain-language** account of automated testing for a non-engineer reader (~2,500 PR checks / ~2,600 post-merge, four system setups), measured 2026-06-20 | **Audience sibling, not a dependency.** When this plan changes what runs, CI-QUALITY.md's figures are refreshed from `.github/workflows/` — the workflow files remain the source of truth | -| [`docs/quality-gates/HANDOFF-mutation-coverage.md`](docs/quality-gates/HANDOFF-mutation-coverage.md) | The advisory mutation + diff-coverage gates and the advisory-first ground rules | **Inherited, unchanged.** This plan adopts "surface, never gate" for coverage/mutation scores | -| [`docs/FEATURE-MAP.md`](docs/FEATURE-MAP.md) | The capability catalog with ✅ / 🔬 / 🔨 / ⏭️ / 🧭 status per feature | **Traceability target** (§0.9). A ✅ row with no P0/P1 test ID is itself a finding | +| [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) | The **gap audit**: 128 gaps across 24 subsystems and six dimensions, phased P0–P7, plus the "what NOT to re-test" list and the verification errata | **The backlog of work.** Part II chapters cite its gap IDs as `FCP:` and inherit its dimension vocabulary. This plan never re-derives a gap it already found, and never re-tests anything on its `covered` / "leave alone" list | +| [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md) + [`WIN2025-ACCEPTANCE.md`](../WIN2025-ACCEPTANCE.md) | **Everything only a real box can prove**: the CI-OWNED / BOX-OWNED split, the 54-row matrix (sections A–H), the eight "false-green" gaps (`W25:S2.1`–`W25:S2.8`), the manual `S1.AC-*` rows, and the executable runner `python -m harness.acceptance` | **Delegated wholesale.** Any `Env: win2025-box` or `ad-lab` row in Part II is a **pointer** (`W25:`), never a restatement. This plan's release exit criteria (§0.6) *require* a WIN2025 pass; it does not describe how to perform one | +| [`docs/testing/VERIFY.md`](../VERIFY.md) | The `messagefoundry verify` command: its five sections, `self` vs `live` smoke, per-DB validation, and — critically — **what a green run does not prove** | **Cited as the deployment-acceptance instrument.** Part II `Method: verify` rows name the section and flags; the semantics live in VERIFY.md | +| [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) | The load rig: profiles, engine load-config knobs, the report schema and exit codes, backend comparison, `--failover`, the `--estate` 1,500-Connection demo, known limitations | **Cited as the performance instrument.** `PERF` chapter rows specify profile + shape + falsifier; they do not re-document the rig | +| [`docs/CI-QUALITY.md`](../../CI-QUALITY.md) | The **plain-language** account of automated testing for a non-engineer reader (~2,500 PR checks / ~2,600 post-merge, four system setups), measured 2026-06-20 | **Audience sibling, not a dependency.** When this plan changes what runs, CI-QUALITY.md's figures are refreshed from `.github/workflows/` — the workflow files remain the source of truth | +| [`docs/quality-gates/HANDOFF-mutation-coverage.md`](../../quality-gates/HANDOFF-mutation-coverage.md) | The advisory mutation + diff-coverage gates and the advisory-first ground rules | **Inherited, unchanged.** This plan adopts "surface, never gate" for coverage/mutation scores | +| [`docs/FEATURE-MAP.md`](../../FEATURE-MAP.md) | The capability catalog with ✅ / 🔬 / 🔨 / ⏭️ / 🧭 status per feature | **Traceability target** (§0.9). A ✅ row with no P0/P1 test ID is itself a finding | ### 0.5.1 The DO-NOT-DUPLICATE rule @@ -396,7 +396,7 @@ pointer. Different assertions, different rigs, both survive. **The two largest consolidations** are the FEATURE-MAP drift guard — **fifteen** near-identical chapter rows collapsing into one `MIG` row extending -[`tests/test_feature_map_claims.py`](tests/test_feature_map_claims.py) — and the "doc paths resolve" +[`tests/test_feature_map_claims.py`](../../../tests/test_feature_map_claims.py) — and the "doc paths resolve" linter, which collapses three. **Why a pointer row is `Cls T` even when its owner's row is `C`.** The pointer asserts a *checkable* @@ -417,10 +417,10 @@ two collisions below survived into the draft. exceptions, no "obvious from context". Bare cross-*chapter* citations are fine — they still resolve inside Part II. 2. Every reference to **another document's** ID carries that document's prefix: **`FCP:`** for a - [`FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) gap ID (`FCP:HA-20`, + [`FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) gap ID (`FCP:HA-20`, `FCP:API-13`, `FCP:P7`, `FCP:DEPLOY-27`); **`W25:`** for a WIN2025 test, matrix row or section (`W25:S2.1`, `W25:S1.AC-3`, `W25:§C`). The acceptance runner's own rows already follow the same - shape and keep it — `ACC:A7`, `ACC:F7` ([`harness/acceptance/matrix.py`](harness/acceptance/matrix.py)). + shape and keep it — `ACC:A7`, `ACC:F7` ([`harness/acceptance/matrix.py`](../../../harness/acceptance/matrix.py)). 3. ADR and BACKLOG numbers keep their existing unambiguous forms (`ADR 0101`, `BACKLOG #219`); they do not collide with the `-` space and take no prefix. @@ -428,11 +428,11 @@ two collisions below survived into the draft. | Where | Bare ID | This plan's row | The foreign row | Correct forms | |---|---|---|---|---| -| `12-ha.md` | `HA-20` | The chapter's own HA-20 test row | The coverage-plan gap **`FCP:HA-20`**, *"Web console HA / cluster page … legacy PySide6 only: test_console_status.py; **NO webconsole test**"*, verdict *"Web console (in scope) has no HA surface; only deprecated PySide6 renders the `/cluster/` routes"* — [`FEATURE-COVERAGE-PLAN.md:1201`](docs/testing/FEATURE-COVERAGE-PLAN.md), inside §16 *HA / active-passive clustering & failover*, lines **1174–1213** | `HA-20` for this plan's row; **`FCP:HA-20`** for the gap | -| `17-api.md` | `API-13` | The chapter's own API-13 test row | The coverage-plan gap **`FCP:API-13`** — *"Resend API (RESEND step-up, cross-channel, idempotency, retention-null 409, FIFO funnel)"* ([`FEATURE-COVERAGE-PLAN.md:1147`](docs/testing/FEATURE-COVERAGE-PLAN.md), §15 *FastAPI engine API surface* `[API]`) | `API-13` for this plan's row; **`FCP:API-13`** for the gap | +| `12-ha.md` | `HA-20` | The chapter's own HA-20 test row | The coverage-plan gap **`FCP:HA-20`**, *"Web console HA / cluster page … legacy PySide6 only: test_console_status.py; **NO webconsole test**"*, verdict *"Web console (in scope) has no HA surface; only deprecated PySide6 renders the `/cluster/` routes"* — [`FEATURE-COVERAGE-PLAN.md:1201`](../FEATURE-COVERAGE-PLAN.md), inside §16 *HA / active-passive clustering & failover*, lines **1174–1213** | `HA-20` for this plan's row; **`FCP:HA-20`** for the gap | +| `17-api.md` | `API-13` | The chapter's own API-13 test row | The coverage-plan gap **`FCP:API-13`** — *"Resend API (RESEND step-up, cross-channel, idempotency, retention-null 409, FIFO funnel)"* ([`FEATURE-COVERAGE-PLAN.md:1147`](../FEATURE-COVERAGE-PLAN.md), §15 *FastAPI engine API surface* `[API]`) | `API-13` for this plan's row; **`FCP:API-13`** for the gap | Both files carried **two different rows under one identifier** — the same failure mode the ledger gate -exists to stop in the ADR/BACKLOG ledgers ([`docs/LEDGER-GATE.md`](docs/LEDGER-GATE.md)): two writers, +exists to stop in the ADR/BACKLOG ledgers ([`docs/LEDGER-GATE.md`](../../LEDGER-GATE.md)): two writers, one number, a clean merge, a corrupted index. **Maintenance — this is checked mechanically.** A convention that is only stated is a convention that @@ -489,7 +489,7 @@ campaign such as a box-acceptance run). | 1 | Full CI green, including the nightly-only heavy legs, on the RC commit | `gh workflow run ci.yml --ref main` (`workflow_dispatch` runs **everything**), not merely the last nightly | | 2 | Every **P0 `T` row** across all 17 chapters passes | Part II status columns + each chapter's §x.4 class declaration | | 3 | Cross-backend (`XBE`) P0/P1 **`T`** rows executed on **all three** backends | `sqlserver-store`, `postgres-store`, and — for the real-server leg — `selfhosted-win2025-sql.yml` job `sqlserver-real` | -| 4 | A WIN2025 box-acceptance pass with no FAIL, and every MANUAL row human-closed | `python -m harness.acceptance` report + the matrix Status column ([`WIN2025-ACCEPTANCE.md`](docs/testing/WIN2025-ACCEPTANCE.md)) | +| 4 | A WIN2025 box-acceptance pass with no FAIL, and every MANUAL row human-closed | `python -m harness.acceptance` report + the matrix Status column ([`WIN2025-ACCEPTANCE.md`](../WIN2025-ACCEPTANCE.md)) | | 5 | `messagefoundry verify` green per backend on the target box, including `--smoke live --check-disposition` | Saved `--report-md` / `--report-json` (metrics only) | | 6 | Windows service install/run/uninstall proven on both Server SKUs | `ci.yml` job `windows-service-smoke` | | 7 | Container path proven | `ci.yml` job `docker-smoke`; `manifest-lint.yml` job `kubeconform` for the k8s manifests | @@ -497,7 +497,7 @@ campaign such as a box-acceptance run). | 9 | Published performance numbers re-measured or explicitly carried forward with their measurement date | `benchmark.yml` baselines + `docs/benchmarks/` | | 10 | Upgrade path proven from the previous released version on each backend (schema migration + queued-message survival) | `MIG` chapter rows | | 11 | No open S0 or S1 defect (§0.8) | BACKLOG | -| 12 | Docs that make status claims are true — FEATURE-MAP statuses, BACKLOG banners, CI-QUALITY figures | `backlog-hygiene` + [`tests/test_backlog_status_check.py`](tests/test_backlog_status_check.py) | +| 12 | Docs that make status claims are true — FEATURE-MAP statuses, BACKLOG banners, CI-QUALITY figures | `backlog-hygiene` + [`tests/test_backlog_status_check.py`](../../../tests/test_backlog_status_check.py) | **Class filter (per §0.4.4).** Criteria 2 and 3 count **`T` rows only**. **`C` rows** — recorded measurements, published numbers, dated owner decisions — are *listed* in the release record as @@ -557,15 +557,15 @@ Severity describes the **defect**; Pri (§0.4.2) describes how much the **test** ### BACKLOG and the ledger - A defect that is **not fixed immediately** becomes a numbered item in - [`docs/BACKLOG.md`](docs/BACKLOG.md) carrying its severity, the failing test ID (or the ID of the + [`docs/BACKLOG.md`](../../BACKLOG.md) carrying its severity, the failing test ID (or the ID of the test that *should* have existed), and the originating review/campaign. - **Numbers are allocated atomically — never grepped.** `pwsh -NoProfile -File scripts\coord\alloc.ps1 -Kind backlog -Title ""` (same for `-Kind adr`), with the ADR's index row added in the *same* commit. A `pre-commit` hook rejects a number that was not allocated. Rationale and the three real collisions: - [`docs/LEDGER-GATE.md`](docs/LEDGER-GATE.md). + [`docs/LEDGER-GATE.md`](../../LEDGER-GATE.md). - **Shipping an item updates its banner in the same PR.** Structural enforcement: - [`tests/test_backlog_status_check.py`](tests/test_backlog_status_check.py) (every item declares + [`tests/test_backlog_status_check.py`](../../../tests/test_backlog_status_check.py) (every item declares exactly one status). Behavioural enforcement: `.github/workflows/backlog-hygiene.yml` job `banner-on-implementation` — a PR that says `BACKLOG #N` and touches engine/IDE code must also update `docs/BACKLOG.md`. @@ -586,7 +586,7 @@ fabricated `0.00/msg`. ### Falsifier discipline (existing, in force) -[ADR 0101](docs/adr/0101-pre-registered-falsifier-discipline-for-performance-measurement.md) — +[ADR 0101](../../adr/0101-pre-registered-falsifier-discipline-for-performance-measurement.md) — *Pre-registered falsifier discipline for performance measurement* — is binding on every `PERF` row and on any claim that will inform a build decision: @@ -631,7 +631,7 @@ and `tests/test_api_health_tokenless.py` as the frozen boundary). ### The reverse rule and its guard Traceability must survive refactoring, so the **binding is asserted by a test wherever it can be**: -[`tests/test_win2025_acceptance.py`](tests/test_win2025_acceptance.py) already proves every matrix row +[`tests/test_win2025_acceptance.py`](../../../tests/test_win2025_acceptance.py) already proves every matrix row binds to a registered probe, that every referenced pytest file exists, and that no probe raises — "so the matrix can't silently rot". Part II adopts the same pattern for its own index: the chapter tables are machine-checkable (IDs unique, referenced files exist, referenced ADR/BACKLOG numbers resolve, and @@ -642,9 +642,9 @@ repo, and two classes of valid citation are deliberately not readable from it. A does not know this will flag sound evidence as broken, and a reviewer who does not know it will read "absent" as "does not exist": -- **BACKLOG items above #231.** The committed [`docs/BACKLOG.md`](docs/BACKLOG.md) is a **published +- **BACKLOG items above #231.** The committed [`docs/BACKLOG.md`](../../BACKLOG.md) is a **published baseline that stops at `## 231.`**; the programme continued past it. The file says so itself at - [`docs/BACKLOG.md:6041`](docs/BACKLOG.md) — *"the file you are reading ends at #231, while #242–#246 + [`docs/BACKLOG.md:6041`](../../BACKLOG.md) — *"the file you are reading ends at #231, while #242–#246 and their successors do not appear in it at all … their absence here is a publishing boundary, not evidence of completion."* Citations above the baseline (e.g. #233, #275, #310) are **sound evidence**; the resolver treats them as valid and never disclaims them. Where a reader may be @@ -714,6 +714,6 @@ deliverable with its own test IDs: refuse an unallocated ADR/BACKLOG number, req row in the same commit, and — the part that actually holds — the **CI backstop**, which re-runs the same rules with `--ci` against a freshly fetched `origin/main` and is *deliberately ungated* in `ci.yml`. The `pre-commit` hook alone is explicitly "a guardrail, not a security boundary" -(`git commit --no-verify` bypasses it, [`docs/LEDGER-GATE.md`](docs/LEDGER-GATE.md) §3), so the +(`git commit --no-verify` bypasses it, [`docs/LEDGER-GATE.md`](../../LEDGER-GATE.md) §3), so the assertion worth owning is the backstop, not the hook. What is out of scope is everything the gate is *not*: the allocator's ergonomics, the worktree helpers, and the ADR analysis tooling. diff --git a/docs/testing/master-test-plan/01-environments-data-and-tooling.md b/docs/testing/master-test-plan/01-environments-data-and-tooling.md index 856dc07e..df81684c 100644 --- a/docs/testing/master-test-plan/01-environments-data-and-tooling.md +++ b/docs/testing/master-test-plan/01-environments-data-and-tooling.md @@ -10,7 +10,7 @@ environment that is "must be built" is a **procurement or provisioning dependenc that need it — those dependencies are priced in §0.17. Vocabulary reminder for this table: **engine shard** = N `serve --shard` processes over one unified -store ([`__main__.py:98`](messagefoundry/__main__.py) `--shard`, [`:115`](messagefoundry/__main__.py) +store ([`__main__.py:98`](../../../messagefoundry/__main__.py) `--shard`, [`:115`](../../../messagefoundry/__main__.py) the supervisor that spawns one per engine shard); **database shard** is the shelved store-splitting axis and appears nowhere in this matrix. @@ -22,18 +22,18 @@ belong to the lab runbook — both are named with their owner in place. | ID | Environment | What it is | What it proves that nothing else can | Cost / effort to stand up | Status | |---|---|---|---|---|---| -| **E1** | Developer PC (Windows 11) + per-worktree `.venv` | The authoring box. Isolated checkout + branch + venv per parallel session via [`scripts/worktree/new.ps1`](scripts/worktree/new.ps1) (`-Sqlserver`, `-Ide`, `-NoInstall`); see [`docs/WORKTREES.md`](docs/WORKTREES.md) | Interactive debugging, the PySide6 harness GUI tabs (Send/Receive/File/Compose/Monitor), and anything needing a human in the loop. It is also the only place the *mutation* and *diff-coverage* gates were ever verified by hand ([`docs/quality-gates/HANDOFF-mutation-coverage.md`](docs/quality-gates/HANDOFF-mutation-coverage.md) §1) | None — already provisioned | **Exists** | +| **E1** | Developer PC (Windows 11) + per-worktree `.venv` | The authoring box. Isolated checkout + branch + venv per parallel session via [`scripts/worktree/new.ps1`](../../../scripts/worktree/new.ps1) (`-Sqlserver`, `-Ide`, `-NoInstall`); see [`docs/WORKTREES.md`](../../WORKTREES.md) | Interactive debugging, the PySide6 harness GUI tabs (Send/Receive/File/Compose/Monitor), and anything needing a human in the loop. It is also the only place the *mutation* and *diff-coverage* gates were ever verified by hand ([`docs/quality-gates/HANDOFF-mutation-coverage.md`](../../quality-gates/HANDOFF-mutation-coverage.md) §1) | None — already provisioned | **Exists** | | **E2** | Linux container CI (GitHub-hosted `ubuntu-latest`) | The cheap-breadth leg. Carries `ruff check`, `ruff format --check`, both `mypy --strict` passes, the ledger gate, the full `pytest` suite, the web-console suite, and every service-container leg | Volume. It is the only environment that runs on **every** PR at 1× minutes, and the only one that hosts the SQL Server / PostgreSQL service containers | $0 (hosted minutes are free on this public repo) | **Exists** | | **E3** | Windows CI legs (hosted `windows-2022`, `windows-2025`) | Two Server SKUs running the same `pytest` suite + the web-console suite; plus the nightly NSSM `windows-service-smoke` on both SKUs | Real Windows sockets, `ProactorEventLoop`, Windows service paths, and the real NSSM install → start → `/health` → MLLP → uninstall path. These are the *deployment* OSes | $0 here (2×-billed, free on a public repo) | **Exists** | -| **E4** | Self-hosted Windows Server 2025 box, all three backends | Two distinct things, deliberately not crossed: (a) the **dispatch-only SQL Server VM** behind [`selfhosted-win2025-sql.yml`](.github/workflows/selfhosted-win2025-sql.yml) (label `mefor-win2025-sql`); (b) the **acceptance box** `WIN-NAFGLU5SH1J` that [`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) owns | OS-level **ODBC Driver 18** discoverability, the NSSM **service identity**, the per-user **DPAPI** key boundary, file ACLs, Windows Firewall, Windows **port-rebind lag**, and the real-host **throughput ceiling** on real storage. WIN2025-TEST-PLAN §S0.2 is the authoritative CI-owned / box-owned split | Hardware exists; the CI runner service is **de-registered** ([`docs/CI-SELFHOSTED-RUNNER.md`](docs/CI-SELFHOSTED-RUNNER.md) — "currently runner-less") | **Partial** — workflow + label exist, no runner registered | +| **E4** | Self-hosted Windows Server 2025 box, all three backends | Two distinct things, deliberately not crossed: (a) the **dispatch-only SQL Server VM** behind [`selfhosted-win2025-sql.yml`](../../../.github/workflows/selfhosted-win2025-sql.yml) (label `mefor-win2025-sql`); (b) the **acceptance box** `WIN-NAFGLU5SH1J` that [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) owns | OS-level **ODBC Driver 18** discoverability, the NSSM **service identity**, the per-user **DPAPI** key boundary, file ACLs, Windows Firewall, Windows **port-rebind lag**, and the real-host **throughput ceiling** on real storage. WIN2025-TEST-PLAN §S0.2 is the authoritative CI-owned / box-owned split | Hardware exists; the CI runner service is **de-registered** ([`docs/CI-SELFHOSTED-RUNNER.md`](../../CI-SELFHOSTED-RUNNER.md) — "currently runner-less") | **Partial** — workflow + label exist, no runner registered | | **E5** | Domain-joined AD / federation lab (AWS) | Three EC2 boxes per `docs/releases/HANDOFF-AD-LAB-aws.md`: **A** = DC for the throwaway `mefor.lab` forest + the AD FS farm; **B** = domain-joined engine host under NSSM; **C** = domain-joined client with Chrome *and* Firefox (may be B for pass 1) | The **entire AD acceptor path**, which the handoff records as *mock-seam only* today (`kerberos_principal` is `# pragma: no cover`; every serve-path TLS/proxy assertion monkeypatches `uvicorn.run`). Kerberos SPN/EPA, gMSA logon, integrated SQL auth, OIDC/AD FS SSO, the MFA-claim gate, and the L18 username-collision privilege-escalation refusal | An EC2 booking window; the runbook insists the four items are planned **"as one window, or not at all"** | **Must be built** | | **E6** | Browser matrix host | Real browsers driving the web console at `/ui` (`messagefoundry_webconsole`, mounted same-origin via `mount_ui`) | Anything that only a browser engine can execute: the WebAuthn/passkey ceremony against a real authenticator, CSP enforcement as the browser applies it (not as the header asserts it), the session watchdog and logout affordance as rendered, and the two-browser AD FS landing assertion (AD-lab cell L7) | Needs at minimum one Windows host with Chrome + Firefox; Safari needs macOS or a cross-browser service | **Must be built** | -| **E7** | VS Code extension test host | The `ide` job in [`ci.yml`](.github/workflows/ci.yml): `ubuntu-latest` + `windows-latest`. `npm run typecheck` → `npm run compile` (esbuild) → `npm run test:unit` (mocha, vscode-free) on **both** legs; `npm test` (`@vscode/test-electron`, a real headless VS Code) on the **Windows** leg only | Extension activation + command registration in a real Extension Host, and the vscode-free model layer (engine link state, the two frozen ADR 0110 boundary allowlists, settings-scope `SEC-005` (ADR 0035's non-loopback credential guard — an IDE control ID, not a row of this plan's SEC chapter), graph/steps/HL7 models, `promoteTarget` resolution) | $0 — already wired | **Exists** | +| **E7** | VS Code extension test host | The `ide` job in [`ci.yml`](../../../.github/workflows/ci.yml): `ubuntu-latest` + `windows-latest`. `npm run typecheck` → `npm run compile` (esbuild) → `npm run test:unit` (mocha, vscode-free) on **both** legs; `npm test` (`@vscode/test-electron`, a real headless VS Code) on the **Windows** leg only | Extension activation + command registration in a real Extension Host, and the vscode-free model layer (engine link state, the two frozen ADR 0110 boundary allowlists, settings-scope `SEC-005` (ADR 0035's non-loopback credential guard — an IDE control ID, not a row of this plan's SEC chapter), graph/steps/HL7 models, `promoteTarget` resolution) | $0 — already wired | **Exists** | | **E8** | Partner / DICOM / FHIR interop endpoint set | Real peers for the non-HL7 connectors: a DICOM C-STORE SCP/SCU peer and DICOMweb STOW-RS receiver, a FHIR server for the SMART Backend Services token + `fhir_lookup` path, an SFTP/FTPS endpoint for `RemoteFile`, and an X12 trading-partner TCP peer | Wire-level interop with a foreign implementation. Everything today terminates in a loopback sink or an in-process fake. WIN2025-TEST-PLAN §S0.3 already records matrix row **`W25:D3` (RemoteFile SFTP/FTP)** as *deferred — no SFTP endpoint provisioned on the box* | Containerized peers (HAPI FHIR, a pynetdicom SCP, an SFTP container) are near-free; a real modality/PACS or an EHR sandbox is a registration + relationship cost | **Must be built** (loopback substitutes exist) | -| **E9** | Two-box HA / failover rig | Two engine hosts sharing one externalized store (SQL Server AOAG per [`docs/AOAG-DEPLOYMENT.md`](docs/AOAG-DEPLOYMENT.md), or PostgreSQL), exercising the leader lease + graph supervisor | Failover across a **machine** boundary: real network partition, real NIC/host loss, VIP behaviour, and the Windows-host recovery *time*. CI already proves the *conformance invariants* (zero acknowledged loss, per-lane FIFO, no split-brain, bounded duplicates) with two `serve` processes on one runner via `tests/test_load_failover_sqlserver.py` / `tests/test_load_failover_postgres.py` — that is a different signal | 2 VMs + a shared server DB | **Partial** — single-host two-process is in CI; two-box is not | -| **E10** | Non-production engine + production-like engine (PUB) | A pair of engines the IDE's **Stage → Promote** can target, wired to a config repo remote. Target resolution is already unit-tested pure ([`ide/src/promoteTarget.ts`](ide/src/promoteTarget.ts), `promote-target.test.ts`) | The promotion chapter's whole subject: a config change moving non-prod → prod-like through `messagefoundry check` (validate / dryrun / posture / build-check / reference-backend), a dry-run POST, then an apply — against two engines with **different derived security postures**. A single dev engine cannot show a posture-divergent promotion | 2 hosts (VMs are fine) + a git remote (a bare repo on a share suffices) | **Must be built** | -| **E11** | Cloud / Kubernetes target (ADR 0047) | The manifests under [`docker/k8s/`](docker/k8s) (`statefulset.yaml`, `ha-postgres.yaml`, `secret.example.yaml`), guided by [`docs/CLOUD-DEPLOYMENT.md`](docs/CLOUD-DEPLOYMENT.md) | That the multi-replica active-passive manifest actually elects a leader, that only the leader binds listeners, and that the L4 MLLP load balancer follows failover. Today [`manifest-lint.yml`](.github/workflows/manifest-lint.yml) proves only that the YAML **schema-validates** (kubeconform) and satisfies grep-level HA policy assertions — nothing applies it | `kind`/`k3d` in CI is ~free; a managed EKS/AKS/GKE cluster is billed and needs a cloud account | **Partial** — manifests + lint exist, no cluster | -| **E12** | Air-gapped / offline install target | A network-isolated host installing from a local wheelhouse, with the config repo as a **bare repo on a network share** — the shape [`docs/INSTALL-GUIDE.md`](docs/INSTALL-GUIDE.md) §5 names for air-gapped sites | That the engine installs, wires, and serves with **no egress**: no PyPI, no `mcr.microsoft.com`, no `packages.microsoft.com` for ODBC 18, no OCSP/CRL fetch, no update check ([ADR 0026](docs/adr/0026-off-box-egress-update-check.md)). A CI runner has egress by construction and can never prove this | One isolated VM + a mirrored wheelhouse + an offline ODBC 18 installer | **Must be built** | +| **E9** | Two-box HA / failover rig | Two engine hosts sharing one externalized store (SQL Server AOAG per [`docs/AOAG-DEPLOYMENT.md`](../../AOAG-DEPLOYMENT.md), or PostgreSQL), exercising the leader lease + graph supervisor | Failover across a **machine** boundary: real network partition, real NIC/host loss, VIP behaviour, and the Windows-host recovery *time*. CI already proves the *conformance invariants* (zero acknowledged loss, per-lane FIFO, no split-brain, bounded duplicates) with two `serve` processes on one runner via `tests/test_load_failover_sqlserver.py` / `tests/test_load_failover_postgres.py` — that is a different signal | 2 VMs + a shared server DB | **Partial** — single-host two-process is in CI; two-box is not | +| **E10** | Non-production engine + production-like engine (PUB) | A pair of engines the IDE's **Stage → Promote** can target, wired to a config repo remote. Target resolution is already unit-tested pure ([`ide/src/promoteTarget.ts`](../../../ide/src/promoteTarget.ts), `promote-target.test.ts`) | The promotion chapter's whole subject: a config change moving non-prod → prod-like through `messagefoundry check` (validate / dryrun / posture / build-check / reference-backend), a dry-run POST, then an apply — against two engines with **different derived security postures**. A single dev engine cannot show a posture-divergent promotion | 2 hosts (VMs are fine) + a git remote (a bare repo on a share suffices) | **Must be built** | +| **E11** | Cloud / Kubernetes target (ADR 0047) | The manifests under [`docker/k8s/`](../../../docker/k8s) (`statefulset.yaml`, `ha-postgres.yaml`, `secret.example.yaml`), guided by [`docs/CLOUD-DEPLOYMENT.md`](../../CLOUD-DEPLOYMENT.md) | That the multi-replica active-passive manifest actually elects a leader, that only the leader binds listeners, and that the L4 MLLP load balancer follows failover. Today [`manifest-lint.yml`](../../../.github/workflows/manifest-lint.yml) proves only that the YAML **schema-validates** (kubeconform) and satisfies grep-level HA policy assertions — nothing applies it | `kind`/`k3d` in CI is ~free; a managed EKS/AKS/GKE cluster is billed and needs a cloud account | **Partial** — manifests + lint exist, no cluster | +| **E12** | Air-gapped / offline install target | A network-isolated host installing from a local wheelhouse, with the config repo as a **bare repo on a network share** — the shape [`docs/INSTALL-GUIDE.md`](../../INSTALL-GUIDE.md) §5 names for air-gapped sites | That the engine installs, wires, and serves with **no egress**: no PyPI, no `mcr.microsoft.com`, no `packages.microsoft.com` for ODBC 18, no OCSP/CRL fetch, no update check ([ADR 0026](../../adr/0026-off-box-egress-update-check.md)). A CI runner has egress by construction and can never prove this | One isolated VM + a mirrored wheelhouse + an offline ODBC 18 installer | **Must be built** | ### Notes that change how an environment is used @@ -45,7 +45,7 @@ belong to the lab runbook — both are named with their owner in place. placeholders in anything committed — the blocking `forbidden-content` context enforces the last one. **On the runbook, precisely.** The authority for E5's test cells is `docs/security/AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18). It is **not missing** — `/docs/security/` - is deliberately git-ignored in the public repo ([`.gitignore:144`](.gitignore), alongside + is deliberately git-ignored in the public repo ([`.gitignore:144`](../../../.gitignore), alongside `docs/reviews/` at `:145` and `docs/marketing/` at `:146`, withheld as ~32 files of posture and risk-register detail that would read as an attacker roadmap). The document exists and the owner confirms it is available to whoever builds the lab; it simply is not readable from this worktree. No @@ -83,19 +83,19 @@ are provisioned, which is why the plan needs both CI service containers and the | Test gate env var | (always on) | `MEFOR_TEST_SQLSERVER=1` — 82 references across `tests/` | `MEFOR_TEST_POSTGRES=1` — 57 references across `tests/` | | Connection env | `--db <path>` | `MEFOR_STORE_BACKEND/SERVER/PORT/DATABASE/AUTH/USERNAME/PASSWORD`, `MEFOR_STORE_TRUST_SERVER_CERTIFICATE` | same `MEFOR_STORE_*` set, plus `MEFOR_STORE_ENCRYPT=false` for the plaintext container | | TLS escape needed in CI | n/a | `MEFOR_ALLOW_INSECURE_TLS=1` — the container's self-signed cert forces `trust_server_certificate=true`, which the store's TLS-hardening guard refuses without this trusted-network dev/test escape | `MEFOR_ALLOW_INSECURE_TLS=1` for the same reason (plaintext container) | -| Known live defect | — | **pyodbc 5.3.0 + py3.14 native segfault** in the C parameter-binding path against the 2025 container (upstream `mkleehammer/pyodbc#1459`, unfixed; 5.3.0 is the newest and the first with py3.14 wheels). Worked around by [`scripts/ci/retry-native-crash.sh`](scripts/ci/retry-native-crash.sh), which retries the whole step **only** on exit 139/134 and never on exit 1 — so it cannot mask a regression | — | +| Known live defect | — | **pyodbc 5.3.0 + py3.14 native segfault** in the C parameter-binding path against the 2025 container (upstream `mkleehammer/pyodbc#1459`, unfixed; 5.3.0 is the newest and the first with py3.14 wheels). Worked around by [`scripts/ci/retry-native-crash.sh`](../../../scripts/ci/retry-native-crash.sh), which retries the whole step **only** on exit 139/134 and never on exit 1 — so it cannot mask a regression | — | **Backend-specific behaviours the plan must keep distinct** (they are deliberate divergences, not bugs): SQL Server carries a response-column cipher pass and the batching path (`FCP:SCALE-9` in -[`FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) is confirmed **SS-only**); +[`FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) is confirmed **SS-only**); PostgreSQL rejects NUL bytes and uses `SELECT … FOR UPDATE SKIP LOCKED` + advisory locks where SQL Server uses pooled claim; `body_ref` is inert on both servers; and a `Reference(...)` config against a SQL Server store is refused at gate time by `messagefoundry check`'s `reference-backend` check. ### The x3 / x2 / once notation -This plan reuses the per-DB notation that [`WIN2025-TEST-MATRIX.md`](docs/testing/WIN2025-TEST-MATRIX.md) -and [`WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) §"ID scheme" already established. Do +This plan reuses the per-DB notation that [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md) +and [`WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) §"ID scheme" already established. Do not invent a second notation: | Notation | Meaning | @@ -122,11 +122,11 @@ is never redirected to a committed file, a ticket, or a CI log. ### 0.13.1 Synthetic generators — the primary source -[`messagefoundry/generators/`](messagefoundry/generators) emits **conformant** HL7 v2.x across the +[`messagefoundry/generators/`](../../../messagefoundry/generators) emits **conformant** HL7 v2.x across the message families the engine handles: `adt`, `oru`, `orm`, `oml`, `orl`, `mdm`, `mfn`, `dft`, `bar`, `ras`, `rde`, `siu`, `vxu`, `documents`, with `all_types.py` as the registry. Two entry points: -- `messagefoundry generate` — the CLI subcommand ([`__main__.py:349`](messagefoundry/__main__.py)). +- `messagefoundry generate` — the CLI subcommand ([`__main__.py:349`](../../../messagefoundry/__main__.py)). - `python -m messagefoundry.generators.adt [--triggers A01,A04] [--count N] [--out DIR]` — the ADT corpus builder: **57 triggers across 25 message structures** (A01–A62 excluding the A19 query event and reserved A56–A59), with segment order and the allowed segment set driven by **hl7apy's own @@ -140,9 +140,9 @@ disposable (§0.13.3) and why the load corpus can only ever emit valid messages | Corpus | Contents | Why committed | |---|---|---| -| [`samples/messages/`](samples/messages) | `adt_a01.hl7`, `adt_batch.hl7`, `x12_270_eligibility.edi` | Fixed inputs for the MLLP sender, the docker smoke, and the X12 path | -| [`samples/messages/hapi-hl7v2/`](samples/messages/hapi-hl7v2) | 7 files + README: ADT^A01 (2.4), ADT^A03 (2.5), OMD^O03 ×2, OML^O21 (2.5.1), a 2.3.1 Z-event ERP^Z99, and an 18-message concatenated batch spanning 2.1–2.4 | **Type and version diversity a generator will not produce.** Vendored verbatim from `hapifhir/hapi-hl7v2` at commit `de1503651040` under **MPL-2.0**; the README carries the full per-file provenance manifest. If any file is ever modified, MPL-2.0 requires that file to carry its source notice | -| [`samples/dicom/generate_sr_sample.py`](samples/dicom/generate_sr_sample.py) | A generator, not a committed binary | Keeps DICOM sample data regenerable and out of git | +| [`samples/messages/`](../../../samples/messages) | `adt_a01.hl7`, `adt_batch.hl7`, `x12_270_eligibility.edi` | Fixed inputs for the MLLP sender, the docker smoke, and the X12 path | +| [`samples/messages/hapi-hl7v2/`](../../../samples/messages/hapi-hl7v2) | 7 files + README: ADT^A01 (2.4), ADT^A03 (2.5), OMD^O03 ×2, OML^O21 (2.5.1), a 2.3.1 Z-event ERP^Z99, and an 18-message concatenated batch spanning 2.1–2.4 | **Type and version diversity a generator will not produce.** Vendored verbatim from `hapifhir/hapi-hl7v2` at commit `de1503651040` under **MPL-2.0**; the README carries the full per-file provenance manifest. If any file is ever modified, MPL-2.0 requires that file to carry its source notice | +| [`samples/dicom/generate_sr_sample.py`](../../../samples/dicom/generate_sr_sample.py) | A generator, not a committed binary | Keeps DICOM sample data regenerable and out of git | ### 0.13.3 The git-ignored corpus @@ -173,7 +173,7 @@ What exists: - **Harness Receive tab fault injection** — `delay then AA` (past the engine timeout, to force a retry), `close (no reply)`, and `fail N then AA` — driving outbound retry / dead-letter / independent draining. -- **[`harness/load/profiles/malformed-load.toml`](harness/load/profiles/malformed-load.toml)** — the +- **[`harness/load/profiles/malformed-load.toml`](../../../harness/load/profiles/malformed-load.toml)** — the robustness-under-load profile. Read its header carefully before planning against it: it carries **well-formed background throughput only**; the malformed / oversized / torn-frame inputs are injected **concurrently from the GUI** while the sustained phase runs. Bad input *cannot* be a @@ -185,7 +185,7 @@ What exists: neutral infrastructure. The PySide6 harness GUI (`harness/`) is simultaneously (a) this plan's *only* route to hostile-input and outbound-fault injection and (b) a **shipped distribution** — the separate `messagefoundry-harness` wheel (`packaging/messagefoundry-harness/`), built and version-checked in -lockstep with the engine by the `release-harness` job ([`release.yml:477`](.github/workflows/release.yml), +lockstep with the engine by the `release-harness` job ([`release.yml:477`](../../../.github/workflows/release.yml), PyPI publish gated on the `PUBLISH_HARNESS` repo variable). **The TRAY chapter (§13d) owns testing it**, and the specific affordances this section leans on are rows there, not assumptions here: the Compose-tab presets (`harness/compose.py:74-77` — "No MSH segment", "Bad version (2.3)") and the Receive-tab fault @@ -200,9 +200,9 @@ that is a tooling item (§0.14.2), not a data item. ### 0.13.6 De-identification — the only sanctioned path from real traffic -[ADR 0030](docs/adr/0030-anonymization-test-harness-tee.md) built -[`messagefoundry/anon/`](messagefoundry/anon) (`hl7.py` / `keying.py` / `rules.py` / `surrogates.py` / -`leak.py`), vendored **byte-identical** to [`tee/anon/`](tee/anon) so the dependency-free tee carries +[ADR 0030](../../adr/0030-anonymization-test-harness-tee.md) built +[`messagefoundry/anon/`](../../../messagefoundry/anon) (`hl7.py` / `keying.py` / `rules.py` / `surrogates.py` / +`leak.py`), vendored **byte-identical** to [`tee/anon/`](../../../tee/anon) so the dependency-free tee carries the same logic. Its contract: - **Two-layer rule model.** Field *selection* is data (`load_rules` over an optional `anon.toml`); @@ -215,7 +215,7 @@ the same logic. Its contract: and **nothing is written**. The exception carries token *categories* only (e.g. `"partner/site token"`), never the offending value — so raising or logging it cannot itself leak PHI. - **The only surface.** `python -m tee anonymize-captures --db <tee.db> --out <file.jsonl> - [--direction corepoint_copy]` ([`tee/__main__.py:286`](tee/__main__.py)) is the sanctioned way to + [--direction corepoint_copy]` ([`tee/__main__.py:286`](../../../tee/__main__.py)) is the sanctioned way to turn captured live traffic into a shareable dataset — plus the harness hooks. There is no other approved route, and a hand-written scrub script is a defect. @@ -256,8 +256,8 @@ deliberately fail-closed: `bootstrap-admin.txt`. Two CI contexts back this up: **`forbidden-content (customer/PHI leak guard)`** -([`security.yml:367`](.github/workflows/security.yml), running -[`scripts/security/scan_forbidden.py --path .`](scripts/security/scan_forbidden.py)) is **blocking and +([`security.yml:367`](../../../.github/workflows/security.yml), running +[`scripts/security/scan_forbidden.py --path .`](../../../scripts/security/scan_forbidden.py)) is **blocking and required**, and **`gitleaks`** is blocking. A committed real token list, a routable IP, a real hostname or a message body turns the build red — which is why the AD-lab handoff makes scrubbing the L17 run record an explicit sub-step rather than a formality. @@ -277,10 +277,10 @@ record an explicit sub-step rather than a formality. | Flake retry | In-run only | `pytest-rerunfailures>=16.0` | E2, E3 | Self-heals the known harness-monitor timing flake. It **records nothing** — see §0.14.2 | | **ruff** | Lint + format, **whole repo** | `ruff check .` / `ruff format --check .` | E2 only (platform-independent) | Scope lives in **one place** — `[tool.ruff] extend-exclude` — and `tests/test_lint_scope_parity.py` fails if the hook and CI drift apart. Pinned `>=0.4,<0.16`: 0.16 turns on RUF022/RUF100/BLE001, ~525 findings | | **mypy (strict)** | Types, both platforms | `mypy messagefoundry messagefoundry_webconsole --exclude 'messagefoundry/tray/'` **and** `mypy --platform win32 messagefoundry` | E2 only | Two passes on Linux so `sys.platform=='win32'` branches (ctypes/DPAPI/service_control/tray) are typed without paying for a Windows mypy | -| Mutation + diff-coverage | Advisory quality signals 6 + 7 | Built in `quality-advisory.yml` (mutmut 3; 2.5.1 crashed on py3.14 and `\|\| true` hid it for months). Local recipe in [`docs/quality-gates/HANDOFF-mutation-coverage.md`](docs/quality-gates/HANDOFF-mutation-coverage.md) | E2 (+ E1 to verify) | `pytest-cov`, `diff-cover`, `mutmut` are **CI-only** installs — adding them to `pyproject.toml` without re-running `uv lock`/`uv export` reds the DEP-1 gate | +| Mutation + diff-coverage | Advisory quality signals 6 + 7 | Built in `quality-advisory.yml` (mutmut 3; 2.5.1 crashed on py3.14 and `\|\| true` hid it for months). Local recipe in [`docs/quality-gates/HANDOFF-mutation-coverage.md`](../../quality-gates/HANDOFF-mutation-coverage.md) | E2 (+ E1 to verify) | `pytest-cov`, `diff-cover`, `mutmut` are **CI-only** installs — adding them to `pyproject.toml` without re-running `uv lock`/`uv export` reds the DEP-1 gate | | **ide** mocha + electron | VS Code extension | `npm run test:unit` (mocha, `--ui tdd`, vscode-free) on both legs; `npm test` → `out/test/runTest.js` (`@vscode/test-electron`) on Windows | E7 | 35 `*.test.ts` modules under `ide/src/test/suite/`. The unit split exists because `npm test` is Windows-only — without it the whole node estate was type-checked and never executed on ubuntu | -| **harness GUI (PySide6)** | The five-tab operator-driven test instrument: Send / Compose / Receive / File / Monitor | `python -m harness` (a separate process; reaches the engine only over the HTTP API via `apiclient/`) | E1, and any host with a display | **Both an instrument and a product.** It is the plan's only route to hostile-input injection (Compose presets) and outbound-fault injection (Receive `REPLY_MODES`), *and* it ships as the separate `messagefoundry-harness` wheel (`packaging/messagefoundry-harness/`, `release-harness` in [`release.yml:477`](.github/workflows/release.yml), publish gated on `PUBLISH_HARNESS`). **Testing it is the TRAY chapter's job (§13d)** — this chapter only consumes it. It needs a human at a display, so nothing it produces is CI-repeatable (§0.13.5) | -| **harness/load** | Throughput, latency, loss, drain, connection scale, engine-shard fan-in | `python -m harness --load <profile\|path> --engine URL [--token T] --sink-port N --report-json --report-csv [--baseline --tolerance] [--db-backend LABEL] [--shard-engine …]`; `--list-profiles` | E2 (smoke), E4 (ceiling) | Three measurement channels (sender / correlation sink / engine poller) answering three different questions — [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md). Never imports the engine; never touches the store | +| **harness GUI (PySide6)** | The five-tab operator-driven test instrument: Send / Compose / Receive / File / Monitor | `python -m harness` (a separate process; reaches the engine only over the HTTP API via `apiclient/`) | E1, and any host with a display | **Both an instrument and a product.** It is the plan's only route to hostile-input injection (Compose presets) and outbound-fault injection (Receive `REPLY_MODES`), *and* it ships as the separate `messagefoundry-harness` wheel (`packaging/messagefoundry-harness/`, `release-harness` in [`release.yml:477`](../../../.github/workflows/release.yml), publish gated on `PUBLISH_HARNESS`). **Testing it is the TRAY chapter's job (§13d)** — this chapter only consumes it. It needs a human at a display, so nothing it produces is CI-repeatable (§0.13.5) | +| **harness/load** | Throughput, latency, loss, drain, connection scale, engine-shard fan-in | `python -m harness --load <profile\|path> --engine URL [--token T] --sink-port N --report-json --report-csv [--baseline --tolerance] [--db-backend LABEL] [--shard-engine …]`; `--list-profiles` | E2 (smoke), E4 (ceiling) | Three measurement channels (sender / correlation sink / engine poller) answering three different questions — [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md). Never imports the engine; never touches the store | | harness failover / connscale / estate | HA, 500–1500 connections, multi-engine estate | `python -m harness --failover …`, `--connscale …`, `--estate …` (each with its own `--list-*-profiles`) | E2 (in `tests/test_load_failover_*`), E4, E9 | The failover orchestrator **spawns** two `serve` nodes and binds ports, so it must run *on* the host under test with a cwd containing `harness/config/load` | | **harness/acceptance** | The 54-row WIN2025 matrix runner | `python -m harness.acceptance [--report-md] [--report-csv] [--xlsx]` | E4 | **Dev-tree artifact** — it shells out to `python -m pytest` at the repo root and reads source by repo-relative path, so the box needs a full source checkout pinned to the deployed wheel version, plus `openpyxl` for `--xlsx` | | **harness/reconcile** | Parallel-run output parity (the Corepoint-cutover tool) | `python -m harness.reconcile capture --out <jsonl> [--host --port]`; `… compare --mefor … --corepoint … [--ignore-segment] [--report-json]` | E1, E4 | Exercised on a synthetic golden pair; the real use is a cutover shadow phase | @@ -326,7 +326,7 @@ Sixteen workflows. This is the complete list; the plan cites these rather than r | Workflow | Legs / jobs | Trigger | Environment | Gates? | Role in this plan | |---|---|---|---|---|---| -| [`ci.yml`](.github/workflows/ci.yml) · `test` | `ubuntu-latest`, `windows-2022`, `windows-2025` — all py3.14. Ledger gate (ungated, Linux); ruff + both mypy passes (Linux only); pytest; web-console pytest | PR, push→main, dispatch. **Not** the nightly cron | E2 + E3 | **Required** ×3 contexts | The functional spine. Docs-only PRs short-circuit the expensive steps via `changes.code` while the required context still reports green | +| [`ci.yml`](../../../.github/workflows/ci.yml) · `test` | `ubuntu-latest`, `windows-2022`, `windows-2025` — all py3.14. Ledger gate (ungated, Linux); ruff + both mypy passes (Linux only); pytest; web-console pytest | PR, push→main, dispatch. **Not** the nightly cron | E2 + E3 | **Required** ×3 contexts | The functional spine. Docs-only PRs short-circuit the expensive steps via `changes.code` while the required context still reports green | | `ci.yml` · `changes` | Path filters → `serverdb`, `docker`, `code`, `ide`, plus the per-repo `matrix` / `ide_matrix` JSON | every `ci.yml` run | E2 | no | Decides which heavy legs run. On a **fork** it emits the ubuntu-only matrices | | `ci.yml` · `ide` | build + typecheck + `test:unit` on ubuntu **and** windows-latest; `@vscode/test-electron` on Windows | dispatch, or PR touching `ide/**` / this workflow | E7 | no | The VS Code extension chapter's CI home | | `ci.yml` · `sqlserver-store` | SQL Server **2022** and **2025** containers. Steps: store suite · coordinator (leader election) · failover (real TTL takeover, 2-node) · DATABASE-connector round-trip · **failover-LOAD** (SIGKILL the primary mid-load) · 10 throughput-lever invariant files · X12 RTE capture/re-ingress | nightly `17 3 * * *`, dispatch, or a PR touching the server-DB surface | E2 + containers | rolled up by `CI gate` | The **x2** server-backend leg for SQL Server. Every step is wrapped in `retry-native-crash.sh` | @@ -336,23 +336,23 @@ Sixteen workflows. This is the complete list; the plan cites these rather than r | `ci.yml` · `windows-service-smoke` | `windows-2022` + `windows-2025`. NSSM install (virtual account `NT SERVICE\MessageFoundry`, `-LockConfigDir`, `--env prod`) → start → `/health` → MLLP → `/messages` → stop → uninstall | nightly + dispatch, this repo only | E3 | rolled up | The only automated proof of the real Windows service path. It runs `prod`, so it also proves the fail-closed prod guards (store key, bounded PHI retention, deny-by-default egress) | | `ci.yml` · `docker-smoke` | Builds slim + `runtime-sqlserver` + the baked smoke image; asserts an ADT reaches **PROCESSED** (not merely RECEIVED); verifies graceful `docker stop` (tini → SIGTERM → lifespan) | nightly, dispatch, or a PR touching image/locks/packaging | E2 | **not** in `CI gate` | The container-runtime leg and the seed for E11 | | `ci.yml` · `ci-gate` | `if: always()`; fails iff a gated leg failed or was cancelled — a **skipped** leg passes | every run | E2 | **Required** | The one stable required context standing in for the conditional/matrix legs, which report unexpanded names when skipped | -| [`security.yml`](.github/workflows/security.yml) | `pip-audit` (+ DEP-1 lock-sync), `npm-audit`, `sbom`, `trivy`, `bandit`, `gitleaks`, `semgrep`, `crypto-inventory`, `forbidden-content` | PR, push→main, daily `0 6 * * *`, dispatch | E2 | **Required:** `bandit`, `pip-audit`, `forbidden-content` | The PHI/customer leak guard and the supply-chain floor. The daily cron bounds CVE exposure on an unchanged tree at ~24 h | -| [`codeql.yml`](.github/workflows/codeql.yml) | python + javascript-typescript | push→main, PR, weekly `0 7 * * 1`, dispatch | E2 | advisory | Taint/data-flow SAST — untrusted HL7/config reaching a sink across function boundaries | -| [`scorecard.yml`](.github/workflows/scorecard.yml) | OpenSSF Scorecard → SARIF + public badge | `branch_protection_rule`, weekly `0 8 * * 1`, push→main, dispatch | E2 | advisory | Supply-chain hygiene regression detector | -| [`zizmor.yml`](.github/workflows/zizmor.yml) | Actions static analysis (`zizmor==1.5.2`) | PR touching `.github/**`, daily `0 6 * * *`, dispatch | E2 | **Blocking** (not in required set) | Guards the CI substrate itself: template injection, over-broad tokens, dangerous triggers | -| [`quality-advisory.yml`](.github/workflows/quality-advisory.yml) | `complexity` (ruff C901 **delta**), `clone` (jscpd), `coverage` (diff-cover, PR-only), `mutation` (mutmut 3), and a `liveness` job that **can** go red when a gate stops measuring | PR, dispatch, nightly `23 4 * * *` | E2 | **Never** required; `tests/test_quality_advisory_invariants.py` pins that | The quality-measurement chapter's home. Deliberately uses workflow-command annotations, **not** SARIF — the reasons are measured and recorded in its header | -| [`benchmark.yml`](.github/workflows/benchmark.yml) | Reference sustainable-rate steps per backend + the active-passive failover profile; metrics-only JSON uploaded | **dispatch only** | E2 + containers | no | Produces the numbers transcribed into `docs/benchmarks/TUNING-BASELINE.md` | -| [`selfhosted-win2025-sql.yml`](.github/workflows/selfhosted-win2025-sql.yml) | The SQL Server store / coordinator / failover / connector suites on **real** Windows Server 2025 + real ODBC 18 | **dispatch only**, label `[self-hosted, windows, mefor-win2025-sql]`; concurrency queues (one shared DB) | E4 | no, and never | The one production-shaped combination hosted runners cannot reach. Dispatch-only *is* the security control — no fork code ever reaches the runner | -| [`freethread-smoke.yml`](.github/workflows/freethread-smoke.yml) | Install + import + smoke subset on cp314t (free-threaded) | weekly `0 6 * * 1`, dispatch | E2 | must never be required | Informational tripwire for [`docs/design/freethread.md`](docs/design/freethread.md). Its header records why belt-and-braces `continue-on-error` made the canary *incapable of reporting a problem* — a useful precedent for every advisory gate in this plan | -| [`manifest-lint.yml`](.github/workflows/manifest-lint.yml) | kubeconform + ADR-0047 HA-policy grep assertions on `docker/k8s/*.yaml` | push/PR touching manifests or `docker/README.md`, dispatch | E2 | additive, not required | The only automated signal on E11 today — and it is schema/policy lint, **not** an applied deployment | -| [`backlog-hygiene.yml`](.github/workflows/backlog-hygiene.yml) | A PR claiming `BACKLOG #N` and touching engine/IDE code must also update `docs/BACKLOG.md` | PR→main | E2 | not required | Keeps the status ledger honest. The structural half rides `tests/test_backlog_status_check.py` in the `test` matrix | -| [`cla.yml`](.github/workflows/cla.yml) | CLA Assistant, signatures on the `cla-signatures` branch | `issue_comment`, `pull_request_target` (opened/synchronize) | E2 | **Required** (`CLA Assistant`) | Contribution gate | -| [`dependabot-auto-merge.yml`](.github/workflows/dependabot-auto-merge.yml) | Scoped auto-merge: patches + dev-only minors, each held unless EVERY named dependency sits on its ecosystem's **allow-set** row (hold-unless-named, whole-group denial), plus a published-GHSA gate and a release-age gate on the security track — all **failing closed** | `pull_request` | E2 | n/a | Why the required-check set is load-bearing: it is the only thing standing between a dependency bump and `main` | -| [`dependabot-lock-resync.yml`](.github/workflows/dependabot-lock-resync.yml) | Re-exports the four lock artifacts onto the Dependabot branch | PR touching `uv.lock` / `pyproject.toml` | E2 | n/a | Must stay in lockstep with security.yml's DEP-1 step — a file the gate diffs but this job does not export is un-fixable by the bot | -| [`vuln-metrics.yml`](.github/workflows/vuln-metrics.yml) | NIST SSDF RV.2 KPIs from real Dependabot PRs + CISA KEV + FIRST EPSS | weekly `0 8 * * 1`, dispatch | E2 | no | Evidence artifact, not a detector | -| [`release.yml`](.github/workflows/release.yml) | Build + SBOM + Sigstore sign + GitHub release + PyPI Trusted Publishing; separate `release-harness` (gated on the `PUBLISH_HARNESS` repo variable) and a `webconsole-v*` tag namespace | tag `v*` / `webconsole-v*`, dispatch (dry-run: builds/signs, never publishes) | E2 | n/a | The release chapter's substrate | - -**Required contexts on `main`** (per [`docs/CI.md`](docs/CI.md); branch protection is the source of +| [`security.yml`](../../../.github/workflows/security.yml) | `pip-audit` (+ DEP-1 lock-sync), `npm-audit`, `sbom`, `trivy`, `bandit`, `gitleaks`, `semgrep`, `crypto-inventory`, `forbidden-content` | PR, push→main, daily `0 6 * * *`, dispatch | E2 | **Required:** `bandit`, `pip-audit`, `forbidden-content` | The PHI/customer leak guard and the supply-chain floor. The daily cron bounds CVE exposure on an unchanged tree at ~24 h | +| [`codeql.yml`](../../../.github/workflows/codeql.yml) | python + javascript-typescript | push→main, PR, weekly `0 7 * * 1`, dispatch | E2 | advisory | Taint/data-flow SAST — untrusted HL7/config reaching a sink across function boundaries | +| [`scorecard.yml`](../../../.github/workflows/scorecard.yml) | OpenSSF Scorecard → SARIF + public badge | `branch_protection_rule`, weekly `0 8 * * 1`, push→main, dispatch | E2 | advisory | Supply-chain hygiene regression detector | +| [`zizmor.yml`](../../../.github/workflows/zizmor.yml) | Actions static analysis (`zizmor==1.5.2`) | PR touching `.github/**`, daily `0 6 * * *`, dispatch | E2 | **Blocking** (not in required set) | Guards the CI substrate itself: template injection, over-broad tokens, dangerous triggers | +| [`quality-advisory.yml`](../../../.github/workflows/quality-advisory.yml) | `complexity` (ruff C901 **delta**), `clone` (jscpd), `coverage` (diff-cover, PR-only), `mutation` (mutmut 3), and a `liveness` job that **can** go red when a gate stops measuring | PR, dispatch, nightly `23 4 * * *` | E2 | **Never** required; `tests/test_quality_advisory_invariants.py` pins that | The quality-measurement chapter's home. Deliberately uses workflow-command annotations, **not** SARIF — the reasons are measured and recorded in its header | +| [`benchmark.yml`](../../../.github/workflows/benchmark.yml) | Reference sustainable-rate steps per backend + the active-passive failover profile; metrics-only JSON uploaded | **dispatch only** | E2 + containers | no | Produces the numbers transcribed into `docs/benchmarks/TUNING-BASELINE.md` | +| [`selfhosted-win2025-sql.yml`](../../../.github/workflows/selfhosted-win2025-sql.yml) | The SQL Server store / coordinator / failover / connector suites on **real** Windows Server 2025 + real ODBC 18 | **dispatch only**, label `[self-hosted, windows, mefor-win2025-sql]`; concurrency queues (one shared DB) | E4 | no, and never | The one production-shaped combination hosted runners cannot reach. Dispatch-only *is* the security control — no fork code ever reaches the runner | +| [`freethread-smoke.yml`](../../../.github/workflows/freethread-smoke.yml) | Install + import + smoke subset on cp314t (free-threaded) | weekly `0 6 * * 1`, dispatch | E2 | must never be required | Informational tripwire for [`docs/design/freethread.md`](../../design/freethread.md). Its header records why belt-and-braces `continue-on-error` made the canary *incapable of reporting a problem* — a useful precedent for every advisory gate in this plan | +| [`manifest-lint.yml`](../../../.github/workflows/manifest-lint.yml) | kubeconform + ADR-0047 HA-policy grep assertions on `docker/k8s/*.yaml` | push/PR touching manifests or `docker/README.md`, dispatch | E2 | additive, not required | The only automated signal on E11 today — and it is schema/policy lint, **not** an applied deployment | +| [`backlog-hygiene.yml`](../../../.github/workflows/backlog-hygiene.yml) | A PR claiming `BACKLOG #N` and touching engine/IDE code must also update `docs/BACKLOG.md` | PR→main | E2 | not required | Keeps the status ledger honest. The structural half rides `tests/test_backlog_status_check.py` in the `test` matrix | +| [`cla.yml`](../../../.github/workflows/cla.yml) | CLA Assistant, signatures on the `cla-signatures` branch | `issue_comment`, `pull_request_target` (opened/synchronize) | E2 | **Required** (`CLA Assistant`) | Contribution gate | +| [`dependabot-auto-merge.yml`](../../../.github/workflows/dependabot-auto-merge.yml) | Scoped auto-merge: patches + dev-only minors, each held unless EVERY named dependency sits on its ecosystem's **allow-set** row (hold-unless-named, whole-group denial), plus a published-GHSA gate and a release-age gate on the security track — all **failing closed** | `pull_request` | E2 | n/a | Why the required-check set is load-bearing: it is the only thing standing between a dependency bump and `main` | +| [`dependabot-lock-resync.yml`](../../../.github/workflows/dependabot-lock-resync.yml) | Re-exports the four lock artifacts onto the Dependabot branch | PR touching `uv.lock` / `pyproject.toml` | E2 | n/a | Must stay in lockstep with security.yml's DEP-1 step — a file the gate diffs but this job does not export is un-fixable by the bot | +| [`vuln-metrics.yml`](../../../.github/workflows/vuln-metrics.yml) | NIST SSDF RV.2 KPIs from real Dependabot PRs + CISA KEV + FIRST EPSS | weekly `0 8 * * 1`, dispatch | E2 | no | Evidence artifact, not a detector | +| [`release.yml`](../../../.github/workflows/release.yml) | Build + SBOM + Sigstore sign + GitHub release + PyPI Trusted Publishing; separate `release-harness` (gated on the `PUBLISH_HARNESS` repo variable) and a `webconsole-v*` tag namespace | tag `v*` / `webconsole-v*`, dispatch (dry-run: builds/signs, never publishes) | E2 | n/a | The release chapter's substrate | + +**Required contexts on `main`** (per [`docs/CI.md`](../../CI.md); branch protection is the source of truth): `CI gate`, `test (ubuntu-latest, py3.14)`, `test (windows-2022, py3.14)`, `test (windows-2025, py3.14)`, `bandit (Python SAST)`, `pip-audit (dependency vulnerabilities)`, `forbidden-content (customer/PHI leak guard)`, `CLA Assistant`. @@ -411,7 +411,7 @@ end-to-end without. **Lead time**, not price, is the binding constraint on most | **C3** | SQL Server licensing | Two majors under test (2022, 2025) | Container images are free to pull for CI; on-box use is **Developer Edition** (free, non-production only). A production-licensed instance is a Phase-2 customer-side item | none for test | nothing in Phase 1 | | **C4** | Windows Server licences / evals | E3 is hosted (free); E4, E5-B/C, E9, E10 are self-provisioned | Eval or existing licences | days | E4/E5/E9/E10 | | **C5** | **AD / federation lab (AWS)** | Boxes A (DC + AD FS), B (engine, domain-joined, NSSM), C (Chrome + Firefox client; may be B for pass 1). Optional pass-2: AD CS, IIS+ARR mTLS front, an Entra tenant | Hourly EC2 for three Windows instances **plus EBS** (artifacts must be written off instance-store — a STOP/START wipes it). Owner approval required before stopping or terminating anything | **book one window**; the runbook is explicit that the four items are planned *as one window, or not at all* | every federation / Kerberos / EPA / gMSA / OIDC row | -| **C6** | **Stand the AD lab up** (the forest itself) | `docs/security/AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18) is the named authority; it is **withheld from the public repo**, not missing (`/docs/security/` is git-ignored at [`.gitignore:144`](.gitignore)) and the owner confirms it is available to whoever builds the lab. The countable work is the build itself: DC promoted for `mefor.lab`, test accounts + groups created (incl. the L18 collision pair), the engine **SPN registered and a keytab issued**, and a **reachable LDAPS endpoint** | Owner/admin time inside the C5 window; no purchase | hours-to-days **inside** the booked C5 window | every federation / Kerberos / EPA / gMSA / OIDC row (jointly with C5) | +| **C6** | **Stand the AD lab up** (the forest itself) | `docs/security/AD-FEDERATION-LAB-RUNBOOK.md` (cells L0–L18) is the named authority; it is **withheld from the public repo**, not missing (`/docs/security/` is git-ignored at [`.gitignore:144`](../../../.gitignore)) and the owner confirms it is available to whoever builds the lab. The countable work is the build itself: DC promoted for `mefor.lab`, test accounts + groups created (incl. the L18 collision pair), the engine **SPN registered and a keytab issued**, and a **reachable LDAPS endpoint** | Owner/admin time inside the C5 window; no purchase | hours-to-days **inside** the booked C5 window | every federation / Kerberos / EPA / gMSA / OIDC row (jointly with C5) | | **C7** | Browser matrix host(s) | Real Chrome + Firefox on Windows (AD-lab cell L7 needs two browsers on a domain-joined client); Safari needs macOS | One Windows VM covers Chrome+Firefox. Safari = a Mac or a paid cross-browser service. A hardware security key for the passkey path is a small one-off | days (Windows) / weeks (Mac or a service contract) | the web-console browser chapter, WebAuthn end-to-end | | **C8** | Interop endpoints | DICOM SCP/SCU peer, DICOMweb STOW-RS receiver, FHIR server, **SFTP/FTPS** (matrix row `W25:D3` is deferred today for exactly this reason), X12 partner TCP | Containerized peers ≈ free and should be the default. A real modality/PACS or EHR sandbox is a **relationship + registration** cost, not a purchase | days (containers) / weeks–months (real peers) | the interop chapter's "foreign implementation" rows | | **C9** | Two-box HA rig (E9) | Machine-boundary failover, VIP behaviour, Windows recovery *time* | 2 VMs + a shared server DB (can reuse C2/C4 capacity) | days | the HA chapter's topology rows | diff --git a/docs/testing/master-test-plan/02-pipeline-reliability.md b/docs/testing/master-test-plan/02-pipeline-reliability.md index f6c2cec0..3b356b3b 100644 --- a/docs/testing/master-test-plan/02-pipeline-reliability.md +++ b/docs/testing/master-test-plan/02-pipeline-reliability.md @@ -16,20 +16,20 @@ destination. In scope: - The **3 + 1 stage** queue — `ingress → routed → outbound`, plus the optional `Stage.RESPONSE` - re-ingress token ([`store/store.py:330-357`](../../messagefoundry/store/store.py)). + re-ingress token ([`store/store.py:330-357`](../../../messagefoundry/store/store.py)). - **ACK-on-receipt**: AA is built only after `enqueue_ingress` durably commits - ([`pipeline/wiring_runner.py:3717-3745`](../../messagefoundry/pipeline/wiring_runner.py)), and the + ([`pipeline/wiring_runner.py:3717-3745`](../../../messagefoundry/pipeline/wiring_runner.py)), and the four pre-ACK failure classes that still **NAK synchronously** (decode `AR` :3497, NUL `AR` :3523, strict-validate `AE` :3660, streaming-detach `AE` :3699). - **Transactional stage handoff** — `route_handoff` / `transform_handoff` / the Step-A combined - `handoff` / `ingress_handoff` ([`store/base.py:289-366`, `:703-726`](../../messagefoundry/store/base.py)) — + `handoff` / `ingress_handoff` ([`store/base.py:289-366`, `:703-726`](../../../messagefoundry/store/base.py)) — and the at-least-once re-run invariant that rests on them. - **Crash recovery**: `reset_stale_inflight` across every stage, and its ownership-scoped form (`OwnedLanes`, ADR 0073) for **engine shards** over one unified store - ([`pipeline/engine.py:809-874`](../../messagefoundry/pipeline/engine.py)). + ([`pipeline/engine.py:809-874`](../../../messagefoundry/pipeline/engine.py)). - The **disposition finalizer as sole authority** and the seven-member `MessageStatus` set (`RECEIVED / ROUTED / UNROUTED / PROCESSED / FILTERED / ERROR / NOT_DEPLOYED`, - [`store/store.py:311-319`](../../messagefoundry/store/store.py)). + [`store/store.py:311-319`](../../../messagefoundry/store/store.py)). - **Seq-only per-lane FIFO** (ADR 0059), the claim family (`claim_next_fifo`, `claim_next_fifo_batch` ADR 0058, `claim_fifo_heads` + `list_fifo_lanes` ADR 0066, `release_claimed` / `reschedule_claimed` ADR 0070), pooled `StageDispatcher` (**the shipped diff --git a/docs/testing/master-test-plan/03-store-and-data-lifecycle.md b/docs/testing/master-test-plan/03-store-and-data-lifecycle.md index cb324211..87d33db0 100644 --- a/docs/testing/master-test-plan/03-store-and-data-lifecycle.md +++ b/docs/testing/master-test-plan/03-store-and-data-lifecycle.md @@ -14,20 +14,20 @@ stops deleting, is green in CI on the production backend. **In scope.** -- The **Store protocol + `open_store` seam** ([store/base.py:1739](messagefoundry/store/base.py)) over +- The **Store protocol + `open_store` seam** ([store/base.py:1739](../../../messagefoundry/store/base.py)) over **SQLite / PostgreSQL / SQL Server**, including `build_store_cipher` - ([base.py:1717](messagefoundry/store/base.py)) and the backend capability flags - ([base.py:179–218](messagefoundry/store/base.py)). + ([base.py:1717](../../../messagefoundry/store/base.py)) and the backend capability flags + ([base.py:179–218](../../../messagefoundry/store/base.py)). - **Schema init + migration**: the ADR 0064 content-hash `schema_meta` fast-path - ([postgres.py:606](messagefoundry/store/postgres.py), [sqlserver.py:1358](messagefoundry/store/sqlserver.py)), - Postgres `_MIGRATION_REV` ([postgres.py:603](messagefoundry/store/postgres.py)), and SQLite's 17 - guarded `ADD COLUMN` on-open migrations ([store.py:2932](messagefoundry/store/store.py)). -- **Encryption at rest**: AES-256-GCM `mfenc:v1`/`v2` ([store/crypto.py](messagefoundry/store/crypto.py)), - cell-AAD binding (`[store].aad_bind`, default **True**, [settings.py:381](messagefoundry/config/settings.py)), + ([postgres.py:606](../../../messagefoundry/store/postgres.py), [sqlserver.py:1358](../../../messagefoundry/store/sqlserver.py)), + Postgres `_MIGRATION_REV` ([postgres.py:603](../../../messagefoundry/store/postgres.py)), and SQLite's 17 + guarded `ADD COLUMN` on-open migrations ([store.py:2932](../../../messagefoundry/store/store.py)). +- **Encryption at rest**: AES-256-GCM `mfenc:v1`/`v2` ([store/crypto.py](../../../messagefoundry/store/crypto.py)), + cell-AAD binding (`[store].aad_bind`, default **True**, [settings.py:381](../../../messagefoundry/config/settings.py)), DEK rotation (`reencrypt_to_active`), the KeyProvider seam, and the Vault/OpenBao Transit cipher (ADR 0138, `mfenc:v3`) **only where they touch store columns, purge write-back, or rotation**. - **Retention / purge / maintenance — the owner-named "DB purging" item, covered exhaustively**: - `RetentionRunner` ([pipeline/retention.py:210](messagefoundry/pipeline/retention.py)), per-connection + `RetentionRunner` ([pipeline/retention.py:210](../../../messagefoundry/pipeline/retention.py)), per-connection retention (ADR 0027), embedded-document pruning (ADR 0042), the time-boxed pass cap (ADR 0137), every `purge_*` / `strip_*` / `wal_checkpoint` / `vacuum` method on all three backends, purge under sustained load, purge vs in-flight rows, purge vs per-lane FIFO, purge attribution + audit trail, @@ -49,7 +49,7 @@ stops deleting, is green in CI on the production backend. | Cipher primitives, KeyProvider root-of-trust (`FCP:CRIT-1`), `require_encryption` fail-closed, DPAPI scope, secret leak assertions | FEATURE-COVERAGE-PLAN §11 `[CRYPTO]` rows 1–10 | `FCP:CRYPTO-3` is the plan-level P0; this chapter tests only the **store-column / purge write-back** consequences | | DR end-to-end audit (snapshot, codec, runner, scheduling, keep-N, standby, restore-verify, CLI) | FEATURE-COVERAGE-PLAN §18 `[DR]` rows 1–27 | this chapter adds **only** the server-DB legs that never run | | On-box `store.connect` per backend, healthy→PROCESSED under the NSSM service identity, DPAPI admin-mints/service-decrypts, `rotate-key` under the service account with `require_encryption=true` | WIN2025-TEST-PLAN `W25:S1.3` / `W25:S2.1` / `W25:S2.2` / `W25:B3` | this chapter only asks that WIN2025 gain the **missing retention/backup rows** | -| The authoritative per-backend purge specification (enforced / no-op (DBA-owned) / DBA-delegated) | [docs/PHI.md §8](docs/PHI.md) lines 924–1053 | it is correct — test against it, don't rewrite it | +| The authoritative per-backend purge specification (enforced / no-op (DBA-owned) / DBA-delegated) | [docs/PHI.md §8](../../PHI.md) lines 924–1053 | it is correct — test against it, don't rewrite it | | Engine-shard partitioning and the unified-store guard | ADR 0063 + `tests/test_sharding.py` | only the **concurrent orphan sweep** interaction appears here | **Objectives.** (1) Make the production backends prove the purge SQL that deletes PHI. (2) Make the @@ -70,7 +70,7 @@ matrix at all. | `tests/test_store_encryption.py` (~45 tests) + `tests/test_store_aad_binding.py` (13 tests) | AES-GCM round-trip, keyless passthrough, wrong-key fail-loud, undecryptable-row dead-letter at claim, on-open migration encrypt, rotation + retired-key bridge, frozen v1 fixture, v2 dispatch, unknown version/alg fail-closed, cell-AAD bind/mismatch/relocation rejection, `rotate-key` v1→v2 in place (SQLite) | | `tests/test_keyprovider.py` + `tests/test_keyprovider_vault.py` + `tests/test_crypto_transit.py` | `auto`/`env`/`dpapi` built-ins, unknown-name and unbuilt-external fail-closed, Vault KEK unwrap, Transit cipher, no key material in exception text | | `tests/test_backup_runner.py` (20) + `test_backup_crypto.py` (18) + `test_restore_verify.py` + `test_backup_restore_atleastonce.py` + `test_cli_backup_dispatch.py` + `test_backup_runner_concurrent_writer.py` | ADR 0049 single-pass orchestration, both snapshot methods consistent + non-mutating, archive under the store DEK, refuse-unencrypted-PHI, keep-N excluding verify-failed, leader gate, daily latch, `.mfbak` tamper/truncate/append/reorder matrix, restore-verify pass/fail/key-mismatch, at-least-once across restore, PHI-free CLI stdout | -| `tests/test_phi_at_rest_inventory.py` (~30 doc-vs-code guards) | Every cipher-covered cell is inventoried in PHI.md §2 with a protection level **and** a stated retention position; every `purge_*` + maintenance method is defined on all three backends and documented per backend ([:356](tests/test_phi_at_rest_inventory.py), [:928](tests/test_phi_at_rest_inventory.py) binds the verdicts to the method bodies); what `purge_message_bodies` blanks is named in §8 ([:377](tests/test_phi_at_rest_inventory.py)); retired false retention claims cannot reappear | +| `tests/test_phi_at_rest_inventory.py` (~30 doc-vs-code guards) | Every cipher-covered cell is inventoried in PHI.md §2 with a protection level **and** a stated retention position; every `purge_*` + maintenance method is defined on all three backends and documented per backend ([:356](../../../tests/test_phi_at_rest_inventory.py), [:928](../../../tests/test_phi_at_rest_inventory.py) binds the verdicts to the method bodies); what `purge_message_bodies` blanks is named in §8 ([:377](../../../tests/test_phi_at_rest_inventory.py)); retired false retention claims cannot reappear | | `tests/test_store_schema_hash.py` (6) + `tests/test_sqlserver_schema_init.py` (5) | ADR 0064 content hash tracks DDL + `_MIGRATION_REV`; applock taken before any CREATE; a current marker skips the batch, the applock and the timeout exemption | | `tests/test_pool_warm.py` (20) + `test_store_read_pool.py` (8) + `test_store_capability_matrix.py` | Warm-target clamping, fence validator, partial/timeout/cancel release, SQLite no-op, SQLite read pool `query_only`, over-provision thresholds (pure fn), capability-flag/doc parity | | `tests/test_cli.py:1383-1500` | The PHI retention posture gate: prod refuse-to-start naming `[security].delete_message_bodies_after_days`, staging/loopback auto-bound to 30 d, `allow_keeping_phi_indefinitely` audited override | @@ -89,14 +89,14 @@ and is the right place to add new documentation assertions rather than writing a chapter must not restate `FCP:CRYPTO-3`. **Two published coverage claims in this area are stale and are corrected by this chapter, not repeated:** -FEATURE-COVERAGE-PLAN **`FCP:STOREF-5`** ([:956](docs/testing/FEATURE-COVERAGE-PLAN.md)) marks -per-connection retention "covered … (incl. three_backend_parity)" and **`FCP:STOREF-6`** ([:957](docs/testing/FEATURE-COVERAGE-PLAN.md)) +FEATURE-COVERAGE-PLAN **`FCP:STOREF-5`** ([:956](../FEATURE-COVERAGE-PLAN.md)) marks +per-connection retention "covered … (incl. three_backend_parity)" and **`FCP:STOREF-6`** ([:957](../FEATURE-COVERAGE-PLAN.md)) the same for document pruning — the parity tests exist but **no CI leg invokes those files** (§2.3 R1/R4). **`FCP:STOREF-18`** still schedules ADR 0105 Phase 3b as a build -([:398](docs/testing/FEATURE-COVERAGE-PLAN.md), [:535](docs/testing/FEATURE-COVERAGE-PLAN.md)) while it ships -([base.py:850](messagefoundry/store/base.py), [api/app.py:3225](messagefoundry/api/app.py), -`tests/test_attachment_download_api.py`), as does [docs/adr/README.md:134](docs/adr/README.md) -(FEATURE-COVERAGE-PLAN [:379](docs/testing/FEATURE-COVERAGE-PLAN.md) already flags the row as stale — +([:398](../FEATURE-COVERAGE-PLAN.md), [:535](../FEATURE-COVERAGE-PLAN.md)) while it ships +([base.py:850](../../../messagefoundry/store/base.py), [api/app.py:3225](../../../messagefoundry/api/app.py), +`tests/test_attachment_download_api.py`), as does [docs/adr/README.md:134](../../adr/README.md) +(FEATURE-COVERAGE-PLAN [:379](../FEATURE-COVERAGE-PLAN.md) already flags the row as stale — the scheduling rows below it were never corrected). ### 2.3 Risk analysis @@ -104,29 +104,29 @@ the scheduling rows below it were never corrected). | Risk | Failure mode | Blast radius | Detected today? | Priority | |---|---|---|---|---| | **R1** ADR 0027 per-connection purge `CASE` has never run on PG/SQL Server (`connection_cutoffs` appears **0** times in `tests/test_postgres_store.py` and `tests/test_sqlserver_store.py`; the AC-8 parity case lives only in `tests/test_per_connection_retention.py:262`, which no CI step names and which the `serverdb` regex at `ci.yml:434` does not match) | A wrong `_pg_cutoff_case` / `_qmark_cutoff_case` predicate purges the **wrong feed's** PHI bodies (irreversible; the count-and-log row survives so nothing looks broken) or silently purges nothing (unbounded PHI at rest) | Every PHI feed on the production backend; irreversible either way | **No.** SQLite stays green in both directions | **P0** | -| **R2** The shipped at-rest **writer** is barely exercised. `[store].aad_bind` defaults `True` → `open_store` builds a `write_v2` / `mfenc:v2` cell-AAD cipher ([base.py:1736](messagefoundry/store/base.py)), but `make_cipher`'s library default is `write_v2=False` and **no** workflow sets `MEFOR_TEST_FORCE_AAD_BIND` (only `tests/conftest.py:126,138` and ADR 0019:516 mention it) | An unbound or mis-threaded `cell_aad` on any write path — purge re-encrypt, document strip write-back, attachment re-seal, restore — yields rows the shipped cipher cannot decrypt (unreadable PHI) or silently drops the binding (ASVS 11.3.3 regression) | Whole store; discovered only in production | **No.** 13 SQLite-only targeted tests | **P0** | -| **R3** The PHI serve retention gate reads **only global** windows ([__main__.py:1968-1975](messagefoundry/__main__.py) builds `unbounded_windows` from `settings.retention.messages_days` / `dead_letter_days` alone) and never consults the registry's per-connection overrides | A PHI instance with a global 30-day window and every inbound at `messages_days=0` passes the fail-closed gate and retains PHI forever | An audited, security-labelled fail-closed control (ASVS 14.2.4) is bypassable by ordinary Connection config | **No.** No warning, no audit entry, no test | **P0** | -| **R4** ADR 0042 `strip_embedded_documents` has never run on PG/SQL Server (**0** occurrences in both server suites) — and it is a select → decrypt → codec-transform → **re-encrypt write-back** over stored PHI bodies ([postgres.py:6395](messagefoundry/store/postgres.py), [sqlserver.py:5626](messagefoundry/store/sqlserver.py)) | A dialect or write-back bug corrupts stored bodies (unparseable HL7) or leaves bulky base64 PHI in place forever | Every document feed on the production backend | **No** | **P1** | -| **R5** `strip_embedded_documents` materializes and **decrypts every eligible row with no LIMIT/TOP/batch** on all three backends ([store.py:8465](messagefoundry/store/store.py), [postgres.py:6420](messagefoundry/store/postgres.py), [sqlserver.py:5650](messagefoundry/store/sqlserver.py)); the ADR 0137 deadline is checked only **before** the phase ([retention.py:443](messagefoundry/pipeline/retention.py)), never inside the per-threshold loop | The first pass after enabling `prune_documents_after` pulls the entire un-stripped backlog of full PHI bodies into engine heap: OOM / engine crash mid-purge, unbounded PHI plaintext in memory, and a pass that blows straight through `max_pass_seconds` | Engine availability + PHI-in-heap exposure at exactly the moment PHI is being reduced | **No.** No test bounds the candidate set | **P1** | -| **R6** Purge under sustained load is untested on every backend. SQLite holds the single writer lock across the whole multi-statement purge transaction (`async with self._lock`, [store.py:8345](messagefoundry/store/store.py)); SQL Server builds an unbatched `#eligible` temp table ([sqlserver.py:5566](messagefoundry/store/sqlserver.py)) and **`messages` has no `LOCK_ESCALATION=DISABLE`** (only `queue` does, [sqlserver.py:1057-1063](messagefoundry/store/sqlserver.py)) | A first large purge escalates to a table X lock on `messages` and blocks ingress inserts, stalling ACKs past the MLLP receive timeout | Availability incident on the production backend, during a scheduled maintenance window | **No.** No concurrent-purge test anywhere; no purge profile in `docs/LOAD-TESTING.md`; no metric to see it coming | **P1** | +| **R2** The shipped at-rest **writer** is barely exercised. `[store].aad_bind` defaults `True` → `open_store` builds a `write_v2` / `mfenc:v2` cell-AAD cipher ([base.py:1736](../../../messagefoundry/store/base.py)), but `make_cipher`'s library default is `write_v2=False` and **no** workflow sets `MEFOR_TEST_FORCE_AAD_BIND` (only `tests/conftest.py:126,138` and ADR 0019:516 mention it) | An unbound or mis-threaded `cell_aad` on any write path — purge re-encrypt, document strip write-back, attachment re-seal, restore — yields rows the shipped cipher cannot decrypt (unreadable PHI) or silently drops the binding (ASVS 11.3.3 regression) | Whole store; discovered only in production | **No.** 13 SQLite-only targeted tests | **P0** | +| **R3** The PHI serve retention gate reads **only global** windows ([__main__.py:1968-1975](../../../messagefoundry/__main__.py) builds `unbounded_windows` from `settings.retention.messages_days` / `dead_letter_days` alone) and never consults the registry's per-connection overrides | A PHI instance with a global 30-day window and every inbound at `messages_days=0` passes the fail-closed gate and retains PHI forever | An audited, security-labelled fail-closed control (ASVS 14.2.4) is bypassable by ordinary Connection config | **No.** No warning, no audit entry, no test | **P0** | +| **R4** ADR 0042 `strip_embedded_documents` has never run on PG/SQL Server (**0** occurrences in both server suites) — and it is a select → decrypt → codec-transform → **re-encrypt write-back** over stored PHI bodies ([postgres.py:6395](../../../messagefoundry/store/postgres.py), [sqlserver.py:5626](../../../messagefoundry/store/sqlserver.py)) | A dialect or write-back bug corrupts stored bodies (unparseable HL7) or leaves bulky base64 PHI in place forever | Every document feed on the production backend | **No** | **P1** | +| **R5** `strip_embedded_documents` materializes and **decrypts every eligible row with no LIMIT/TOP/batch** on all three backends ([store.py:8465](../../../messagefoundry/store/store.py), [postgres.py:6420](../../../messagefoundry/store/postgres.py), [sqlserver.py:5650](../../../messagefoundry/store/sqlserver.py)); the ADR 0137 deadline is checked only **before** the phase ([retention.py:443](../../../messagefoundry/pipeline/retention.py)), never inside the per-threshold loop | The first pass after enabling `prune_documents_after` pulls the entire un-stripped backlog of full PHI bodies into engine heap: OOM / engine crash mid-purge, unbounded PHI plaintext in memory, and a pass that blows straight through `max_pass_seconds` | Engine availability + PHI-in-heap exposure at exactly the moment PHI is being reduced | **No.** No test bounds the candidate set | **P1** | +| **R6** Purge under sustained load is untested on every backend. SQLite holds the single writer lock across the whole multi-statement purge transaction (`async with self._lock`, [store.py:8345](../../../messagefoundry/store/store.py)); SQL Server builds an unbatched `#eligible` temp table ([sqlserver.py:5566](../../../messagefoundry/store/sqlserver.py)) and **`messages` has no `LOCK_ESCALATION=DISABLE`** (only `queue` does, [sqlserver.py:1057-1063](../../../messagefoundry/store/sqlserver.py)) | A first large purge escalates to a table X lock on `messages` and blocks ingress inserts, stalling ACKs past the MLLP receive timeout | Availability incident on the production backend, during a scheduled maintenance window | **No.** No concurrent-purge test anywhere; no purge profile in `docs/LOAD-TESTING.md`; no metric to see it coming | **P1** | | **R7** Retention, purge, VACUUM and backup appear in **no** acceptance or on-box evidence matrix. `harness/acceptance/matrix.py` has store rows B1–B6 / C1–C8 and no retention or backup row; WIN2025-TEST-PLAN / -MATRIX / -ACCEPTANCE never mention retention, purge, vacuum, `.mfbak` or restore | A fully green on-box sign-off is compatible with retention never running and backups never being written | Every release sign-off | **No.** FEATURE-COVERAGE-PLAN §18 records the DR half; the retention half is unrecorded | **P1** | -| **R8** No retention observability. `api/metrics.py` exports 22 series with **no** DB-size gauge, purged-row counters, last-successful-pass timestamp, or the ADR 0137 `capped` flag. The only signals are the `storage_threshold` alert ([retention.py:658](messagefoundry/pipeline/retention.py)) and the `retention_purge` audit detail ([:943](messagefoundry/pipeline/retention.py)). DB size *is* on `GET /status` ([app.py:4430,4512](messagefoundry/api/app.py)) but not scrapeable | An operator cannot tell maintenance is falling behind, capped every interval, or that the store is growing — until the disk fills. ADR 0137 §3 states the cap exists "so an operator can see that maintenance is falling behind"; the only surface is a JSON blob in the audit log | Silent capacity failure | **Partially** (`/status` DB size only) | **P1** | -| **R9** `sweep_orphan_attachments` DELETEs `refcount<=0` and header-less chunk groups with **no age or grace guard** ([store.py:3540-3549](messagefoundry/store/store.py), [postgres.py:4509](messagefoundry/store/postgres.py), [sqlserver.py:7319](messagefoundry/store/sqlserver.py)) and runs at **every** process start, **not leader-gated** ([engine.py:882](messagefoundry/pipeline/engine.py) — the comment says "safe on any node"). The ADR 0105 two-object commit deliberately commits chunks at refcount 0 *before* the incref lands | A restarting sibling (engine shard, HA standby) sweeps a peer's in-flight chunks. Fails closed (ingress rolls back, no ACK) — but repeatedly kills a busy node's large-document ingress. With no reconciliation sweep (Phase 3a declined one), a refcount over-count retains PHI chunks forever, undetected | Large-document ingress availability; silent PHI retention | **No.** No concurrent-sweep test | **P1** | -| **R10** `outbox.payload` (the SQL Server legacy table) carries PHI, is in `_CIPHER_COLUMNS` ([sqlserver.py:2222](messagefoundry/store/sqlserver.py)), is **recreated on every open** ([sqlserver.py:988](messagefoundry/store/sqlserver.py)), and is touched by **no purge on any backend** ([PHI.md:1039](docs/PHI.md) documents it as retained forever) | If any path still writes it, PHI accumulates outside every retention window and outside the audited purge | Direct ASVS 14.2.7 / HIPAA minimum-necessary failure the count-and-log invariant would not surface | **No.** No test asserts the table stays empty or that no writer targets it | **P1** | -| **R11** `[retention].wal_checkpoint_seconds` / `vacuum_at` on a server DB are **silent no-ops** ([sqlserver.py:5804,5808](messagefoundry/store/sqlserver.py), [postgres.py:6565,6569](messagefoundry/store/postgres.py)) — no settings validation, no serve warning, and the runner's startup log still prints `vacuum_at=…` ([retention.py:296-310](messagefoundry/pipeline/retention.py)) as if enabled. [docs/FEATURE-MAP.md:97](docs/FEATURE-MAP.md) says "Retention / purge / maintenance ✅ (SQLite, PG, SQL Server)" with no DBA-delegated caveat | Operator configures nightly reclamation on SQL Server, sees it in the log, sees no error, believes disk is being reclaimed. Space grows until the volume fills | Hard outage. PHI.md §8 gets this right; the map, the log and the config surface do not | **No** | **P1** | +| **R8** No retention observability. `api/metrics.py` exports 22 series with **no** DB-size gauge, purged-row counters, last-successful-pass timestamp, or the ADR 0137 `capped` flag. The only signals are the `storage_threshold` alert ([retention.py:658](../../../messagefoundry/pipeline/retention.py)) and the `retention_purge` audit detail ([:943](../../../messagefoundry/pipeline/retention.py)). DB size *is* on `GET /status` ([app.py:4430,4512](../../../messagefoundry/api/app.py)) but not scrapeable | An operator cannot tell maintenance is falling behind, capped every interval, or that the store is growing — until the disk fills. ADR 0137 §3 states the cap exists "so an operator can see that maintenance is falling behind"; the only surface is a JSON blob in the audit log | Silent capacity failure | **Partially** (`/status` DB size only) | **P1** | +| **R9** `sweep_orphan_attachments` DELETEs `refcount<=0` and header-less chunk groups with **no age or grace guard** ([store.py:3540-3549](../../../messagefoundry/store/store.py), [postgres.py:4509](../../../messagefoundry/store/postgres.py), [sqlserver.py:7319](../../../messagefoundry/store/sqlserver.py)) and runs at **every** process start, **not leader-gated** ([engine.py:882](../../../messagefoundry/pipeline/engine.py) — the comment says "safe on any node"). The ADR 0105 two-object commit deliberately commits chunks at refcount 0 *before* the incref lands | A restarting sibling (engine shard, HA standby) sweeps a peer's in-flight chunks. Fails closed (ingress rolls back, no ACK) — but repeatedly kills a busy node's large-document ingress. With no reconciliation sweep (Phase 3a declined one), a refcount over-count retains PHI chunks forever, undetected | Large-document ingress availability; silent PHI retention | **No.** No concurrent-sweep test | **P1** | +| **R10** `outbox.payload` (the SQL Server legacy table) carries PHI, is in `_CIPHER_COLUMNS` ([sqlserver.py:2222](../../../messagefoundry/store/sqlserver.py)), is **recreated on every open** ([sqlserver.py:988](../../../messagefoundry/store/sqlserver.py)), and is touched by **no purge on any backend** ([PHI.md:1039](../../PHI.md) documents it as retained forever) | If any path still writes it, PHI accumulates outside every retention window and outside the audited purge | Direct ASVS 14.2.7 / HIPAA minimum-necessary failure the count-and-log invariant would not surface | **No.** No test asserts the table stays empty or that no writer targets it | **P1** | +| **R11** `[retention].wal_checkpoint_seconds` / `vacuum_at` on a server DB are **silent no-ops** ([sqlserver.py:5804,5808](../../../messagefoundry/store/sqlserver.py), [postgres.py:6565,6569](../../../messagefoundry/store/postgres.py)) — no settings validation, no serve warning, and the runner's startup log still prints `vacuum_at=…` ([retention.py:296-310](../../../messagefoundry/pipeline/retention.py)) as if enabled. [docs/FEATURE-MAP.md:97](../../FEATURE-MAP.md) says "Retention / purge / maintenance ✅ (SQLite, PG, SQL Server)" with no DBA-delegated caveat | Operator configures nightly reclamation on SQL Server, sees it in the log, sees no error, believes disk is being reclaimed. Space grows until the volume fills | Hard outage. PHI.md §8 gets this right; the map, the log and the config surface do not | **No** | **P1** | | **R12** Backup and DR against a real server-DB store have **never executed**: `test_backup_runner_server_db_{postgres,sqlserver}.py`, `test_dr7_server_config_only_backup_{postgres,sqlserver}.py`, `test_dr_server_seed_gate_{postgres,sqlserver}.py` are all `MEFOR_TEST_*`-gated and named by **no** CI step | The `DbaDelegatedError` → config-only fallback, the `dr_backup` audit row on a server-DB store, and the ADR 0102 live seed gate are the whole DR story for the production backend, asserted only against a monkeypatched SQLite store | DR sign-off on the production backend | **No** (`FCP:DR-7` flags the shape; the sharper point is the tests exist and never run) | **P1** | -| **R13** ADR 0027 D3 / AC-6 promises the audit row records "the per-connection cutoffs **+ per-connection purged counts**". The shipped detail ([retention.py:943-990](messagefoundry/pipeline/retention.py)) records `messages_overrides` / `dead_letter_overrides` plus only **aggregate** `messages_purged` / `dead_purged`; the AC-6 test asserts exactly that | An auditor cannot answer "how many bodies did feed X lose in this pass". The ADR's own acceptance criterion is not met | Purge attribution — the audit trail for irreversible PHI deletion | **No** (silent doc/code divergence) | **P2** | +| **R13** ADR 0027 D3 / AC-6 promises the audit row records "the per-connection cutoffs **+ per-connection purged counts**". The shipped detail ([retention.py:943-990](../../../messagefoundry/pipeline/retention.py)) records `messages_overrides` / `dead_letter_overrides` plus only **aggregate** `messages_purged` / `dead_purged`; the AC-6 test asserts exactly that | An auditor cannot answer "how many bodies did feed X lose in this pass". The ADR's own acceptance criterion is not met | Purge attribution — the audit trail for irreversible PHI deletion | **No** (silent doc/code divergence) | **P2** | | **R14** No purge-at-scale, VACUUM-duration, freelist/index-bloat or disk-reclamation measurement on any backend. The ADR 0137 cap is proved only with an injected fake monotonic clock (`test_retention.py:632-733`) | The recommended `max_pass_seconds` (~14400) and off-peak `vacuum_at` guidance are unvalidated; an operator sizing a maintenance window has no number. A VACUUM that outruns the window blocks the whole DB and the cap by design cannot stop it | Maintenance-window planning; a self-inflicted outage | **No** | **P2** | | **R15** No **legal-hold / litigation-hold / per-message retention exception** exists anywhere (grep across `messagefoundry/` and `docs/` finds only `CLA.md:40` and a counsel brief, both patent-litigation). The only exception is per-connection `messages_days=0`, which is feed-wide | A HIPAA / e-discovery hold on one patient's messages cannot be honoured without disabling an entire feed's window (over-retaining everyone else's PHI); no artifact proves held data was preserved | Regulatory / discovery exposure | **No** — the mechanism does not exist | **P2** | | **R16** Schema drift and mixed-version fleets untested. ADR 0064 accepts that out-of-band drift is no longer healed on open (remedy: manual `DELETE FROM schema_meta`) and that two builds alternating opens re-run the full batch under the exclusive lock. SQLite carries 17 ad-hoc `ADD COLUMN` migrations with exactly **one** upgrade-path test (`test_retention.py:374`, the #306 `last_used_at` column) | A hand-dropped index is silently permanent (plans degrade with no error); a rolling upgrade across an HA pair or engine-shard fleet reintroduces the startup convoy ADR 0064 removed | Cold-start latency + query-plan degradation; upgrade risk | **No.** No pre-migration fixture DB exists | **P2** | | **R17** Purge vs **per-lane FIFO** asserted only indirectly. `purge_message_bodies` UPDATEs `queue` rows on the same hot table/indexes the claimers seek; the FIFO suites (`test_seq_only_fifo`, `test_claim_fifo_heads`, `test_batch_claim_fifo`) never run a purge | A lock or index interaction that blocks or reorders a lane head is invisible to both suites | Ordering guarantee — a core reliability claim | **No** | **P2** | | **R18** Retention passes consume the shared server-DB pool (`_timed_acquire` / `_acquire`) with **no reservation or priority**; ADR 0062's inverted-U sizing was measured **without** a concurrent maintenance pass | At the measured knee (N≈32 inbounds, pool 40) a long purge holding slots for minutes shifts the whole engine along the inverted-U | Engine-wide throughput during maintenance | **No.** Pool-wait metrics exist; nothing correlates them with a pass | **P2** | | **R19** Postgres `rotate-key` column coverage is thinner than SQL Server's (SS asserts the full response/state pass at `test_sqlserver_store.py:983`; PG only via attachment-chunk re-seal at `test_postgres_store.py:2801`). **No interrupted-rotation resume test on any backend** | A half-rotated Postgres store where one column family was missed leaves rows readable only under a retired key; drop that key and the PHI is unrecoverable | Irrecoverable PHI | **No** (recorded as `FCP:CRYPTO-4`; restated here because it is *store-column*, not cipher-primitive, coverage) | **P2** | -| **R20** [docs/FEATURE-MAP.md](docs/FEATURE-MAP.md) §5 is materially behind: zero mention of ADRs 0019/0027/0042/0049/0062/0064/0105/0137; no row for DR backup, streaming attachments, the KeyProvider seam, pool sizing or schema-init. `tests/test_feature_map_claims.py` guards links and ASVS framing only | The capability catalog a reviewer, adopter and the next test-plan author reads overstates maintenance parity and understates what is built | Trust in the published capability status | **No** | **P2** | -| **R21** [ADR 0019:523](docs/adr/0019-pluggable-keyprovider-hsm-kms-vault.md) still states cell-bound AAD is "opt-in via `[store].aad_bind` (**off by default**)" while [settings.py:381](messagefoundry/config/settings.py) ships `aad_bind: bool = True` (flipped by ADR 0148) | The security ADR that owns the `mfenc` format contract misstates the shipped default — a reader planning a rotation or a restore reasons from the wrong posture | ASVS evidence quality | **No** | **P2** | -| **R22** **A purge pass that dies mid-transaction is untested on every backend.** Every suite kills the *process* between passes, never inside one: the multi-statement purge transaction (SQLite under `self._lock`, [store.py:8345](messagefoundry/store/store.py); the SQL Server `#eligible` temp-table batch, [sqlserver.py:5566](messagefoundry/store/sqlserver.py)) and the in-process day marker `_last_vacuum_day` ([retention.py:251,997-1005](messagefoundry/pipeline/retention.py)) are never crash-tested together | A half-committed purge leaves bodies blanked with metadata intact (or the reverse), strands rows nothing will revisit, or advances the VACUUM/day marker past work that never ran — so the next pass skips it. Irreversible in the deleting direction, silent in the retaining direction | Every PHI feed; a maintenance-window crash is exactly when nobody is watching | **No.** No mid-transaction kill anywhere in `test_retention.py` | **P1** | -| **R23** **No disk-full / ENOSPC injection anywhere.** The purge, the WAL checkpoint/VACUUM ([store.py:8650,8658](messagefoundry/store/store.py)) and the `.mfbak` write all assume the volume has room; `OSError` is caught broadly in the app-log phases ([retention.py:687,742,798,838,887,929](messagefoundry/pipeline/retention.py)) but no test drives a full volume. `disk_free_bytes` is `0` on server DBs and the only capacity signal is the `storage_threshold` alert ([retention.py:658](messagefoundry/pipeline/retention.py)) keyed on `max_db_mb`, not on free space | The single most common hospital-host failure. A volume that fills mid-purge, mid-checkpoint or mid-archive can corrupt a SQLite WAL, leave a truncated archive that later "restores", or drop received messages with no alert — and nothing proves the engine fails *closed* or recovers when space is freed | Whole-store availability + silent message loss + a worthless backup | **No** | **P1** | -| **R24** **Destructive operator error has no owning coverage.** Nothing refuses or audits: seeding/restoring an **older** archive over a store holding newer messages (the ADR 0102 vintage probe only guards DR activation, [dr.py:397-453](messagefoundry/pipeline/dr.py)); a per-connection purge override naming a Connection that does not exist (a typo silently falls through to the global cutoff); `rotate-key` run with a new active key and **no** prior key in `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` (the guard at [__main__.py:3462-3470](messagefoundry/__main__.py) only checks that an *active* key exists); `connection remove` deleting a Connection that still has queued rows ([connections_edit.py:202](messagefoundry/config/connections_edit.py) is config-only and never consults the store) | The realistic loss path is an operator, not a bug: unrecoverable PHI (rotation without the retired key), the wrong feed purged, queued rows orphaned by a config edit, or newer messages overwritten by an old seed — each irreversible and, today, unaudited | The whole store; the highest-consequence, least-tested class in this chapter | **No** | **P1** | +| **R20** [docs/FEATURE-MAP.md](../../FEATURE-MAP.md) §5 is materially behind: zero mention of ADRs 0019/0027/0042/0049/0062/0064/0105/0137; no row for DR backup, streaming attachments, the KeyProvider seam, pool sizing or schema-init. `tests/test_feature_map_claims.py` guards links and ASVS framing only | The capability catalog a reviewer, adopter and the next test-plan author reads overstates maintenance parity and understates what is built | Trust in the published capability status | **No** | **P2** | +| **R21** [ADR 0019:523](../../adr/0019-pluggable-keyprovider-hsm-kms-vault.md) still states cell-bound AAD is "opt-in via `[store].aad_bind` (**off by default**)" while [settings.py:381](../../../messagefoundry/config/settings.py) ships `aad_bind: bool = True` (flipped by ADR 0148) | The security ADR that owns the `mfenc` format contract misstates the shipped default — a reader planning a rotation or a restore reasons from the wrong posture | ASVS evidence quality | **No** | **P2** | +| **R22** **A purge pass that dies mid-transaction is untested on every backend.** Every suite kills the *process* between passes, never inside one: the multi-statement purge transaction (SQLite under `self._lock`, [store.py:8345](../../../messagefoundry/store/store.py); the SQL Server `#eligible` temp-table batch, [sqlserver.py:5566](../../../messagefoundry/store/sqlserver.py)) and the in-process day marker `_last_vacuum_day` ([retention.py:251,997-1005](../../../messagefoundry/pipeline/retention.py)) are never crash-tested together | A half-committed purge leaves bodies blanked with metadata intact (or the reverse), strands rows nothing will revisit, or advances the VACUUM/day marker past work that never ran — so the next pass skips it. Irreversible in the deleting direction, silent in the retaining direction | Every PHI feed; a maintenance-window crash is exactly when nobody is watching | **No.** No mid-transaction kill anywhere in `test_retention.py` | **P1** | +| **R23** **No disk-full / ENOSPC injection anywhere.** The purge, the WAL checkpoint/VACUUM ([store.py:8650,8658](../../../messagefoundry/store/store.py)) and the `.mfbak` write all assume the volume has room; `OSError` is caught broadly in the app-log phases ([retention.py:687,742,798,838,887,929](../../../messagefoundry/pipeline/retention.py)) but no test drives a full volume. `disk_free_bytes` is `0` on server DBs and the only capacity signal is the `storage_threshold` alert ([retention.py:658](../../../messagefoundry/pipeline/retention.py)) keyed on `max_db_mb`, not on free space | The single most common hospital-host failure. A volume that fills mid-purge, mid-checkpoint or mid-archive can corrupt a SQLite WAL, leave a truncated archive that later "restores", or drop received messages with no alert — and nothing proves the engine fails *closed* or recovers when space is freed | Whole-store availability + silent message loss + a worthless backup | **No** | **P1** | +| **R24** **Destructive operator error has no owning coverage.** Nothing refuses or audits: seeding/restoring an **older** archive over a store holding newer messages (the ADR 0102 vintage probe only guards DR activation, [dr.py:397-453](../../../messagefoundry/pipeline/dr.py)); a per-connection purge override naming a Connection that does not exist (a typo silently falls through to the global cutoff); `rotate-key` run with a new active key and **no** prior key in `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` (the guard at [__main__.py:3462-3470](../../../messagefoundry/__main__.py) only checks that an *active* key exists); `connection remove` deleting a Connection that still has queued rows ([connections_edit.py:202](../../../messagefoundry/config/connections_edit.py) is config-only and never consults the store) | The realistic loss path is an operator, not a bug: unrecoverable PHI (rotation without the retired key), the wrong feed purged, queued rows orphaned by a config edit, or newer messages overwritten by an old seed — each irreversible and, today, unaudited | The whole store; the highest-consequence, least-tested class in this chapter | **No** | **P1** | ### 2.4 Test matrix @@ -223,16 +223,16 @@ plan. | STORE-61 | Backend capability flags gate the attachment surface | Negative/Security | pytest | container-CI | x3 | T | P2 | With `supports_streaming_attachments` forced False, `put_attachment` / `sweep_orphan_attachments` raise rather than silently no-op, and the engine's startup sweep is skipped without an error log | | STORE-62 | `open_store` unknown-backend and cipher-provider fail-closed | Negative/Security | pytest | dev-PC | n/a | T | P2 | `backend=mysql` raises `NotImplementedError`; `cipher_provider=aesgcm2` raises a `ValueError` naming the two valid providers; an unbuilt external `key_provider` (`aws_kms`/`azure_kv`/`gcp_kms`/`pkcs11`) raises `KeyProviderError` with no key material in the message | | STORE-63 | Purge pass killed **mid-transaction** leaves the store consistent | HA/Resilience | pytest | container-CI | x3 | T | P1 | `SIGKILL`/connection-abort the engine inside `purge_message_bodies`' delete/blank batch (fault-injected between the body blank and the metadata NULL). On re-open: **no** row exists with `raw = ''` and `metadata IS NOT NULL` (or the reverse), row counts reconcile against a pre-kill snapshot, and no `queue`/`messages` row is left claimed-but-unowned. Repeated at 5 distinct injection points × 3 backends with no surviving inconsistency | -| STORE-64 | An interrupted purge is resumable and never advances a watermark past unpurged data | HA/Resilience | pytest | container-CI | x3 | T | P1 | After the STORE-63 kill, the next `RetentionRunner.run_once()` purges **exactly** the rows the killed pass did not, with no double-decref of a shared attachment and no rows stranded past the cutoff. `_last_vacuum_day` ([retention.py:251,548,1005](messagefoundry/pipeline/retention.py)) and every phase's last-run marker are **unchanged** by a pass that did not complete, so the skipped phase stays due; a planted "mark first, work second" ordering fails the test | -| STORE-65 | Disk-full (ENOSPC) **mid-purge** fails closed and recovers | HA/Resilience | pytest | container-CI | x3 | T | P1 | Inject `OSError(ENOSPC)` (loopback/quota-bounded volume on SQLite; a filled data file on PG/SQL Server) during a purge pass: the pass aborts with a logged `ERROR`, the store re-opens cleanly, **no acknowledged message is lost** and none silently changes disposition, and an operator-visible `storage_threshold` alert fires (`AlertSink.storage_threshold`, [pipeline/alerts.py:87](messagefoundry/pipeline/alerts.py)) — see the ALERT chapter for the sink/notification half. Freeing space and re-running completes the purge with no manual repair | -| STORE-66 | ENOSPC **mid WAL-checkpoint / VACUUM** does not corrupt the store | HA/Resilience | pytest | dev-PC | SQLite | T | P1 | Fill the volume during `wal_checkpoint()` ([store.py:8650](messagefoundry/store/store.py)) and during `vacuum()` ([store.py:8658](messagefoundry/store/store.py)): each raises rather than silently returning, `PRAGMA integrity_check` returns `ok` afterwards, the `-wal` sidecar is not truncated mid-frame, ingress continues to fail closed (NAK, no accepted-and-dropped message), and the next successful pass reclaims the space | +| STORE-64 | An interrupted purge is resumable and never advances a watermark past unpurged data | HA/Resilience | pytest | container-CI | x3 | T | P1 | After the STORE-63 kill, the next `RetentionRunner.run_once()` purges **exactly** the rows the killed pass did not, with no double-decref of a shared attachment and no rows stranded past the cutoff. `_last_vacuum_day` ([retention.py:251,548,1005](../../../messagefoundry/pipeline/retention.py)) and every phase's last-run marker are **unchanged** by a pass that did not complete, so the skipped phase stays due; a planted "mark first, work second" ordering fails the test | +| STORE-65 | Disk-full (ENOSPC) **mid-purge** fails closed and recovers | HA/Resilience | pytest | container-CI | x3 | T | P1 | Inject `OSError(ENOSPC)` (loopback/quota-bounded volume on SQLite; a filled data file on PG/SQL Server) during a purge pass: the pass aborts with a logged `ERROR`, the store re-opens cleanly, **no acknowledged message is lost** and none silently changes disposition, and an operator-visible `storage_threshold` alert fires (`AlertSink.storage_threshold`, [pipeline/alerts.py:87](../../../messagefoundry/pipeline/alerts.py)) — see the ALERT chapter for the sink/notification half. Freeing space and re-running completes the purge with no manual repair | +| STORE-66 | ENOSPC **mid WAL-checkpoint / VACUUM** does not corrupt the store | HA/Resilience | pytest | dev-PC | SQLite | T | P1 | Fill the volume during `wal_checkpoint()` ([store.py:8650](../../../messagefoundry/store/store.py)) and during `vacuum()` ([store.py:8658](../../../messagefoundry/store/store.py)): each raises rather than silently returning, `PRAGMA integrity_check` returns `ok` afterwards, the `-wal` sidecar is not truncated mid-frame, ingress continues to fail closed (NAK, no accepted-and-dropped message), and the next successful pass reclaims the space | | STORE-67 | ENOSPC **mid `.mfbak` write** never leaves a "restorable" truncated archive | HA/Resilience | pytest | dev-PC | SQLite | T | P1 | Fill the destination volume during `BackupRunner`'s archive write: the run fails with a named error, writes a `dr_backup` audit row recording the failure, and either removes the partial archive or leaves one that `restore-verify` rejects (**never** exits 0). The keep-N prune does not count the failed archive; freeing space and re-running produces an archive that restore-verifies | -| STORE-68 | Free-space threshold — not just `max_db_mb` — raises an operator-visible capacity alert | Functional | pytest | container-CI | x3 | T | P1 | A free-space signal exists alongside the `max_db_mb` size threshold ([retention.py:658](messagefoundry/pipeline/retention.py)): crossing a configured free-space floor fires exactly one `storage_threshold` alert carrying path + sizes only (no message content), and the alert clears/re-arms once space is freed. On PG / SQL Server, where `disk_free_bytes` is `0` by construction ([postgres.py:6592](messagefoundry/store/postgres.py), [sqlserver.py:9205](messagefoundry/store/sqlserver.py)), the row asserts the documented DBA-delegated position instead of a fabricated number | -| STORE-69 | The **unbounded** strip path measurably degrades (anti-vacuity control for STORE-16) | Performance | pytest | dev-PC | SQLite | T | P1 | Against the pre-fix `strip_embedded_documents` ([store.py:8425](messagefoundry/store/store.py)) with a synthetic backlog of N large document bodies, peak process RSS grows approximately linearly in N and exceeds a declared bound at the target N; the bounded-batch implementation (STORE-16) stays flat under the same load. Without this control STORE-16 can pass vacuously on a backlog too small to materialize | -| STORE-70 | Restoring / seeding an **older** archive over a live store is refused or audited | Negative/Security | pytest | container-CI | x3 | T | P1 | Restoring a `.mfbak` whose newest `received_at` predates the live store's newest message either **refuses** with a named vintage reason, or completes only under an explicit operator override that writes one audit row naming the actor, both vintages and the message count at risk. The ADR 0102 vintage probe ([dr.py:397-453](messagefoundry/pipeline/dr.py)) guards DR activation; this row extends the same posture to an operator-driven restore. **Never a silent overwrite** | +| STORE-68 | Free-space threshold — not just `max_db_mb` — raises an operator-visible capacity alert | Functional | pytest | container-CI | x3 | T | P1 | A free-space signal exists alongside the `max_db_mb` size threshold ([retention.py:658](../../../messagefoundry/pipeline/retention.py)): crossing a configured free-space floor fires exactly one `storage_threshold` alert carrying path + sizes only (no message content), and the alert clears/re-arms once space is freed. On PG / SQL Server, where `disk_free_bytes` is `0` by construction ([postgres.py:6592](../../../messagefoundry/store/postgres.py), [sqlserver.py:9205](../../../messagefoundry/store/sqlserver.py)), the row asserts the documented DBA-delegated position instead of a fabricated number | +| STORE-69 | The **unbounded** strip path measurably degrades (anti-vacuity control for STORE-16) | Performance | pytest | dev-PC | SQLite | T | P1 | Against the pre-fix `strip_embedded_documents` ([store.py:8425](../../../messagefoundry/store/store.py)) with a synthetic backlog of N large document bodies, peak process RSS grows approximately linearly in N and exceeds a declared bound at the target N; the bounded-batch implementation (STORE-16) stays flat under the same load. Without this control STORE-16 can pass vacuously on a backlog too small to materialize | +| STORE-70 | Restoring / seeding an **older** archive over a live store is refused or audited | Negative/Security | pytest | container-CI | x3 | T | P1 | Restoring a `.mfbak` whose newest `received_at` predates the live store's newest message either **refuses** with a named vintage reason, or completes only under an explicit operator override that writes one audit row naming the actor, both vintages and the message count at risk. The ADR 0102 vintage probe ([dr.py:397-453](../../../messagefoundry/pipeline/dr.py)) guards DR activation; this row extends the same posture to an operator-driven restore. **Never a silent overwrite** | | STORE-71 | A per-connection retention override naming an unknown Connection cannot silently purge the wrong feed | Negative/Security | pytest | dev-PC | x3 | T | P2 | A `connection_cutoffs` key (or per-connection `messages_days`) naming a Connection absent from the registry is rejected at load with a `ValidationError`/`ValueError` naming the key — it must **not** fall through to the global cutoff. A typo'd override therefore cannot blank a feed the operator meant to keep; the negative case (a correctly-named override) still purges only its own feed | -| STORE-72 | `rotate-key` with no retired key present refuses before writing anything | Negative/Security | pytest | dev-PC | x3 | T | P1 | With a **new** active key and `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` unset while the store already holds rows sealed under the prior key, `rotate-key` exits non-zero **before** re-encrypting any row and names the missing retired key (today's guard at [__main__.py:3462-3470](messagefoundry/__main__.py) only checks that an *active* key exists). A probe read proves at least one existing row is undecryptable under the active key alone; after the run, every row still decrypts under the prior key — zero rows re-sealed | -| STORE-73 | Removing a Connection that still has queued rows is refused or audited | Negative/Security | pytest | dev-PC | x3 | T | P2 | `remove_connection` ([connections_edit.py:202](messagefoundry/config/connections_edit.py)) against a Connection with undelivered `queue` rows either refuses with a count of the orphaned rows, or removes it only under an explicit `--force` that writes one audit row naming the actor and the abandoned row count. Either way the rows stay in the store and remain visible/replayable — **never** silently orphaned | +| STORE-72 | `rotate-key` with no retired key present refuses before writing anything | Negative/Security | pytest | dev-PC | x3 | T | P1 | With a **new** active key and `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` unset while the store already holds rows sealed under the prior key, `rotate-key` exits non-zero **before** re-encrypting any row and names the missing retired key (today's guard at [__main__.py:3462-3470](../../../messagefoundry/__main__.py) only checks that an *active* key exists). A probe read proves at least one existing row is undecryptable under the active key alone; after the run, every row still decrypts under the prior key — zero rows re-sealed | +| STORE-73 | Removing a Connection that still has queued rows is refused or audited | Negative/Security | pytest | dev-PC | x3 | T | P2 | `remove_connection` ([connections_edit.py:202](../../../messagefoundry/config/connections_edit.py)) against a Connection with undelivered `queue` rows either refuses with a count of the orphaned rows, or removes it only under an explicit `--force` that writes one audit row naming the actor and the abandoned row count. Either way the rows stay in the store and remain visible/replayable — **never** silently orphaned | ### 2.5 Detailed scenarios @@ -780,8 +780,8 @@ seven **C** rows (STORE-22, -23, -25, -28, -34, -36, -60) must have their measur MIG-09, which STORE-41 points at. 14. **Free-space signal.** Should the store gain a **free-space** threshold alongside `max_db_mb` (a `[retention].min_free_disk_mb`, or a percentage), given `disk_free_bytes` is `0` by - construction on PG / SQL Server ([postgres.py:6592](messagefoundry/store/postgres.py), - [sqlserver.py:9205](messagefoundry/store/sqlserver.py))? If it is SQLite-only, that asymmetry must + construction on PG / SQL Server ([postgres.py:6592](../../../messagefoundry/store/postgres.py), + [sqlserver.py:9205](../../../messagefoundry/store/sqlserver.py))? If it is SQLite-only, that asymmetry must be documented rather than silently shipped. *Blocks:* STORE-68, and the alert half of STORE-65. 15. **Destructive-operator posture.** For each case in STORE-70/-71/-72/-73 — older archive over a live store, an override naming an unknown Connection, `rotate-key` with no retired key, diff --git a/docs/testing/master-test-plan/04-high-availability-and-dr.md b/docs/testing/master-test-plan/04-high-availability-and-dr.md index 2a8d3c0a..82b91f66 100644 --- a/docs/testing/master-test-plan/04-high-availability-and-dr.md +++ b/docs/testing/master-test-plan/04-high-availability-and-dr.md @@ -11,7 +11,7 @@ This chapter covers the **active-passive HA** model MessageFoundry actually ships: N engine nodes over **one shared server-DB store** (PostgreSQL or SQL Server), exactly one of which holds a self-fencing `leader_lease` row and runs the wired graph. Concretely: -- **Coordinator seam** — `ClusterCoordinator` Protocol, `NullCoordinator` (single-node byte-identical default), `DbCoordinator` (asyncpg), `SqlServerCoordinator` (aioodbc / `MERGE WITH(HOLDLOCK)`), and `build_coordinator` dispatch ([`pipeline/cluster.py:137-311`](messagefoundry/pipeline/cluster.py), [`:326-983`](messagefoundry/pipeline/cluster.py), [`:986-1058`](messagefoundry/pipeline/cluster.py), [`pipeline/cluster_sqlserver.py:78-549`](messagefoundry/pipeline/cluster_sqlserver.py)). +- **Coordinator seam** — `ClusterCoordinator` Protocol, `NullCoordinator` (single-node byte-identical default), `DbCoordinator` (asyncpg), `SqlServerCoordinator` (aioodbc / `MERGE WITH(HOLDLOCK)`), and `build_coordinator` dispatch ([`pipeline/cluster.py:137-311`](../../../messagefoundry/pipeline/cluster.py), [`:326-983`](../../../messagefoundry/pipeline/cluster.py), [`:986-1058`](../../../messagefoundry/pipeline/cluster.py), [`pipeline/cluster_sqlserver.py:78-549`](../../../messagefoundry/pipeline/cluster_sqlserver.py)). - **Election + fencing** — acquire-or-renew in one statement against the DB clock (`cluster.py:840-900`), the DB-free monotonic fence watchdog (`cluster.py:902-935`), the H1 leader-epoch token and its store-side claim guard (`store/postgres.py:2686-2688, 2816-2820, 2934-2938`; `store/sqlserver.py:796-802, 6003-6007, 6152-6156, 6320`), clean release on stop (`cluster.py:937-962`), ADR 0096 leader preference / non-promotable standby (`cluster.py:871-888`, `config/settings.py:2896-2910`). - **Leader-gated work** — the graph supervisor (`pipeline/engine.py:1154-1268`), on-promotion in-flight recovery (`pipeline/leader_tasks.py:143-162` on PostgreSQL; `store.reset_stale_inflight()` on SQL Server, `engine.py:1177-1187`), the periodic expired-lease reclaim sweep (`leader_tasks.py:52-141`), leader-gated poll-source intake, and the leader-only WRITE singletons — retention, reclaim, DR backup (`engine.py:953-990`, `pipeline/dr_backup.py:257-259`). - **Observability** — `GET /cluster/status` and `GET /cluster/nodes` (`api/app.py:4643-4698`), the derived single-live-leader projection (`cluster.py:555-604`), `leadership_acquired` / `leadership_lost` alerts (`cluster.py:966-983`, `pipeline/alert_sinks.py:98-104, 807-824`), and the web console status page's cluster/nodes/DR tables and `leaderless → health=down` derivation (`messagefoundry_webconsole/pages/monitoring.py:471-508`, `routes/status.py:58-97`). diff --git a/docs/testing/master-test-plan/05-connections-and-transports.md b/docs/testing/master-test-plan/05-connections-and-transports.md index cb9faed3..050dd95b 100644 --- a/docs/testing/master-test-plan/05-connections-and-transports.md +++ b/docs/testing/master-test-plan/05-connections-and-transports.md @@ -5,23 +5,23 @@ ## 4. Connections & Transports Matrix **ID prefix:** `CONN` · **Surface:** engine (`messagefoundry/transports/`) with CI-leg, harness and W2025-box legs -· **Primary risk:** the shared frame decoder treats a start byte **inside an open frame** as ordinary payload ([`framing.py:102-126`](messagefoundry/transports/framing.py)), so a peer that aborts mid-frame and re-syncs by re-sending SB has two clinical messages silently concatenated into one — wrong clinical content downstream **and** a count-and-log record of one received message instead of two, with no error, no `ERROR` disposition and no connection event. +· **Primary risk:** the shared frame decoder treats a start byte **inside an open frame** as ordinary payload ([`framing.py:102-126`](../../../messagefoundry/transports/framing.py)), so a peer that aborts mid-frame and re-syncs by re-sending SB has two clinical messages silently concatenated into one — wrong clinical content downstream **and** a count-and-log record of one received message instead of two, with no error, no `ERROR` disposition and no connection event. ### 4.1 Scope & objectives This chapter covers **every registered inbound/outbound connector and the connection-level machinery around them**: -- **17 `ConnectorType` values** ([`config/models.py:28-53`](messagefoundry/config/models.py)) across **24 registrations** and **19 `ConnectionSpec` factories** in [`config/wiring.py`](messagefoundry/config/wiring.py) (`MLLP`, `Tcp`, `X12`, `Http`, `File`, `Timer`, `Loopback`, `PassThrough`, `Rest`, `FHIR`, `Email`, `Direct`, `DICOM`, `DICOMweb`, `Database`, `DatabasePoll`, `Soap`, `Sftp`, `Ftp`). -- **Wire-level framing** — the shared delimiter codec ([`transports/framing.py`](messagefoundry/transports/framing.py): `FrameCodec`, `FrameDecoder.feed:102`, `codec_for:143`) and its MLLP binding (`MLLPDecoder`, [`mllp.py:285`](messagefoundry/transports/mllp.py)); partial, coalesced, oversized, truncated and interleaved frames. +- **17 `ConnectorType` values** ([`config/models.py:28-53`](../../../messagefoundry/config/models.py)) across **24 registrations** and **19 `ConnectionSpec` factories** in [`config/wiring.py`](../../../messagefoundry/config/wiring.py) (`MLLP`, `Tcp`, `X12`, `Http`, `File`, `Timer`, `Loopback`, `PassThrough`, `Rest`, `FHIR`, `Email`, `Direct`, `DICOM`, `DICOMweb`, `Database`, `DatabasePoll`, `Soap`, `Sftp`, `Ftp`). +- **Wire-level framing** — the shared delimiter codec ([`transports/framing.py`](../../../messagefoundry/transports/framing.py): `FrameCodec`, `FrameDecoder.feed:102`, `codec_for:143`) and its MLLP binding (`MLLPDecoder`, [`mllp.py:285`](../../../messagefoundry/transports/mllp.py)); partial, coalesced, oversized, truncated and interleaved frames. - **MLLP** — source (`mllp.py:1299`), destination (`mllp.py:595`), TLS/mTLS (`_mllp_ssl_context:499`), ack modes (`build_ack:329`), ADR 0067 persistent (`mllp.py:643`), ADR 0124 `no_ack` delivery-on-write (`_send_once_no_ack:897`, `_send_persistent_no_ack:1046`), per-outbound encoding override (`:402`/`:427`). -- **Raw TCP** (`tcp.py:570-571`) and **X12 raw-TCP** (`x12.py:596-597`) — both permanently TLS-free by design ([ADR 0153](docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) decision 4 / BACKLOG #311 (above the published #231 baseline), `tcp.py:150-152`, which states the declaration is "PERMANENT, STRUCTURAL … not a transitional one"). -- **File** (`file.py:891-892`) incl. ADR 0123 gzip, ADR 0129 `after_read='leave'` + `ProcessedFileLedger` ([`base.py:65-88`](messagefoundry/transports/base.py)), ADR 0132 alternate Windows credential ([`wincred.py`](messagefoundry/transports/wincred.py)); **RemoteFile** SFTP/FTP/FTPS (`remotefile.py:1016-1017`). -- **HTTP family** — `Rest` (`rest.py:1544`), `Soap` (`soap.py:844`, WS-Security/mTLS ADR 0015, WSDL import ADR 0122), `FHIR` (`fhir.py:647`) + SMART Backend Services ([`smart.py`](messagefoundry/transports/smart.py)), `DICOMweb` STOW-RS (`dicomweb.py:382`), generic outbound auth ([`http_auth.py`](messagefoundry/transports/http_auth.py)), ADR 0126 forward proxy. +- **Raw TCP** (`tcp.py:570-571`) and **X12 raw-TCP** (`x12.py:596-597`) — both permanently TLS-free by design ([ADR 0153](../../adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) decision 4 / BACKLOG #311 (above the published #231 baseline), `tcp.py:150-152`, which states the declaration is "PERMANENT, STRUCTURAL … not a transitional one"). +- **File** (`file.py:891-892`) incl. ADR 0123 gzip, ADR 0129 `after_read='leave'` + `ProcessedFileLedger` ([`base.py:65-88`](../../../messagefoundry/transports/base.py)), ADR 0132 alternate Windows credential ([`wincred.py`](../../../messagefoundry/transports/wincred.py)); **RemoteFile** SFTP/FTP/FTPS (`remotefile.py:1016-1017`). +- **HTTP family** — `Rest` (`rest.py:1544`), `Soap` (`soap.py:844`, WS-Security/mTLS ADR 0015, WSDL import ADR 0122), `FHIR` (`fhir.py:647`) + SMART Backend Services ([`smart.py`](../../../messagefoundry/transports/smart.py)), `DICOMweb` STOW-RS (`dicomweb.py:382`), generic outbound auth ([`http_auth.py`](../../../messagefoundry/transports/http_auth.py)), ADR 0126 forward proxy. - **DICOM DIMSE** — C-STORE SCP source (`dicom.py:719`) and C-STORE SCU + C-ECHO destination (`dicom.py:689`/`:720`). - **DATABASE** destination + DB-IN poll source (`database.py:947-948`) and the read-only `db_lookup` carve-out (ADR 0010); the `fhir_lookup` carve-out (ADR 0043, `fhir.py:991`). - **Email/SMTP** (`email.py:265`, ADR 0029) and **Direct S/MIME-over-SMTP** (`direct.py:373`, ADR 0085 PR1). - **Inbound HTTP/1.1 listener** (`http_listener.py:488`, ADR 0023) and the internal/generative sources: `Timer` (`timer.py:341`), `Loopback` (`loopback.py:53`), `PassThrough` (`passthrough.py:68`). -- **Connection lifecycle + operator seams** — the nine `/connections/{name}/…` routes ([`api/app.py:1915,1924,1933,1944,1973,2023,2072,2141,2294`](messagefoundry/api/app.py)), the 17 `test_connection` reachability overrides against the `TestNotSupportedError` default (`base.py:288`/`:327`), the eight-arm fail-closed egress allow-list ([`settings.py` `EgressSettings`](messagefoundry/config/settings.py): `allowed_mllp`, `allowed_tcp`, `allowed_file_dirs`, `allowed_http`, `allowed_db`, `allowed_remote`, `allowed_smtp`, `allowed_direct`, plus `deny_by_default`), `connections.toml` round-trip, connection-level fault isolation, the ADR 0021 §7 connection-event log and ACK capture, and the TLS cert-expiry monitor ([`pipeline/cert_expiry.py:98-130`](messagefoundry/pipeline/cert_expiry.py)). +- **Connection lifecycle + operator seams** — the nine `/connections/{name}/…` routes ([`api/app.py:1915,1924,1933,1944,1973,2023,2072,2141,2294`](../../../messagefoundry/api/app.py)), the 17 `test_connection` reachability overrides against the `TestNotSupportedError` default (`base.py:288`/`:327`), the eight-arm fail-closed egress allow-list ([`settings.py` `EgressSettings`](../../../messagefoundry/config/settings.py): `allowed_mllp`, `allowed_tcp`, `allowed_file_dirs`, `allowed_http`, `allowed_db`, `allowed_remote`, `allowed_smtp`, `allowed_direct`, plus `deny_by_default`), `connections.toml` round-trip, connection-level fault isolation, the ADR 0021 §7 connection-event log and ACK capture, and the TLS cert-expiry monitor ([`pipeline/cert_expiry.py:98-130`](../../../messagefoundry/pipeline/cert_expiry.py)). - **Partner-side failure injection** — slow peer, accept-and-never-ACK, half-open socket, abortive RST, cert expiry. **Explicitly NOT in scope here** — cited, never re-planned: diff --git a/docs/testing/master-test-plan/07-config-wiring-and-cli.md b/docs/testing/master-test-plan/07-config-wiring-and-cli.md index f401f642..51723f09 100644 --- a/docs/testing/master-test-plan/07-config-wiring-and-cli.md +++ b/docs/testing/master-test-plan/07-config-wiring-and-cli.md @@ -11,7 +11,7 @@ This chapter covers the **configuration bundle and the command-line surface that reads, writes and gates it**: -- **Code-first wiring loader + `Registry`** — `load_config` ([`messagefoundry/config/wiring.py:3793`](messagefoundry/config/wiring.py)), `validate_config` (`:4213`), `Registry` (`:2779`), the `_*` helper-module skip and the `_SiblingHelperFinder` restricted to `_`-prefixed names (`:3729`, SEC-019 / CWE-427). +- **Code-first wiring loader + `Registry`** — `load_config` ([`messagefoundry/config/wiring.py:3793`](../../../messagefoundry/config/wiring.py)), `validate_config` (`:4213`), `Registry` (`:2779`), the `_*` helper-module skip and the `_SiblingHelperFinder` restricted to `_`-prefixed names (`:3729`, SEC-019 / CWE-427). - **Config-source trust** — POSIX group/world-writable + foreign-owner refusal (`wiring.py:4147`, `:4124`) and the Windows NTFS-DACL guard (`_evaluate_config_dacl:3889`, `_assert_safe_config_source_windows:3917`, ADR 0036), plus the `MEFOR_ALLOW_INSECURE_CONFIG_SOURCE` dev escape (`settings.py:244/247`). - **`connections.toml` config-as-data (ADR 0007)** — the read path (`connections_file.py:156/273`, `_TRANSPORTS:70`), the comment-preserving write path (`connections_edit.py:161/202/349/363`), the `connection list|upsert|remove|schema` CLI (`__main__.py:3788`), the `connection schema` form contract (`connection_schema.py:56`), and the console→TOML `flagged` write seam (ADR 0007 amendment). - **Service settings** — precedence CLI > env(`MEFOR_*`) > toml > default (`settings.py:4128`), the `MEFOR_<SECTION>_<KEY>` parser (`:3777`) against `_SECTIONS` (`:109`), the ADR 0118 `[security]` desugar + relocated-key rejection (`:3861`, `:3848`), secrets-in-file warning (`:3789`), and `extra="ignore"` tolerance (`ServiceSettings:3623`). diff --git a/docs/testing/master-test-plan/08-publishing-and-promotion.md b/docs/testing/master-test-plan/08-publishing-and-promotion.md index 09dcd431..0fa6d1fa 100644 --- a/docs/testing/master-test-plan/08-publishing-and-promotion.md +++ b/docs/testing/master-test-plan/08-publishing-and-promotion.md @@ -12,29 +12,29 @@ This chapter owns **the whole path by which authored configuration reaches a running engine**, and specifically the **non-production vs production** distinction. Concretely: -- **The IDE Stage → Promote flow** — [`ide/src/promote.ts`](../../ide/src/promote.ts) (validate → +- **The IDE Stage → Promote flow** — [`ide/src/promote.ts`](../../../ide/src/promote.ts) (validate → environment pick → engine-shard pick → host policy → env-aware dry-run pre-flight → modal confirm → - apply), [`ide/src/promoteTarget.ts`](../../ide/src/promoteTarget.ts) (pure target resolution), - [`ide/src/engineTarget.ts`](../../ide/src/engineTarget.ts) (SEC-005 host policy). + apply), [`ide/src/promoteTarget.ts`](../../../ide/src/promoteTarget.ts) (pure target resolution), + [`ide/src/engineTarget.ts`](../../../ide/src/engineTarget.ts) (SEC-005 host policy). - **The engine reload contract** — `POST /config/reload` - ([`messagefoundry/api/app.py:2741-2891`](../../messagefoundry/api/app.py)), reload-root confinement - (`Engine._resolve_reload_target`, [`engine.py:1508-1521`](../../messagefoundry/pipeline/engine.py)), + ([`messagefoundry/api/app.py:2741-2891`](../../../messagefoundry/api/app.py)), reload-root confinement + (`Engine._resolve_reload_target`, [`engine.py:1508-1521`](../../../messagefoundry/pipeline/engine.py)), `config:deploy` + `require_step_up`, ADR 0041 D2 dual-control, and the quiesce-and-swap in - `RegistryRunner.reload` ([`wiring_runner.py:3046-3184`](../../messagefoundry/pipeline/wiring_runner.py)). + `RegistryRunner.reload` ([`wiring_runner.py:3046-3184`](../../../messagefoundry/pipeline/wiring_runner.py)). - **The web console config-deploy page** — `/ui/config` + - [`messagefoundry_webconsole/routes/config.py`](../../messagefoundry_webconsole/routes/config.py) + [`messagefoundry_webconsole/routes/config.py`](../../../messagefoundry_webconsole/routes/config.py) (fixed `ReloadRequest(config_dir=None, dry_run=False)`) and the provenance badge in - [`pages/config.py`](../../messagefoundry_webconsole/pages/config.py). + [`pages/config.py`](../../../messagefoundry_webconsole/pages/config.py). - **Attestation & attribution** — ADR 0041 D1 content fingerprint - ([`config/fingerprint.py`](../../messagefoundry/config/fingerprint.py)), `GET /config/provenance`, + ([`config/fingerprint.py`](../../../messagefoundry/config/fingerprint.py)), `GET /config/provenance`, the `config_reload*` audit family, and ADR 0041 D3 wheel self-attestation - ([`integrity.py`](../../messagefoundry/integrity.py)) where it interacts with a publish. + ([`integrity.py`](../../../messagefoundry/integrity.py)) where it interacts with a publish. - **Environment values and target isolation** — `environments/<env>.toml` + `MEFOR_VALUE_*` - ([`config/environments.py`](../../messagefoundry/config/environments.py)), ADR 0050 project-root + ([`config/environments.py`](../../../messagefoundry/config/environments.py)), ADR 0050 project-root anchoring, deferred `env()` resolution *on the target*, and wrong-target / environment-crossing safety. - **Multi-target publishing** — several named `messagefoundry.environments` entries, engine-shard sub-targets within one environment, and cluster config-version convergence - ([`pipeline/config_convergence.py`](../../messagefoundry/pipeline/config_convergence.py)). + ([`pipeline/config_convergence.py`](../../../messagefoundry/pipeline/config_convergence.py)). - **Atomicity, partial publish, rollback, and behaviour of in-flight messages / open connections across a publish**, including publishing during a cluster failover. - **Per-artifact-kind publish semantics** — Router/Handler `*.py`, `connections.toml`, `codesets/*`, @@ -42,7 +42,7 @@ specifically the **non-production vs production** distinction. Concretely: config and AI policy in `messagefoundry.toml`. - **Version control as the delivery mechanism** — [`docs/VERSION-CONTROL.md`](../VERSION-CONTROL.md), the offline git init + `messagefoundry check` pre-commit hook - ([`ide/src/sourceControl.ts`](../../ide/src/sourceControl.ts)), and the air-gapped `git bundle` path. + ([`ide/src/sourceControl.ts`](../../../ide/src/sourceControl.ts)), and the air-gapped `git bundle` path. **Explicitly NOT in scope here (owned elsewhere — cited, not restated):** diff --git a/docs/testing/master-test-plan/09-engine-api.md b/docs/testing/master-test-plan/09-engine-api.md index 0530456c..aa2a429f 100644 --- a/docs/testing/master-test-plan/09-engine-api.md +++ b/docs/testing/master-test-plan/09-engine-api.md @@ -13,7 +13,7 @@ harness at runtime, with no failing test on either side. ### 8.1 Scope & objectives This chapter covers the single FastAPI application built by -`create_app()` / `create_managed_app()` ([`messagefoundry/api/app.py:1018`](messagefoundry/api/app.py), +`create_app()` / `create_managed_app()` ([`messagefoundry/api/app.py:1018`](../../../messagefoundry/api/app.py), `:5156`) — verified live at **105 route objects** (104 `APIRoute` + the `/ws/stats` `APIWebSocketRoute`; 67 declared in `api/app.py`, 38 in `api/auth_routes.py`), **109** with `expose_docs=True`, **203** with `serve_ui=True`. @@ -24,7 +24,7 @@ In scope: Pydantic models (`api/models.py` 89 + `api/auth_models.py` 32); OpenAPI drift; cross-language mirrors (`apiclient/client.py`, the IDE's TypeScript DTOs, the tray's `/health` key literal). - **Deny-by-default authorization plumbing** as it is *enforced on the wire*: the nine factories in - [`api/security.py`](messagefoundry/api/security.py) — `require` (:182), `require_paced` (:250), + [`api/security.py`](../../../messagefoundry/api/security.py) — `require` (:182), `require_paced` (:250), `require_service_cert` (:430), `require_phi_read` (:504), `require_step_up` (:569), `require_reauth_only` (:609), `require_step_up_action` (:651), `require_reauth_only_action` (:694), `authorize_ws` (:768) — plus `enforce_phi_read_hop` (:480), `enforce_phi_read_pacing` (:523), diff --git a/docs/testing/master-test-plan/11-web-console.md b/docs/testing/master-test-plan/11-web-console.md index 592efcee..a936edd5 100644 --- a/docs/testing/master-test-plan/11-web-console.md +++ b/docs/testing/master-test-plan/11-web-console.md @@ -14,14 +14,14 @@ ever drives the console through a real browser, a real TLS listener, or a real r This chapter covers the same-origin, server-rendered browser operations console `messagefoundry_webconsole` — grafted onto the engine's FastAPI app by -`mount_ui` ([`messagefoundry_webconsole/mount.py:69`](messagefoundry_webconsole/mount.py)), ADR 0065 / +`mount_ui` ([`messagefoundry_webconsole/mount.py:69`](../../../messagefoundry_webconsole/mount.py)), ADR 0065 / ADR 0143. It is the **sole** operator console; the PySide6 desktop console was retired (BACKLOG #103) and PySide6 now backs only the standalone test harness. ~11.2k LOC of Python plus 1,506 lines of `app.js` and 345 lines of `app.css`, shipped as a separately-versioned second wheel (`packaging/messagefoundry-webconsole/`) with its own ~347-test suite run as a second CI step. **In scope.** Every page and flow of the 98-route `/ui` surface pinned at -[`packaging/messagefoundry-webconsole/tests/golden/ui_routes.txt`](packaging/messagefoundry-webconsole/tests/golden/ui_routes.txt): +[`packaging/messagefoundry-webconsole/tests/golden/ui_routes.txt`](../../../packaging/messagefoundry-webconsole/tests/golden/ui_routes.txt): login and provider select, Kerberos/SPNEGO SSO, OIDC federated login, TOTP MFA lifecycle, WebAuthn passkeys, must-change-password and `/ui/mfa` confinement; the connections dashboard, live fragment poll and `/ws/stats` server-rendered enrichment, bulk connection control, per-connection and bulk statistics @@ -43,18 +43,18 @@ performance and pagination at scale, and concurrent operators. | Area | Owner | |---|---| -| The 32-row `/ui` coverage-gap audit (rows `FCP:UI-1`..`FCP:UI-32`, six dimensions) | [`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) §23, lines 1469–1517. Cited, not restated. Open rows: **FCP:UI-8**, **FCP:UI-23**, **FCP:UI-32**; **FCP:UI-12 closed 2026-07-13** (P2 STATUS block, `:142-152`). (`FCP:` marks a FEATURE-COVERAGE-PLAN ID; a bare `WEB-nn` is always this plan's own row.) | -| Package architecture, the three-layer seam handshake, the version-skew gate, dev-and-test instructions | [`docs/WEBCONSOLE-PACKAGE.md`](docs/WEBCONSOLE-PACKAGE.md) | +| The 32-row `/ui` coverage-gap audit (rows `FCP:UI-1`..`FCP:UI-32`, six dimensions) | [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) §23, lines 1469–1517. Cited, not restated. Open rows: **FCP:UI-8**, **FCP:UI-23**, **FCP:UI-32**; **FCP:UI-12 closed 2026-07-13** (P2 STATUS block, `:142-152`). (`FCP:` marks a FEATURE-COVERAGE-PLAN ID; a bare `WEB-nn` is always this plan's own row.) | +| Package architecture, the three-layer seam handshake, the version-skew gate, dev-and-test instructions | [`docs/WEBCONSOLE-PACKAGE.md`](../../WEBCONSOLE-PACKAGE.md) | | Config-deploy semantics, promote/stage, provenance | the **PUB** chapter (this chapter tests only the console's *rendering and gating* of config reload) | -| Windows Server 2025 host / service-identity acceptance | [`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](docs/testing/WIN2025-TEST-MATRIX.md) — but note: **neither carries a single `/ui` row** (verified: zero matches for `/ui`, "web console", "webconsole"). WEB-57/WEB-59 below add them (`W25:` marks a WIN2025 / acceptance-matrix ID); the *host* setup stays theirs. | -| Pipeline throughput | [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md). This chapter adds only console-induced load and the engine-throughput delta it causes. | +| Windows Server 2025 host / service-identity acceptance | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md) — but note: **neither carries a single `/ui` row** (verified: zero matches for `/ui`, "web console", "webconsole"). WEB-57/WEB-59 below add them (`W25:` marks a WIN2025 / acceptance-matrix ID); the *host* setup stays theirs. | +| Pipeline throughput | [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md). This chapter adds only console-induced load and the engine-throughput delta it causes. | | The VS Code IDE extension (Steps view, connections graph, ADR 0076/0091/0103) | the **IDE** chapter | | Engine JSON API authz/PHI semantics | the **API** chapter. The console calls the same handlers through `UiDeps`; this chapter tests the `/ui` *gate* and *projection* on top. | **Recon corrections made during authoring.** 1. The golden route/write-action files live at `packaging/messagefoundry-webconsole/tests/golden/`, **not** `tests/golden/` — `tests/golden/` holds only `webconsole_seam.snapshot`. -2. "Console default-ON" is narrower than stated: default-ON applies to **loopback binds only**. A *non-explicit* default-on console on an exposed bind **auto-degrades to JSON-only with a warning** ([`messagefoundry/__main__.py:1703-1727`](messagefoundry/__main__.py)); only an explicit `[security].serve_web_console=true` reaches the exposure ladder at `:1734-1827`. -3. The `[security]` public-origin key is `web_console_public_address` (aliased to `[api].public_origin`, [`config/settings.py:3479,3815`](messagefoundry/config/settings.py)) — the CLI messages name the `[security]` form. +2. "Console default-ON" is narrower than stated: default-ON applies to **loopback binds only**. A *non-explicit* default-on console on an exposed bind **auto-degrades to JSON-only with a warning** ([`messagefoundry/__main__.py:1703-1727`](../../../messagefoundry/__main__.py)); only an explicit `[security].serve_web_console=true` reaches the exposure ladder at `:1734-1827`. +3. The `[security]` public-origin key is `web_console_public_address` (aliased to `[api].public_origin`, [`config/settings.py:3479,3815`](../../../messagefoundry/config/settings.py)) — the CLI messages name the `[security]` form. 4. Simulated-scheme ASGI clients live in `test_ui_hardening.py:34-37`, not `conftest.py:37` (which is the shared `engine` fixture). 5. `app.js` **does** set `aria-sort` (`app.js:708`) and the sort trigger **is** a native `<button>` (`app.js:794-804`) — keyboard-operable. What has no keyboard path is column **resize** (`app.js:832-863`, pointer events only) and column **reorder** (`app.js:880-911`, native HTML5 drag only). There is not one `keydown` listener in the file. 6. Page-builder approval line numbers: `pages/config.py:96-105`, `pages/connections.py:207-212`, `pages/messages.py:676-685`. @@ -134,7 +134,7 @@ another chapter, this chapter scopes no separate work, and the row exists only s coverage stays legible end-to-end. A pointer row keeps its ID, carries Method `—` and Cls **T**, and gates through its owner's row, not a second time here. -**Foreign IDs are prefixed.** `FCP:` = a [`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) +**Foreign IDs are prefixed.** `FCP:` = a [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) gap ID (`FCP:UI-8`), `W25:` = a WIN2025 / `harness/acceptance/matrix.py` row (`W25:A7`). A bare `WEB-nn` is always this plan's own row; unprefixed cross-chapter IDs (`MIG-28`, `TRAY-67`) are other chapters of *this* plan. diff --git a/docs/testing/master-test-plan/15-alerting-and-observability.md b/docs/testing/master-test-plan/15-alerting-and-observability.md index 5555e46c..608c7607 100644 --- a/docs/testing/master-test-plan/15-alerting-and-observability.md +++ b/docs/testing/master-test-plan/15-alerting-and-observability.md @@ -28,28 +28,28 @@ control actions; the emit sites in `wiring_runner.py`, `stage_dispatcher.py`, `c `secret_rotation.py`, `update_check.py`, `gcm_invocations.py`, `retention.py`, `dr_backup.py`, `dr.py`, `cluster.py`, `cluster_sqlserver.py`, `integrity.py`, `engine.py`, `reference_sync.py`, `state_convergence.py`; `POST /alerts/test-email`; `GET /alerts/rules`; the -`messagefoundry alert list|add|remove` CLI + [`config/alerts_edit.py`](messagefoundry/config/alerts_edit.py); +`messagefoundry alert list|add|remove` CLI + [`config/alerts_edit.py`](../../../messagefoundry/config/alerts_edit.py); the web console `/ui/alerts` page + nav bell; the VS Code alert editor -([`ide/src/alertEditor.ts`](ide/src/alertEditor.ts)); alert-storm behaviour; alert delivery across a +([`ide/src/alertEditor.ts`](../../../ide/src/alertEditor.ts)); alert-storm behaviour; alert delivery across a failover; and **PHI leakage in alert payloads (metadata-only by contract)**. **In scope — Observability.** Prometheus `GET /metrics` and the OpenTelemetry seam -([`api/metrics.py`](messagefoundry/api/metrics.py)) — **`/metrics` label cardinality and scrape cost +([`api/metrics.py`](../../../messagefoundry/api/metrics.py)) — **`/metrics` label cardinality and scrape cost are owned here** (ALERT-40 / ALERT-61; the API chapter's `API-54` is a pointer row to them, no separate work scoped there); `GET /stats`, `GET /metrics/history`, `GET /graph/edges`, `POST /statistics/reset` and the `/ws/stats` socket; stdlib logging -([`logging_setup.py`](messagefoundry/logging_setup.py)) — the three PHI filters, levels, JSON format, +([`logging_setup.py`](../../../messagefoundry/logging_setup.py)) — the three PHI filters, levels, JSON format, syslog UDP/TCP/TLS off-box forwarding, NSSM capture + `AppRotateBytes` rotation; runtime verbosity (`GET`/`PATCH /logging/level`) and the redacted `GET /logs/tail` (ADR 0130); the support bundle -([`support/bundle.py`](messagefoundry/support/bundle.py), [`support/redact.py`](messagefoundry/support/redact.py)); -Windows crash-dump suppression ([`crashdump.py`](messagefoundry/crashdump.py), ADR 0152); the +([`support/bundle.py`](../../../messagefoundry/support/bundle.py), [`support/redact.py`](../../../messagefoundry/support/redact.py)); +Windows crash-dump suppression ([`crashdump.py`](../../../messagefoundry/crashdump.py), ADR 0152); the `connection_event` log (ADR 0021 §7); the cluster observability API (ADR 0008); and the cross-cutting question **can an operator actually detect each failure mode the other chapters inject**. **ID convention (plan-wide).** A bare `ALERT-nn` / `G-nn` in this chapter is **this plan's own row**. Every reference to another document's ID carries a prefix: **`FCP:`** for a -[`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) gap ID and **`W25:`** +[`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) gap ID and **`W25:`** for a WIN2025 plan/matrix test ID. The collision is real, not hypothetical: FEATURE-COVERAGE-PLAN §19 has its own `FCP:ALERT-1..FCP:ALERT-24` (different subjects from ours) and the WIN2025 matrix has a `W25:G4` that is not this chapter's G4 risk. @@ -58,11 +58,11 @@ has its own `FCP:ALERT-1..FCP:ALERT-24` (different subjects from ours) and the W | Area | Owner — cite, do not restate | |---|---| -| The 24-row subsystem coverage-gap audit `FCP:ALERT-1..FCP:ALERT-24` (six dimensions, per-feature verdicts) | [`docs/testing/FEATURE-COVERAGE-PLAN.md` §19, lines 1300-1341](docs/testing/FEATURE-COVERAGE-PLAN.md) — this chapter **supersedes four stale rows** (see 14.2) and otherwise inherits it | -| The PHI posture of every log/alert stream (14 rows: format, sink, ACL, retention, PHI class) | [`docs/PHI.md` §7 "Logging inventory (16.1.1 / 16.2.3)", rows 1-14](docs/PHI.md) — CI-guarded by `tests/test_phi_logging_inventory.py` | -| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](docs/testing/WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](harness/acceptance/matrix.py:454)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | -| On-box deployment acceptance (`messagefoundry verify` host/store/smoke/manual/federation sections) | [`docs/testing/VERIFY.md`](docs/testing/VERIFY.md) — it has no alerting or logging section; nothing here duplicates it | -| Load generation mechanics, profiles, governor, SLO verdicts | [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md) — this chapter only *consumes* the `connscale` / `sustained-overload` profiles | +| The 24-row subsystem coverage-gap audit `FCP:ALERT-1..FCP:ALERT-24` (six dimensions, per-feature verdicts) | [`docs/testing/FEATURE-COVERAGE-PLAN.md` §19, lines 1300-1341](../FEATURE-COVERAGE-PLAN.md) — this chapter **supersedes four stale rows** (see 14.2) and otherwise inherits it | +| The PHI posture of every log/alert stream (14 rows: format, sink, ACL, retention, PHI class) | [`docs/PHI.md` §7 "Logging inventory (16.1.1 / 16.2.3)", rows 1-14](../../PHI.md) — CI-guarded by `tests/test_phi_logging_inventory.py` | +| Windows Server 2025 host/service-identity acceptance (NSSM install, gMSA, ACLs, reboot autostart) | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md). **Note:** the whole WIN2025 estate contains exactly one alerting row — matrix row `W25:G4` "/cluster observability + alerts + dead-letters page", claimed ONCE, `Coverage.PYTEST`, delegated to `tests/test_cluster.py` + `tests/test_alert_rules.py` ([`harness/acceptance/matrix.py:454-462`](harness/acceptance/matrix.py:454)). The WIN2025 plan itself has **zero** occurrences of "alert". The host-side alerting rows below are therefore **new**, not duplicates | +| On-box deployment acceptance (`messagefoundry verify` host/store/smoke/manual/federation sections) | [`docs/testing/VERIFY.md`](../VERIFY.md) — it has no alerting or logging section; nothing here duplicates it | +| Load generation mechanics, profiles, governor, SLO verdicts | [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) — this chapter only *consumes* the `connscale` / `sustained-overload` profiles | | Cluster leadership/lease/failover mechanics themselves | the HA/DR chapter — here only the **alerting behaviour across** a leadership move | | ADR 0020 raw-frame Protocol Data / Protocol Text capture | **declined** — ADR 0020 is "Superseded (raw-frame scope) 2026-07-13"; the metadata subset shipped as the ADR 0021 §7 `connection_event` log. The supersede decision is already carried in FEATURE-COVERAGE-PLAN `FCP:ALERT-22`/`FCP:MLLP-22` (lines 371/400/537). No test rows | @@ -134,7 +134,7 @@ ALERT-13 and ALERT-11. |---|---|---|---|---| | **G1 — three alert vocabularies bound only by convention** | An emit site's `type` literal, `_ALERT_EVENT_TYPES`, and the `AlertSink` Protocol / `LoggingAlertSink` / `NotifierAlertSink` method sets drift. A type absent from the routable set can never be escalated, routed, suppressed or muted by any operator rule; a method present only on `NotifierAlertSink` raises `AttributeError` on the logging fallback | Every deployment. Drift has fired **twice** already: `lane_stuck` + `rcsi_off_degraded` once shipped unroutable — **both are members today** ([`settings.py:2515-2516`](messagefoundry/config/settings.py:2515)), rule-targetable end to end at `test_alert_rules.py:357/370/395`, so the only live drift is that no guard stops the next one; `content_match` is **still** divergent (only on `NotifierAlertSink` at [`alert_sinks.py:676`](messagefoundry/pipeline/alert_sinks.py:676), absent from the Protocol and the fallback). Note the non-obvious third form: `AlertSink.saturation_rising()` emits type `"saturation"` — method name ≠ event type, so a naive guard would false-fail | **No.** No guard exists in either direction | **P0** | | **G2 — `content_match` has no Handler-reachable surface** | ADR 0133 AC-3 says "WHEN a Handler emits a `content_match`" — but there is no export in `messagefoundry/__init__.py` (unlike `db_lookup`/`fhir_lookup` at lines 32-33/150/153), no injected sink on a Handler context, and no dry-run path. Every test calls `sink.content_match(...)` directly | The differentiating Corepoint "Action Point" parity capability is unusable in practice; the PHI-free-by-contract guarantee has never been exercised through real Handler code | **No** — AC-3/AC-4 pass at the sink so nothing fails | **P0** | -| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](messagefoundry/pipeline/alert_sinks.py:100)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | +| **G3 — `connection_started` is mapped but emitted nowhere** | `_AUTO_RESOLVE["connection_started"] = "connection_stopped"` ([`alert_sinks.py:100`](messagefoundry/pipeline/alert_sinks.py:100)) but a repo-wide search finds **no emit site**. A lane that STOPs on an internal error and is later restarted (by hand or by a #144 `control_action`) leaves its `alert_instance` permanently `open` | `alerts_active` on the connections dashboard stays non-zero forever; `/alerts/active` accumulates; the nav bell's `list_active_alerts(limit=200)` ([`webconsole/routes/status.py:166`](../../../messagefoundry_webconsole/routes/status.py)) saturates. Textbook alert fatigue — the operator learns to ignore the list and the next real stop is missed | **Partially** — `tests/test_phi_logging_inventory.py:439` *documents* the dead key; nothing asserts the operator consequence | **P0** | | **G4 — the alert/security-notify SMTP hop is encrypted but unauthenticated** | `send_plain_email` calls `smtp.starttls()` with **no** `SSLContext` ([`alert_sinks.py:384`](messagefoundry/pipeline/alert_sinks.py:384)), so the stdlib default applies (`check_hostname=False`, `verify_mode=CERT_NONE`). The same transport carries the ASVS 6.3.5/6.3.7 per-user security notifications | A MITM on the SMTP hop reads every alert body (connection names, queue shapes, `safe_exc` detail strings) and every account-security notification (username, source IP, failed-attempt count) — and can suppress them | **No, and the docs disagree with themselves.** `docs/PHI.md` row 11 states the caveat plainly; `docs/BACKLOG.md:5152` claims the sink "uses STARTTLS with a verifying context by design". No test pins either | **P0** | | **G5 — `[alerts]` is startup-only** | `app.state.alerts_settings` is assigned only at app construction ([`api/app.py:1120`](messagefoundry/api/app.py:1120)) and lifespan startup (`:5485`). `POST /config/reload` (`:2741`) re-runs the `--config` graph, never the service-settings TOML | An operator adds a suppression rule mid-incident via the IDE or `messagefoundry alert add`; the IDE re-lists from the **file** and shows it; `/alerts/rules` still shows the **startup** set; the running notifier keeps paging until a restart. The requirement is documented only in `alerts_edit.py:19-21` and the CLI docstring — nowhere an operator looks | **No** | **P0** | | **G6 — IDE alert editor offers 4 of the 18 event types and 7 of the 15 fields** | [`ide/src/alertEditor.ts:13-19`](ide/src/alertEditor.ts:13) offers a 5-entry dropdown — `any` plus only `connection_stopped`/`queue_buildup`/`storage_threshold`/`cert_expiry`; `:25-32` supports only `event_type`/`connection`/`min_depth`/`min_oldest_seconds`/`severity`/`transports`/`cooldown_seconds` (7 of the 15 `AlertRule` fields). `ide/src/test/suite/` has **no** alert test file (35 suites, none for the alert editor) | An operator on the supported GUI authoring path cannot rule on **14 of the 18** signals, nor set `id`, `recipients`, `mute`, `escalate`, `schedule`, `content_label`, `control_action`, `control_target` | **No** — the `ide` CI leg runs and tests nothing here | **P1** | @@ -153,7 +153,7 @@ ALERT-13 and ALERT-11. | **G19 — `/metrics` has no cardinality or scrape-cost bound** | `gather_snapshot` does per-scrape store reads; exporter tests use a handful of connections | A 200+ connection estate on a 15 s Prometheus scrape adds measurable store load, and an unbounded label set (connection × destination × status) can blow up a TSDB | **No** | **P2** | | **G20 — off-box log forwarding has no live-collector round-trip** | `tests/test_logging.py:371-660` covers handler construction, TLS context, CA anchoring, client cert and unreachable-collector tolerance — nothing asserts a record **arrives**, correctly framed, at a collector | This is the SIEM evidence path (`docs/PHI.md` rows 3/4). An RFC 5425 length-prefix or framing regression means audit rows silently never reach the SIEM while the engine reports nothing wrong | **No** | **P2** | | **G21 — the notifier is constructed only on the `serve` path** | `notifier_from_settings` is called once, in the `create_serve_app` lifespan ([`api/app.py:5316-5318`](messagefoundry/api/app.py:5316)). Any other engine path (embedded `Engine`, `messagefoundry check`, dryrun, most tests) silently runs on `LoggingAlertSink` | A future deployment mode or a lifespan reorder would leave a production engine with no webhook/email notifier and only `WARNING` log lines | **No** — nothing asserts the runner's sink identity after a `serve` lifespan | **P2** | -| **G22 — no bounded-latency assertion that a new alert reaches an operator surface** | The nav bell + `/ui/alerts` are poll-driven (`/ui/nav-status` ~every 15 s, [`webconsole/routes/status.py:128`](messagefoundry_webconsole/routes/status.py)); `/ws/stats` pushes queue counts, **not** alerts | An operator watching the console can be a poll interval behind a stopped connection with no indication the view is stale | **No** — the webconsole tests assert rendering and RBAC, not freshness | **P2** | +| **G22 — no bounded-latency assertion that a new alert reaches an operator surface** | The nav bell + `/ui/alerts` are poll-driven (`/ui/nav-status` ~every 15 s, [`webconsole/routes/status.py:128`](../../../messagefoundry_webconsole/routes/status.py)); `/ws/stats` pushes queue counts, **not** alerts | An operator watching the console can be a poll interval behind a stopped connection with no indication the view is stale | **No** — the webconsole tests assert rendering and RBAC, not freshness | **P2** | | **G23 — three ledger/catalog documents lie about this area** | `docs/FEATURE-MAP.md` §9 lists 8 rows and omits ADR 0044 alert state, escalation, templates, per-rule recipients, control actions, **16 of the 18** event types (§9 names only `connection_stopped` and `queue_buildup`), the support bundle, crashdump suppression, the `connection_event` log, `/logs/tail`, `/metrics/history` and host metrics; §10 is still titled "Surfaces — Admin Console (PySide6)" with an Alerts page row at `:172`, and `:131` asserts "The PySide6 desktop console stays (additive)" though `messagefoundry/console/` does not exist. `docs/BACKLOG.md` #171 (lines 5768-5780) still banners DEMAND-GATE and states "there is no runtime/per-area verbosity control and no interactive in-console log viewer" — both are BUILT | FEATURE-MAP reaches the **public mirror** (`tests/test_feature_map_claims.py:3`). It understates the shipped alerting surface and overstates a retired one; the BACKLOG lies about build state — exactly what `backlog-hygiene.yml` exists to prevent | **No** — the existing guard tests links and ASVS score claims, not row currency | **P2** | ### 14.4 Test matrix diff --git a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md index a016dd35..edb5d2af 100644 --- a/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md +++ b/docs/testing/master-test-plan/16-security-phi-and-supply-chain.md @@ -16,7 +16,7 @@ prove a guard actually **refuses**, rather than asserting it exists. It covers: - **Transport posture** — the TLS floor, the forward-secrecy cipher gate, KEX-group pinning, the bind ladder, and the cleartext-hop refusal authority - ([`config/tls_policy.py:435` `insecure_hop_disposition`](messagefoundry/config/tls_policy.py), + ([`config/tls_policy.py:435` `insecure_hop_disposition`](../../../messagefoundry/config/tls_policy.py), `:481 enforce_insecure_hop`; ADR 0092 as amended by **ADR 0153**, which deleted the `is_phi` ALLOW arm and `audited_opt_out` so **no data label can permit a cleartext hop**). - **Posture configuration** — `[security]` (ADR 0118), the `[security].enforcement` REFUSE/WARN dial diff --git a/docs/testing/master-test-plan/18-interop-migration-and-uat.md b/docs/testing/master-test-plan/18-interop-migration-and-uat.md index 9603ee8b..4a02a42c 100644 --- a/docs/testing/master-test-plan/18-interop-migration-and-uat.md +++ b/docs/testing/master-test-plan/18-interop-migration-and-uat.md @@ -11,8 +11,8 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut over to*, *upgrade*, and *accept* MessageFoundry: -1. **Legacy import.** The deterministic Corepoint action-list importer (`messagefoundry import corepoint`, [`messagefoundry/corepoint_import.py`](messagefoundry/corepoint_import.py), ADR 0086) and the second partner-artifact import path, WSDL (ADR 0122). Mirth has **no deterministic importer** — see §17.9 Q3. -2. **Parallel-run / cutover validation.** The `tee` relay + parity engine ([`tee/`](tee/), docs/TEE-RELAY.md, BACKLOG #14) and the parallel-run reconcile harness ([`harness/reconcile/`](harness/reconcile/)), plus the primitives they depend on: per-outbound `simulate` ([`config/wiring.py:2551`](messagefoundry/config/wiring.py)) and `deployed=False` placeholders (ADR 0111). +1. **Legacy import.** The deterministic Corepoint action-list importer (`messagefoundry import corepoint`, [`messagefoundry/corepoint_import.py`](../../../messagefoundry/corepoint_import.py), ADR 0086) and the second partner-artifact import path, WSDL (ADR 0122). Mirth has **no deterministic importer** — see §17.9 Q3. +2. **Parallel-run / cutover validation.** The `tee` relay + parity engine ([`tee/`](../../../tee), docs/TEE-RELAY.md, BACKLOG #14) and the parallel-run reconcile harness ([`harness/reconcile/`](../../../harness/reconcile)), plus the primitives they depend on: per-outbound `simulate` ([`config/wiring.py:2551`](../../../messagefoundry/config/wiring.py)) and `deployed=False` placeholders (ADR 0111). 3. **Store migration posture.** SQLite → server-DB is **greenfield-only**; this chapter tests that the constraint is *enforced or loudly surfaced*, not merely written down. 4. **Version upgrade, store schema migration, rollback, and compatibility** of config, store, and the four version seams (engine↔web console, engine↔harness, engine↔IDE connection schema, engine↔adopter config repo). 5. **Adopter CI** — the scaffolded config repo and its generated `check.yml`. @@ -25,24 +25,24 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | Area | Owner | |---|---| -| Corepoint import coverage gaps (real-export reconciliation, CLI wrapper, batch-split FileSource, `anonymize-captures` no-body-to-stdout) | [`docs/testing/FEATURE-COVERAGE-PLAN.md`](docs/testing/FEATURE-COVERAGE-PLAN.md) rows **FCP:CFG-22** (:188, :481, :1406), **FCP:CLI-17** (:185, :478, :1442), **FCP:FILE-9** (:169, :462, :639), **FCP:ANON-9** (:73, :426, :1359) | -| On-box reconcile capability proof under the NSSM service identity | [`docs/testing/WIN2025-TEST-PLAN.md`](docs/testing/WIN2025-TEST-PLAN.md) **W25:S3.10** (:622-641) | +| Corepoint import coverage gaps (real-export reconciliation, CLI wrapper, batch-split FileSource, `anonymize-captures` no-body-to-stdout) | [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) rows **FCP:CFG-22** (:188, :481, :1406), **FCP:CLI-17** (:185, :478, :1442), **FCP:FILE-9** (:169, :462, :639), **FCP:ANON-9** (:73, :426, :1359) | +| On-box reconcile capability proof under the NSSM service identity | [`docs/testing/WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) **W25:S3.10** (:622-641) | | Day-1 human-closed box acceptance (AD/Kerberos, TOTP, NSSM lifecycle, no-console-flash, disposition walk, firewall, ACLs) | WIN2025-TEST-PLAN **W25:S1.AC-\*** (:306-322) + Appendix E | | Box-level sign-off green set | WIN2025-TEST-PLAN **W25:S6.3** (:1193-1200) | -| Staged rollout gates Stage 0→3 | [`docs/EARLY-ADOPTER-GUIDE.md`](docs/EARLY-ADOPTER-GUIDE.md) §11 (:561-630) | -| Honest limits of the adopter gate | [`docs/ADOPTER-CI.md`](docs/ADOPTER-CI.md) §5 (:136-158) | -| The engine↔console seam contract and bump procedure | [`docs/WEBCONSOLE-PACKAGE.md`](docs/WEBCONSOLE-PACKAGE.md) §2-§3 | -| Throughput ceilings, connection-scale, failover timing | [`docs/LOAD-TESTING.md`](docs/LOAD-TESTING.md) + `harness/load/` | +| Staged rollout gates Stage 0→3 | [`docs/EARLY-ADOPTER-GUIDE.md`](../../EARLY-ADOPTER-GUIDE.md) §11 (:561-630) | +| Honest limits of the adopter gate | [`docs/ADOPTER-CI.md`](../../ADOPTER-CI.md) §5 (:136-158) | +| The engine↔console seam contract and bump procedure | [`docs/WEBCONSOLE-PACKAGE.md`](../../WEBCONSOLE-PACKAGE.md) §2-§3 | +| Throughput ceilings, connection-scale, failover timing | [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) + `harness/load/` | | **Database shards** (splitting the store across multiple DBs) | ADR 0039 — **shelved**. The built scaling axis is the **engine shard** (ADR 0037/0063: N `serve --shard` processes over ONE unified store) and is not a migration concern | -**Vocabulary note for this chapter.** `messagefoundry/corepoint_import.py` contains a `Channel` dataclass ([:215](messagefoundry/corepoint_import.py)) — that is the *incumbent's* term for a Corepoint integration unit, used only inside the importer's parse model. Its **output** is a module with a named inbound Connection, an `@router`, and `@handler`s: a graph, not a bundling element. No test in this chapter may assert the existence of a MessageFoundry "channel"/"route" object. +**Vocabulary note for this chapter.** `messagefoundry/corepoint_import.py` contains a `Channel` dataclass ([:215](../../../messagefoundry/corepoint_import.py)) — that is the *incumbent's* term for a Corepoint integration unit, used only inside the importer's parse model. Its **output** is a module with a named inbound Connection, an `@router`, and `@handler`s: a graph, not a bundling element. No test in this chapter may assert the existence of a MessageFoundry "channel"/"route" object. **Cross-document ID prefixes (plan-wide convention).** This plan owns its own gap-ID space (`MIG-06`, `HA-02`, `API-13`, …). `docs/testing/FEATURE-COVERAGE-PLAN.md` and the WIN2025 documents use *colliding* IDs — there is a `CLI-17`, an `HA-20` and an `API-13` in more than one of them. Therefore **every reference to another document's ID carries a prefix**: `FCP:` for a FEATURE-COVERAGE-PLAN row (`FCP:CLI-17`), `W25:` for a WIN2025 test ID (`W25:S3.10`). A bare `MIG-nn` — or any bare ID in this plan's own format — always means **this plan's own row**. **The publishing boundary (plan-wide convention, owned by this chapter).** This repository is the **public** repo (`github.com/MEFORORG/MessageFoundry`). Part of the evidence this plan cites is real, current and owner-held but deliberately **not published here**. Absence from a public checkout is a **publishing boundary, not a missing artifact** — a reviewer may not record "file not found" against it as a coverage defect, and no row may be weakened, hedged or dropped on that ground. -- **`docs/BACKLOG.md` is a published *baseline* that stops at `## 231.`** ([:6883](docs/BACKLOG.md) is the last item heading; the boundary is stated in prose at [:269](docs/BACKLOG.md), [:4394](docs/BACKLOG.md) and [:6041](docs/BACKLOG.md) — *"the file you are reading ends at #231, while #242–#246 and their successors do not appear in it at all … their absence here is a publishing boundary, not evidence of completion"*). The ledger continues past it. **A citation above #231 — #233, #234, #242–#246, #275, #279, #285, #306, #310, #313, #744 and their successors — is sound evidence**; it simply does not resolve inside the public tree. Do not disclaim it, do not call it unverifiable, do not strip it. Where a public-only reader needs the hint, annotate it **once and neutrally** — "(above the published #231 baseline)" — never on every occurrence. In-baseline citations (#14, #105, #124, #131, #149, #180) resolve normally. -- **`docs/security/` (~32 files), `docs/reviews/` and `docs/marketing/` are gitignored post-cutover** ([`.gitignore:144-146`](.gitignore); the rationale — "an attacker roadmap" — is in the same block at :147-149). The AD-federation lab runbook, the off-loopback deployment guide and the ASVS assessment set **exist** for the owner: they are **withheld from the public repo, not absent**. Write "withheld", never "missing" or "does not exist". +- **`docs/BACKLOG.md` is a published *baseline* that stops at `## 231.`** ([:6883](../../BACKLOG.md) is the last item heading; the boundary is stated in prose at [:269](../../BACKLOG.md), [:4394](../../BACKLOG.md) and [:6041](../../BACKLOG.md) — *"the file you are reading ends at #231, while #242–#246 and their successors do not appear in it at all … their absence here is a publishing boundary, not evidence of completion"*). The ledger continues past it. **A citation above #231 — #233, #234, #242–#246, #275, #279, #285, #306, #310, #313, #744 and their successors — is sound evidence**; it simply does not resolve inside the public tree. Do not disclaim it, do not call it unverifiable, do not strip it. Where a public-only reader needs the hint, annotate it **once and neutrally** — "(above the published #231 baseline)" — never on every occurrence. In-baseline citations (#14, #105, #124, #131, #149, #180) resolve normally. +- **`docs/security/` (~32 files), `docs/reviews/` and `docs/marketing/` are gitignored post-cutover** ([`.gitignore:144-146`](../../../.gitignore); the rationale — "an attacker roadmap" — is in the same block at :147-149). The AD-federation lab runbook, the off-loopback deployment guide and the ASVS assessment set **exist** for the owner: they are **withheld from the public repo, not absent**. Write "withheld", never "missing" or "does not exist". - **Therefore** a test row may cite withheld evidence and still be sound. Where a doc-anchored *suite* cannot see its document from a public checkout, the engineering answer is the MIG-29 / MIG-30 treatment — make the skip explicit, censused and CI-answerable — not a claim that the document is gone. - **What the plan owes the reader** is one statement of which evidence a public-only reader cannot open: everything under those three directories, plus any BACKLOG number above #231. This subsection is that statement; Part I §0.5 carries the same rule for readers who start there. MIG-74's linter enforces it mechanically, classifying a withheld path as `WITHHELD` against a checked-in manifest instead of failing the run. @@ -81,22 +81,22 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov **DONE — do not re-plan.** The `tee` parity stack and the `anon` vendoring-drift guard are mature and adversarially tested; add nothing there beyond the CLI-boundary PHI assertion in MIG-43. The ADR 0060 FIFO index migration is the best-tested migration in the repo, including a deliberate downgrade simulator — the vintage matrix (MIG-06..MIG-13) must *extend* its backend-parametrized fixture pattern, not duplicate it. The engine↔console seam **snapshot** gate is correct and complete at the code layer; what is missing is entirely at the *packaging and runbook* layer (MIG-15, MIG-16). `verify`'s section model, MANUAL honesty, and report writers are settled. The `release.yml` leak/idempotence/OIDC gates are settled. -**Two recon corrections that change the plan.** (a) The Corepoint **role layer is wired** — `_parse_statement` calls `parse_roles` / `_role_verb` / `_operands_from_roles` at [`corepoint_import.py:1289-1301`](messagefoundry/corepoint_import.py) and dispatches into `_map_roles` at :1346, which emits real `Action` objects (`copy_field` / `set_field` / `append_to_field`, :1152-1170). `SPEC-105 §0`'s "Vocabulary `Action` objects produced: **0**" is a **pre-wiring measurement** and must not be quoted as current state. (b) `docs/BACKLOG.md` is a **published baseline ending at `## 231.`** ([:6883](docs/BACKLOG.md) is the last heading; the boundary prose is at [:6041](docs/BACKLOG.md) and [:4394](docs/BACKLOG.md)). **#313 sits above that baseline and is sound evidence** — the ledger continues past what this repo publishes, so a row may rest on it; it simply cannot be opened from a public checkout (see the publishing-boundary convention in §17.1). What must not happen is a row treating the baseline as the whole ledger: #105's own entry says as much at [:269](docs/BACKLOG.md) — "the real gate is #313, which is invisible from this baseline". +**Two recon corrections that change the plan.** (a) The Corepoint **role layer is wired** — `_parse_statement` calls `parse_roles` / `_role_verb` / `_operands_from_roles` at [`corepoint_import.py:1289-1301`](../../../messagefoundry/corepoint_import.py) and dispatches into `_map_roles` at :1346, which emits real `Action` objects (`copy_field` / `set_field` / `append_to_field`, :1152-1170). `SPEC-105 §0`'s "Vocabulary `Action` objects produced: **0**" is a **pre-wiring measurement** and must not be quoted as current state. (b) `docs/BACKLOG.md` is a **published baseline ending at `## 231.`** ([:6883](../../BACKLOG.md) is the last heading; the boundary prose is at [:6041](../../BACKLOG.md) and [:4394](../../BACKLOG.md)). **#313 sits above that baseline and is sound evidence** — the ledger continues past what this repo publishes, so a row may rest on it; it simply cannot be opened from a public checkout (see the publishing-boundary convention in §17.1). What must not happen is a row treating the baseline as the whole ledger: #105's own entry says as much at [:269](../../BACKLOG.md) — "the real gate is #313, which is invisible from this baseline". ### 17.3 Risk analysis | Risk | Failure mode | Blast radius | Detected today? | Priority | |---|---|---|---|---| -| Greenfield-only rule enforced nowhere | An operator flips `[store].type` to `postgres`/`sqlserver`; `open_store` ([`store/base.py:1739`](messagefoundry/store/base.py)) selects the backend with **no prior-backend detection, no drain check, no warning**. The engine starts clean on an empty schema | The populated SQLite store — in-flight ingress/routed/outbound rows plus all retained PHI history and audit — is silently orphaned. Named as "Greenfield-only data loss" at `docs/releases/v0.1-PLAN.md:457`; the only shipped mitigation is prose in 6 documents and a manual checkbox at `docs/DEPLOY-SERVER-DB.md:274` | **No** | **P0** | -| Engine upgrade past a seam bump **bricks the whole engine**, not just `/ui` | `[api].serve_ui` defaults **True** ([`settings.py:672`](messagefoundry/config/settings.py)). `create_app`'s `serve_ui` tail calls `assert_engine_seam(ENGINE_UI_SEAM)` ([`api/app.py:4951`](messagefoundry/api/app.py)) and lets `UiSeamMismatch` propagate out of `create_app`. `SUPPORTED_ENGINE_SEAMS == frozenset({15})` — strict lockstep, no N-1 | `messagefoundry serve` **fails to start** on a PHI box mid-upgrade. Intake stops. The runbooks operators follow (EARLY-ADOPTER-GUIDE §13 step 4 :679, INSTALL-GUIDE §10 :306-313) bump only `messagefoundry==<new>` and never mention the console wheel | Fails loud at startup — but nothing tests the *sequence*, and nothing warns beforehand | **P0** | +| Greenfield-only rule enforced nowhere | An operator flips `[store].type` to `postgres`/`sqlserver`; `open_store` ([`store/base.py:1739`](../../../messagefoundry/store/base.py)) selects the backend with **no prior-backend detection, no drain check, no warning**. The engine starts clean on an empty schema | The populated SQLite store — in-flight ingress/routed/outbound rows plus all retained PHI history and audit — is silently orphaned. Named as "Greenfield-only data loss" at `docs/releases/v0.1-PLAN.md:457`; the only shipped mitigation is prose in 6 documents and a manual checkbox at `docs/DEPLOY-SERVER-DB.md:274` | **No** | **P0** | +| Engine upgrade past a seam bump **bricks the whole engine**, not just `/ui` | `[api].serve_ui` defaults **True** ([`settings.py:672`](../../../messagefoundry/config/settings.py)). `create_app`'s `serve_ui` tail calls `assert_engine_seam(ENGINE_UI_SEAM)` ([`api/app.py:4951`](../../../messagefoundry/api/app.py)) and lets `UiSeamMismatch` propagate out of `create_app`. `SUPPORTED_ENGINE_SEAMS == frozenset({15})` — strict lockstep, no N-1 | `messagefoundry serve` **fails to start** on a PHI box mid-upgrade. Intake stops. The runbooks operators follow (EARLY-ADOPTER-GUIDE §13 step 4 :679, INSTALL-GUIDE §10 :306-313) bump only `messagefoundry==<new>` and never mention the console wheel | Fails loud at startup — but nothing tests the *sequence*, and nothing warns beforehand | **P0** | | The documented install-time seam defence does not exist | WEBCONSOLE-PACKAGE.md §2 claims a three-layer handshake whose layer 1 is "the PEP 508 range on the engine dependency fails an out-of-range pair at `pip`/`uv` resolve". `packaging/messagefoundry-webconsole/pyproject.toml:38-42` declares a **bare `"messagefoundry"`** with a comment conceding "a bare dep is fine for now" | `pip install -U messagefoundry` resolves happily across a seam bump. The first symptom is the P0 above. A documented control that is absent is worse than a known-absent one: the runbook author trusted it | **No** — nothing binds the doc claim to the metadata | **P0** | -| Reconcile diff output is PHI and **ungated** | `harness/reconcile/report.py` `render_text` prints up to 20 mismatched messages' field-level `left`/`right` **values** by default (:63-66); `render_json` **always** emits them (:31-42). `_run_compare` ([`__main__.py:80-93`](harness/reconcile/__main__.py)) prints unconditionally and writes `--report-json` verbatim | This is the tool an operator runs during a live cutover, and its output is what gets pasted into a ticket. `tee` gates the same class of output behind `--show-diffs` with an explicit warning (`tee/__main__.py:269`, :451-453); the reconcile harness has **no equivalent** | **No** | **P0** | -| Reconcile capture writes PHI bodies by default | `CaptureSink` accepts `anonymizer=` ([`capture.py:46`](harness/reconcile/capture.py)) but `_run_capture` never passes it and the `capture` subparser exposes **no** `--anonymize`/`--salt-env` flag | During a shadow phase the JSONL holds real production bodies on disk, on a non-production box, with no purge story | **No** | **P0** | -| No store upgrade/downgrade vintage matrix | SQLite `_migrate` ([`store/store.py:2933`](messagefoundry/store/store.py)) is ~120 lines of `PRAGMA table_info`-probed idempotent ALTERs with **no schema version number**. It is exercised only from a fresh DB plus one legacy-`outbox` fixture. No test opens a DB written by an *older released engine*; none opens a *newer* DB with an older engine | ADR 0060 is the precedent: a re-key that "silently never landed on upgraded DBs" and shipped green. Both EARLY-ADOPTER-GUIDE §13 and INSTALL-GUIDE §10 prescribe pin-back rollback as the primary lever, against a store the newer engine has already opened, with nothing behind it | **No** | **P0** | +| Reconcile diff output is PHI and **ungated** | `harness/reconcile/report.py` `render_text` prints up to 20 mismatched messages' field-level `left`/`right` **values** by default (:63-66); `render_json` **always** emits them (:31-42). `_run_compare` ([`__main__.py:80-93`](../../../harness/reconcile/__main__.py)) prints unconditionally and writes `--report-json` verbatim | This is the tool an operator runs during a live cutover, and its output is what gets pasted into a ticket. `tee` gates the same class of output behind `--show-diffs` with an explicit warning (`tee/__main__.py:269`, :451-453); the reconcile harness has **no equivalent** | **No** | **P0** | +| Reconcile capture writes PHI bodies by default | `CaptureSink` accepts `anonymizer=` ([`capture.py:46`](../../../harness/reconcile/capture.py)) but `_run_capture` never passes it and the `capture` subparser exposes **no** `--anonymize`/`--salt-env` flag | During a shadow phase the JSONL holds real production bodies on disk, on a non-production box, with no purge story | **No** | **P0** | +| No store upgrade/downgrade vintage matrix | SQLite `_migrate` ([`store/store.py:2933`](../../../messagefoundry/store/store.py)) is ~120 lines of `PRAGMA table_info`-probed idempotent ALTERs with **no schema version number**. It is exercised only from a fresh DB plus one legacy-`outbox` fixture. No test opens a DB written by an *older released engine*; none opens a *newer* DB with an older engine | ADR 0060 is the precedent: a re-key that "silently never landed on upgraded DBs" and shipped green. Both EARLY-ADOPTER-GUIDE §13 and INSTALL-GUIDE §10 prescribe pin-back rollback as the primary lever, against a store the newer engine has already opened, with nothing behind it | **No** | **P0** | | Adopter install docs pin a three-releases-stale version | `docs/INSTALL-GUIDE.md:55`, `:61`, `:144`; `docs/USER-GUIDE.md:97`, `:113`; `docs/EARLY-ADOPTER-GUIDE.md:153`, `:156`; `docs/MENTAL-MODEL.md:343` all say `messagefoundry==0.1.0` while `messagefoundry/__init__.py` is **`0.3.2`** | `docs/SUPPORT-POLICY.md:9` supports only the latest release with no back-ports, on a 15-day critical / 30-day high SLA. A naive user lands on an unsupported build and their scaffolded `audit-pin` job reds on day one. INSTALL-GUIDE:144 additionally *misdescribes* the scaffold, which pins `__version__` correctly (`scaffold.py:370`) | **No** — `test_install_instruction_provenance.py` checks extras and distribution names, never versions | **P1** | | No doc-executability tests over the adopter guides | `tests/test_off_loopback_runbook.py:179` / `:218` / `:89` is the proven pattern — every fenced TOML block loads through the real loader and fails on a *silently ignored* key; every printed CLI command names a registered subcommand — applied **only** to a private security runbook | INSTALL-GUIDE / USER-GUIDE / EARLY-ADOPTER-GUIDE carry dozens of copy-pasteable commands and TOML blocks that are the only on-ramp. A relocated settings key or renamed subcommand breaks first-run with nothing failing. The off-loopback guard exists because this class shipped **twice** | **No** | **P1** | | Two of the largest doc-drift guards are inert in every public checkout | `docs/security/` is **withheld, not missing** — gitignored post-cutover (`.gitignore:144`), ~32 files that exist for the owner. Because a public checkout cannot see them, `tests/test_off_loopback_runbook.py` skips at **module level** (:47-55, 12 tests) and `tests/test_threat_model_doc_drift.py` skips at the **accessor** (:256), so its code-only tests survive but ~88 doc-anchored assertions do not (24 test functions, most doc-anchored) | A silent skip reads as green: the suite reports success and the coverage claim survives. This is the self-defending false-green class `tests/test_cutover_slug_rot.py` was written about | Partially — the accessor-level skip in the threat-model module is a deliberate, documented mitigation | **P1** | -| `verify` still probes the **retired** PySide6 console and names a non-existent extra | `check_console_importable` ([`verify/checks.py:180-195`](messagefoundry/verify/checks.py)) tests `PySide6` importability, labels it "Console importable", and instructs "install the `[console]` extra". `pyproject.toml` declares no `console` extra (`dev, dicom, fhir, harness, otel, postgres, sftp, sqlserver, vault, webauthn, x12, xml`). `docs/testing/VERIFY.md:40` documents the row | The shipped on-box acceptance tool asserts a retired surface and prints an install instruction that cannot work. It escapes `test_install_instruction_provenance.py` because `_EXTRA_REF` matches only `messagefoundry[...]` and `_BARE_NAME_INSTALL` requires a `pip install` verb. Worse: **`verify` has no web-console seam check at all** — the most upgrade-fragile pair on the box is unverified | **No** | **P1** | +| `verify` still probes the **retired** PySide6 console and names a non-existent extra | `check_console_importable` ([`verify/checks.py:180-195`](../../../messagefoundry/verify/checks.py)) tests `PySide6` importability, labels it "Console importable", and instructs "install the `[console]` extra". `pyproject.toml` declares no `console` extra (`dev, dicom, fhir, harness, otel, postgres, sftp, sqlserver, vault, webauthn, x12, xml`). `docs/testing/VERIFY.md:40` documents the row | The shipped on-box acceptance tool asserts a retired surface and prints an install instruction that cannot work. It escapes `test_install_instruction_provenance.py` because `_EXTRA_REF` matches only `messagefoundry[...]` and `_BARE_NAME_INSTALL` requires a `pip install` verb. Worse: **`verify` has no web-console seam check at all** — the most upgrade-fragile pair on the box is unverified | **No** | **P1** | | No adopter-facing upgrade smoke over the pinned-wheel path | `windows-service-smoke` installs from the source tree. No leg installs release N-1 from an index, runs traffic, then upgrades in place to N against the same store | This is exactly the sequence EARLY-ADOPTER-GUIDE §13 prescribes and the only one that exercises `_migrate` against a store written by a real prior release under a real service account | **No** | **P1** | | No rollback rehearsal | EARLY-ADOPTER-GUIDE §13 prescribes config rollback via audited `POST /config/reload` as the primary lever, engine pin-back as secondary, with a warning that store-level changes are not trivially reversible. None of it is exercised as a sequence | Rollback is the 2 a.m. lever. If the reload-root confinement, the quiesce-and-swap, or the dead-letter replay misbehaves under a partially-drained queue, the recovery path itself loses messages | **No** | **P1** | | No vendor/partner interoperability rig | No Epic / Oracle Health / lab / radiology / HIE fixture; no non-conformant dialect corpus; no live SMART sandbox leg; no real PACS. FHIR `test_connection` does GET `/metadata` but as **reachability only** (`transports/fhir.py:505-508`, `:992`) — the CapabilityStatement body is never parsed and `fhir_version` (`R4B`/`R5`/`STU3`) is never cross-checked against it | Every partner-facing claim — SMART Backend Services against Epic/Oracle Health (`transports/smart.py:5`, FEATURE-MAP:134), DICOM SCU "Mirth-sender parity" (`transports/dicom.py:38`), FHIR version support — rests on loopback mocks. A vendor quirk breaks a live feed with no failing test, surfacing as an `ERROR` disposition on real clinical traffic | **No** | **P1** | @@ -104,14 +104,14 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | Corepoint importer has no regression guard at real-export shape or scale | The only fixtures are 120 lines of hand-authored XML against a real export of 19,682 elements / 187 `<ActionList>` / 14,049 statement positions (`SPEC-105 §0`) | `HANDOFF-105` records the old fixture passing while **0 of 5,013 paths resolved** against real data, because the fixture omitted the connective and the handle: green tests, inert feature. The accounting invariant is still asserted only over 120 synthetic lines | **No** | **P1** | | Reconcile CLI exit-code gating untested | `_run_compare` returns `0 if result.clean else 1` — the thing a per-connection cutover sign-off gates on. No test drives `python -m harness.reconcile` | If argument parsing silently drops `--blank`/`--sort-segment`/`--ignore-segment` or the exit code stops gating, a sign-off passes on a genuinely divergent feed and the **cutover decision itself is wrong** | **No** | **P1** | | Scaffolded adopter CI workflow never executed | `tests/test_scaffold_requirements.py` parses `check.yml` as YAML and greps strings; nothing runs `init` → clean venv → `pip install -r requirements.txt` → `messagefoundry check` as a fresh adopter would | A template slip, a renamed flag, or a changed exit code breaks every adopter's CI simultaneously and nothing here fails. `test_scaffold.py` runs `check` **in-process**, which does not exercise the pinned-wheel install path | **No** | **P1** | -| No DST-transition test for the per-connection `Schedule` | `tests/test_connection_scheduler.py:111` sets `America/New_York` but asserts only a plain summer offset. `ActiveWindow.contains` ([`config/models.py:375`](messagefoundry/config/models.py)) is never exercised across spring-forward (the skipped 02:00-03:00 hour) or fall-back (the repeated hour) | A Connection that fails to auto-start on the DST Sunday is an intake outage; one that fails to park stays open outside its contracted window. Both are silent — the scheduler logs a clean start/stop either way and no disposition changes | **No** | **P2** | +| No DST-transition test for the per-connection `Schedule` | `tests/test_connection_scheduler.py:111` sets `America/New_York` but asserts only a plain summer offset. `ActiveWindow.contains` ([`config/models.py:375`](../../../messagefoundry/config/models.py)) is never exercised across spring-forward (the skipped 02:00-03:00 hour) or fall-back (the repeated hour) | A Connection that fails to auto-start on the DST Sunday is an intake outage; one that fails to park stays open outside its contracted window. Both are silent — the scheduler logs a clean start/stop either way and no disposition changes | **No** | **P2** | | Retention VACUUM and DR-backup schedules use bare system-local wall clock | `retention.py:997-1010` and `dr_backup.py:241-243`, `:960` use `time.localtime` with a local-day dedupe key (`_day_key`) and **no operator-visible IANA zone** — unlike `Schedule`, which is zone-declared | A missed nightly backup or purge on the DST Sunday is invisible until a restore or a HIPAA retention audit; a host timezone change silently moves the maintenance window with no config change | **No** | **P2** | | No locale / non-UTF-8 codepage coverage beyond `--help` | `locale` is unused across `messagefoundry/`; hardening is a `reconfigure(errors="replace")` at `__main__.py:49-56` plus `_safe_print` (`:4283`). One test exists: `tests/test_cli.py:43`, top-level `--help` only | On a stock non-en-US Windows Server the console codepage is not cp1252. A config path, partner name, or log line with non-ASCII can raise `UnicodeEncodeError` inside CLI output or NSSM-captured stdout — and the guides' copy-paste commands are the first thing to hit it | Partially (one surface) | **P2** | | `FEATURE-MAP.md` is materially stale in this area | §10 "Surfaces — Admin Console (PySide6)" (:162-173) lists ✅ rows for a **retired** console; §13 (:209-211) marks version single-sourcing, CHANGELOG refresh, and `release.yml` as 🔨 though all three shipped; the whole MIG surface (Corepoint import, tee, reconcile, timezone, verify, support bundle, update-check) has **no row at all** | It is the public status catalog an evaluator reads. `test_feature_map_claims.py` guards only ASVS-score quoting and link targets; nothing asserts a row matches reality | **No** | **P2** | | An adopter guide denies a shipped route | `docs/EARLY-ADOPTER-GUIDE.md:635` states "there is no Prometheus exporter" while `api/app.py:4146` registers `GET /metrics` (and `:4156` `/metrics/history`), shipped per FEATURE-MAP:159 | An adopter builds day-2 monitoring on log-scraping — a worse posture than what ships, and one that makes the EARLY-ADOPTER-GUIDE §11 Stage 2 "monitoring in place" gate harder to meet than necessary | **No** | **P2** | | `messagefoundry-harness` publish is variable-gated while a test asserts it published | `release.yml:20-24` gates the harness PyPI publish behind repo variable `PUBLISH_HARNESS`; `tests/test_install_instruction_provenance.py:56` lists it in `_PUBLISHED_DISTRIBUTIONS` | An adopter following the guide to install the harness — the tool the EARLY-ADOPTER-GUIDE §11 Stage 0 gates depend on — may hit an unresolvable install | **No** | **P2** | | IDE schema-version constant is duplicated and cross-language-unpinned | `SUPPORTED_SCHEMA_VERSION = 1` appears in **both** `ide/src/connectionForm.ts:74` and `ide/src/connectionSchemaModel.ts:21`; neither is pinned against the engine's `SCHEMA_VERSION` (`config/connection_schema.py:45`) | The forward-compat *refusal* is correct and tested — the residual is two constants that can diverge from each other, silently splitting `isSupported()` from `parseConnectionSchema()` | Partially — the refusal is tested; the duplication is not | **P2** | -| `check`'s dryrun green-skips a present-but-empty fixtures dir | `_check_dryrun` returns `ok=True, skipped=True` when a real dir contains no `**/*.hl7` ([`checks.py:1061-1066`](messagefoundry/checks.py)). An explicitly-given *missing* path correctly FAILS | The adopter's merge gate reads green while asserting nothing about behaviour. The scaffold *does* ship `messages/sets/example_adt.hl7`, so a fresh repo is fine — the hazard is an adopter who empties or relocates the corpus | Documented at ADOPTER-CI §5.1; not surfaced at runtime as a distinct verdict | **P2** | +| `check`'s dryrun green-skips a present-but-empty fixtures dir | `_check_dryrun` returns `ok=True, skipped=True` when a real dir contains no `**/*.hl7` ([`checks.py:1061-1066`](../../../messagefoundry/checks.py)). An explicitly-given *missing* path correctly FAILS | The adopter's merge gate reads green while asserting nothing about behaviour. The scaffold *does* ship `messages/sets/example_adt.hl7`, so a fresh repo is fine — the hazard is an adopter who empties or relocates the corpus | Documented at ADOPTER-CI §5.1; not surfaced at runtime as a distinct verdict | **P2** | | No day-2 support-workflow validation | `support bundle` content is well tested; nothing exercises the loop *reproduce a fault → collect → the bundle diagnoses it* | SUPPORT-POLICY sets a vendor fast-response SLA that depends on the adopter's first artifact being sufficient. A bundle that redacts away the diagnostic signal, or omits the failing Connection, turns a 1-day triage into a multi-round PHI-risky exchange | **No** | **P2** | ### 17.4 Test matrix @@ -135,7 +135,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-03 | The greenfield-only claim is single-sourced across all six documents | Compat | pytest | dev-PC | n/a | T | P1 | A doc-drift test asserts the greenfield-only sentence in `DEPLOY-SERVER-DB.md:11`, `EARLY-ADOPTER-GUIDE.md:100`, `FEATURE-MAP.md:98`, `SYSTEM-REQUIREMENTS.md:113`, `AOAG-DEPLOYMENT.md:309`, `releases/v0.1-PLAN.md:110` all resolve, and that whichever code path implements MIG-01 is named by at least one of them | | MIG-04 | `in_pipeline` — the cutover checklist's drain signal — is a real, reachable field | Functional | pytest | dev-PC | x3 | T | P1 | `GET /status` (`api/app.py:4134`) and `GET /metrics` both expose `in_pipeline` sourced from `store.in_pipeline_depth()`; the value is 0 on a fresh store, N after N un-drained sends, back to 0 after drain. `DEPLOY-SERVER-DB.md:274`'s checkbox names a field an operator can actually read | | MIG-05 | No shipped text advertises a cross-backend migration tool | Compat | pytest | dev-PC | n/a | T | P2 | No `docs/*.md` or CLI help string offers a SQLite→server-DB copy/convert command. BACKLOG #180 (:5940) remains the only place it is described, and is marked planned | -| MIG-06 | SQLite vintage matrix — forward open (N-3 … N-1 → current) from a **pre-migration fixture DB** | Upgrade | pytest | dev-PC | SQLite | T | P0 | **Owner of the store vintage / schema-upgrade matrix** — STORE-40, STORE-41, STORE-42 and STORE-43 point at MIG-06/MIG-09/MIG-10; no separate work is scoped there. The fixture is a **pre-migration DB**: a checked-in generator per vintage V emits the *exact DDL as it stood at vintage V* — before the current `_MIGRATION_REV` and before the ADR 0060 seq-trailing index rename — plus a handful of rows, so the DB the current engine opens has genuinely never seen `_migrate`. No binary `.db` is committed; a generator keeps the fixture reviewable and PHI-free by construction. For each vintage, open with the current engine and assert: `_migrate` ([`store/store.py:2933`](messagefoundry/store/store.py)) completes; every column in `_MESSAGE_MIGRATIONS`, plus `audit_log.row_hash`, `audit_log.client`, `users.channel_scope`, `roles.permissions`, is present; every pre-existing message row is readable with its original `status` and decrypts under the configured key; `in_pipeline_depth()` matches the vintage's queued count; and the post-open `schema_meta` hash equals the current `_schema_hash()` | +| MIG-06 | SQLite vintage matrix — forward open (N-3 … N-1 → current) from a **pre-migration fixture DB** | Upgrade | pytest | dev-PC | SQLite | T | P0 | **Owner of the store vintage / schema-upgrade matrix** — STORE-40, STORE-41, STORE-42 and STORE-43 point at MIG-06/MIG-09/MIG-10; no separate work is scoped there. The fixture is a **pre-migration DB**: a checked-in generator per vintage V emits the *exact DDL as it stood at vintage V* — before the current `_MIGRATION_REV` and before the ADR 0060 seq-trailing index rename — plus a handful of rows, so the DB the current engine opens has genuinely never seen `_migrate`. No binary `.db` is committed; a generator keeps the fixture reviewable and PHI-free by construction. For each vintage, open with the current engine and assert: `_migrate` ([`store/store.py:2933`](../../../messagefoundry/store/store.py)) completes; every column in `_MESSAGE_MIGRATIONS`, plus `audit_log.row_hash`, `audit_log.client`, `users.channel_scope`, `roles.permissions`, is present; every pre-existing message row is readable with its original `status` and decrypts under the configured key; `in_pipeline_depth()` matches the vintage's queued count; and the post-open `schema_meta` hash equals the current `_schema_hash()` | | MIG-07 | Server-backend vintage matrix — forward open | Upgrade | pytest + CI-leg | container-CI | x2 | T | P0 | Same assertions against SQL Server and PostgreSQL **pre-migration** vintage schemas, driven off `_schema_hash()` / `_MIGRATION_REV`. A vintage marker forces a full guarded re-run (ADR 0064); the applock precedes every CREATE (extends `test_sqlserver_schema_init.py:175`) | | MIG-08 | **Downgrade** — a store written by the current engine, opened by the prior released wheel | Upgrade | CI-leg | container-CI | x3 | T | P0 | Install the previous published engine wheel into a clean venv; open the current-engine store. Either it opens and every prior message row is readable, **or** it refuses with a message naming the schema mismatch. A silent partial open, a dropped column, or a re-stamped older `schema_meta` hash is a FAIL — those three outcomes are what makes this row falsifiable rather than a recorded observation. The surviving branch is written up as the authoritative answer to §17.9 Q6 | | MIG-09 | ADR 0064 **alternating-hash opens** are pinned | Upgrade | pytest | container-CI | x2 | T | P1 | Beyond a single newer-hash open: drive the **alternating** sequence current → prior → current → prior against one persistent store, four opens, asserting after each. With a `schema_meta` row carrying a *newer* hash than the running engine's `_schema_hash()`, the documented behaviour holds (full guarded re-run of the older DDL batch, or refusal) and the older DDL batch cannot drop, rename, or narrow a column that only the newer schema defines. Across the alternation the hash marker must not thrash — the newer engine must not have to re-run its full batch on every re-open after an older engine re-stamped the marker — and no column added by the newer schema is lost at any point in the cycle | @@ -147,7 +147,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov | MIG-15 | The web console declares a PEP 508 engine range | Upgrade | pytest | dev-PC | n/a | T | P0 | `packaging/messagefoundry-webconsole/pyproject.toml` `[project].dependencies` contains a bounded `messagefoundry>=X,<Y` specifier, not a bare name; and `WEBCONSOLE-PACKAGE.md` §2's install-time layer-1 claim is asserted against that specifier. Fails today (bare `"messagefoundry"`, :42) | | MIG-16 | The upgrade runbooks name every wheel that must move together | Compat | pytest | dev-PC | n/a | T | P1 | Drive from the packaging tree, not a hardcoded list: for every distribution this repo builds that an operator installs alongside the engine, the upgrade sections of `INSTALL-GUIDE.md` §10 and `EARLY-ADOPTER-GUIDE.md` §13 name it whenever `[api].serve_ui` is in scope. Fails today — both bump only `messagefoundry==<new>` | | MIG-17 | `verify` reports the engine↔console seam | Functional | pytest | dev-PC | n/a | T | P1 | A new `host.webconsole` check reports PASS with both seam values when the installed console supports `ENGINE_UI_SEAM`, FAIL naming both values when it does not, and SKIP when `serve_ui` is off or the console is absent. Report renders in `--report-md` and `--report-json` | -| MIG-18 | `verify` stops asserting the retired PySide6 console and stops naming a non-existent extra → **pointer** | Negative/Security | — | dev-PC | n/a | T | P1 | **Covered by TRAY; no separate work scoped.** TRAY owns the `[console]` extra / `check_console_importable` provenance deliverable (WEB-58 also points there). Context retained for this chapter's readers only: `check_console_importable` ([`verify/checks.py:178-192`](messagefoundry/verify/checks.py)) still probes `PySide6` as "the console" and its SKIP message at :185 instructs "install the `[console]` extra", which `pyproject.toml` does not declare; `docs/testing/VERIFY.md:40` documents the row. The widened extra-detector in `tests/test_install_instruction_provenance.py` (matching a bare `[<name>] extra` phrase, not only `messagefoundry[...]` — the pattern that let this through) lands with TRAY's row | +| MIG-18 | `verify` stops asserting the retired PySide6 console and stops naming a non-existent extra → **pointer** | Negative/Security | — | dev-PC | n/a | T | P1 | **Covered by TRAY; no separate work scoped.** TRAY owns the `[console]` extra / `check_console_importable` provenance deliverable (WEB-58 also points there). Context retained for this chapter's readers only: `check_console_importable` ([`verify/checks.py:178-192`](../../../messagefoundry/verify/checks.py)) still probes `PySide6` as "the console" and its SKIP message at :185 instructs "install the `[console]` extra", which `pyproject.toml` does not declare; `docs/testing/VERIFY.md:40` documents the row. The widened extra-detector in `tests/test_install_instruction_provenance.py` (matching a bare `[<name>] extra` phrase, not only `messagefoundry[...]` — the pattern that let this through) lands with TRAY's row | | MIG-19 | IDE `SUPPORTED_SCHEMA_VERSION` is single-sourced and pinned to the engine | Compat | ide-mocha + pytest | dev-PC | n/a | T | P2 | The TS constant is defined once (currently duplicated at `connectionForm.ts:74` and `connectionSchemaModel.ts:21`) and a pytest assertion pins it against `config/connection_schema.SCHEMA_VERSION`, so a `SCHEMA_VERSION` bump without an extension bump reds the engine suite. The existing loud-refusal test (`connection-schema.test.ts:79`) stays | | MIG-20 | Engine and harness stay lockstep at build time | Compat | pytest | dev-PC | n/a | T | P2 | `packaging/messagefoundry-harness/pyproject.toml` sources its version from `messagefoundry/__init__.py` (:13) and depends on `messagefoundry[harness]` (:41); assert both, and that no doc instructs a harness version different from the engine's | | MIG-21 | Every `messagefoundry==X.Y.Z` in shipped text equals `__version__` | Compat | pytest | dev-PC | n/a | T | P1 | Extract every literal engine pin from `docs/*.md` + `README.md` + `messagefoundry/**/*.py` and assert equality with `messagefoundry.__version__`. Fails today: 0.1.0 in 4 documents vs 0.3.2. Placeholders (`<version>`, `X.Y.Z`, `$V`) are explicitly exempt and the exemption list cannot rot vacuous | @@ -219,7 +219,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov **Steps.** 1. Write `messagefoundry.toml` with `[store] type = "sqlite"` and `path = "<tmp>/mefor.db"`. -2. Open the store through `open_store(StoreSettings(...))` ([`store/base.py:1739`](messagefoundry/store/base.py)). Ingest 25 synthetic ADT messages produced by `messagefoundry generate` into the ingress stage without draining, so `await store.in_pipeline_depth() == 25`. Close. +2. Open the store through `open_store(StoreSettings(...))` ([`store/base.py:1739`](../../../messagefoundry/store/base.py)). Ingest 25 synthetic ADT messages produced by `messagefoundry generate` into the ingress stage without draining, so `await store.in_pipeline_depth() == 25`. Close. 3. Confirm the file exists and is non-trivial (`> 8 KiB`), and re-open/close once to prove the fixture is a *real, migrated* store rather than a blank file. 4. Flip `[store] type` to `postgres` (then `sqlserver`) against the **same config root**, leaving the SQLite file in place, and call `open_store` again with `caplog.at_level(logging.WARNING)`. @@ -241,7 +241,7 @@ This chapter covers the paths an organisation actually walks to *adopt*, *cut ov **Steps — forward (in-process, `pytest`).** 1. Check in a **generator per vintage**, not a binary DB: a small module that emits the exact DDL + a handful of rows for schema vintage V. This keeps the fixture reviewable, keeps PHI out by construction, and survives a repo that cannot carry binaries. -2. For each vintage, materialize the DB, open it with the current engine, and assert: `_migrate` ([`store/store.py:2933`](messagefoundry/store/store.py)) completes; every column named in `_MESSAGE_MIGRATIONS`, plus `audit_log.row_hash`, `audit_log.client`, `users.channel_scope`, `roles.permissions`, is present; each pre-existing message row is fetchable with its original `status`; `in_pipeline_depth()` equals the vintage's queued count. +2. For each vintage, materialize the DB, open it with the current engine, and assert: `_migrate` ([`store/store.py:2933`](../../../messagefoundry/store/store.py)) completes; every column named in `_MESSAGE_MIGRATIONS`, plus `audit_log.row_hash`, `audit_log.client`, `users.channel_scope`, `roles.permissions`, is present; each pre-existing message row is fetchable with its original `status`; `in_pipeline_depth()` equals the vintage's queued count. 3. Re-open and assert **zero** DDL executed (probe the `PRAGMA table_info` call count on SQLite; the `schema_meta` marker skip on the server backends). **Steps — backward (two processes, CI leg).** diff --git a/docs/testing/master-test-plan/19-execution-phasing-and-sign-off.md b/docs/testing/master-test-plan/19-execution-phasing-and-sign-off.md index 041f2c65..043dff35 100644 --- a/docs/testing/master-test-plan/19-execution-phasing-and-sign-off.md +++ b/docs/testing/master-test-plan/19-execution-phasing-and-sign-off.md @@ -16,7 +16,7 @@ countable go/no-go. Three conventions before anything else, because mixing them is the single easiest way to corrupt this plan: -- **Phase IDs here are `E0`–`E7`** (*execution*). [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../../docs/testing/FEATURE-COVERAGE-PLAN.md) +- **Phase IDs here are `E0`–`E7`** (*execution*). [`docs/testing/FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) already owns `P0`–`P7` for its own phase roadmap and `FCP:CRIT-n`/`FCP:STORE-n`/`FCP:PIPE-14`-style **gap** IDs in a *different* ID space from this plan's row IDs. Never write "P2" meaning a phase of this plan, and never assume `STORE-10` means the same row in both documents — it does not. @@ -357,7 +357,7 @@ this phase is a *set of parallel tracks*, not a serial block. | Real directory | AD DS domain: writable DC + DNS + Kerberos SPN + a real OIDC IdP — **plus the owner-held lab runbook released to whoever builds it** (see §18.10) | 27 `AUTH` rows: `AUTH-07`, `AUTH-08`, `AUTH-10`..`AUTH-13`, `AUTH-15`..`AUTH-24`, `AUTH-28`, `AUTH-33`, `AUTH-34`, `AUTH-36`, `AUTH-38`..`AUTH-40`, `AUTH-51`, `AUTH-52`, `AUTH-58`, `AUTH-64` | **7 are campaign-gate P0s** — `AUTH-08`, `AUTH-11`, `AUTH-12`, `AUTH-17`, `AUTH-19`, `AUTH-20`, `AUTH-28`. ADR 0142 status is literally "code COMPLETE, awaiting lab validation". `AUTH-09`, `AUTH-26`, `AUTH-27` and `AUTH-29` are P0 but run on `dev-PC` — do **not** park them here | | Two-box HA / domain | A second Windows host; optionally an AlwaysOn AG lab and a k8s cluster | `HA-36`..`HA-40`, `HA-51`, `HA-52` | **`HA-52` is the campaign-gate P0** (floating VIP / L4 LB, real-sender reconnect through a VIP move). The DB-restart-under-load drill is already owned manually by **`W25:S4.10`** — build only the automated arm. `HA-03`, `HA-04`, `HA-06`, `HA-22` and `HA-45` were previously parked here in error: they run on `container-CI` / `dev-PC` and belong in E1/E2 | | Promotion pipeline (the **two-engine publishing rig**) | A non-production **and** a production-like engine, separately administered (`PUB2`) | `PUB-11`, `PUB-46`, `PUB-69` | **`PUB-11` is the campaign-gate P0.** Cross-node fingerprint divergence needs two real nodes, and a promote against localhost cannot exercise the remote `config_dir: null` path. `PUB-10`, `PUB-18`, `PUB-20`, `PUB-21`, `PUB-32`, `PUB-33` and `PUB-53` run on `container-CI` and belong in E3 | -| Host acceptance | The W2025 box | Owned by [`WIN2025-TEST-PLAN`](../../docs/testing/WIN2025-TEST-PLAN.md) — **do not restate** | This plan contributes only the automated arms it converts | +| Host acceptance | The W2025 box | Owned by [`WIN2025-TEST-PLAN`](../WIN2025-TEST-PLAN.md) — **do not restate** | This plan contributes only the automated arms it converts | | Upgrade & vintage | A store written by a prior engine vintage; both wheels | `MIG-01`, `MIG-02`, `MIG-06`..`MIG-17`, `MIG-47`, `MIG-49` | Console seam bump bricks the whole engine, not just `/ui`. `MIG-18` is a pointer row — no separate work | | Reconcile PHI fix | none (prerequisite, do in E3/E4) | `MIG-36`, `MIG-37`, `MIG-38` | **Blocking prerequisite for `W25:S3.10`** — the reconcile harness leaks field values by default | | Real-hardware SQL Server | The existing self-hosted `mefor-win2025-sql` VM | `STORE`, `HA`, `PIPE` server-DB rows re-run on real Windows + real ODBC | No procurement: the runner exists, dispatch-only; just turn the VM on | @@ -518,8 +518,8 @@ only (SQL Server / PostgreSQL) · `1` = once, backend-independent · `M` = manua **What runs on the W2025 box.** `PIPE`, `STORE`, `PARSE`, `API`, `ALERT`, `PERF` and `MIG` at `x3` (once per backend, under the NSSM service identity); `HA` at `x2`; `CONN` at `x3` for the transports; `CFG` and `PUB` once; `AUTH`, `WEB`, `IDE`, `TRAY` and `SEC` as manual human-closed rows. -**The box's own acceptance content is owned by [`WIN2025-TEST-PLAN`](../../docs/testing/WIN2025-TEST-PLAN.md) -and stamped through [`WIN2025-TEST-MATRIX`](../../docs/testing/WIN2025-TEST-MATRIX.md) via +**The box's own acceptance content is owned by [`WIN2025-TEST-PLAN`](../WIN2025-TEST-PLAN.md) +and stamped through [`WIN2025-TEST-MATRIX`](../WIN2025-TEST-MATRIX.md) via `python -m harness.acceptance` — this plan adds the automated arms and does not re-run `W25:S0.5`'s "do NOT re-run on the box" list.** @@ -796,7 +796,7 @@ footnote. ### 22.1 Artifacts The per-tool artifact table is already written and correct — see -[`WIN2025-TEST-PLAN` §`W25:S6.1`](../../docs/testing/WIN2025-TEST-PLAN.md). **Do not restate it.** It covers +[`WIN2025-TEST-PLAN` §`W25:S6.1`](../WIN2025-TEST-PLAN.md). **Do not restate it.** It covers `verify --report-md/--report-json`, `check --json`, `graph --json`, `audit-verify` (which **exits 0 even on FAIL** — its output must be string-parsed), `harness.acceptance --report-md/--report-csv/--xlsx`, `harness --scenario`, `harness --load --report-json/--report-csv` with `--baseline`/`--tolerance`, @@ -926,7 +926,7 @@ by a later chapter re-auditing an earlier one. Each gets three things: 2. a regression test that fails on the pre-fix code; 3. a note naming **which of the six coverage dimensions missed it** (functional / security / performance / HA / PHI / cross-backend — the dimensions - [`FEATURE-COVERAGE-PLAN`](../../docs/testing/FEATURE-COVERAGE-PLAN.md) already defines). + [`FEATURE-COVERAGE-PLAN`](../FEATURE-COVERAGE-PLAN.md) already defines). Track the count per chapter per phase. A chapter whose escape count is rising is a chapter whose audit was wrong, and it gets re-audited — **not** a chapter that needs more rows. @@ -1003,7 +1003,7 @@ Part II corrected its own recon 17 times out of 17. That is the maintenance prob ### 25.1 The ADR-ships-with-test-IDs rule This is not new machinery — it is an extension of what the repo already enforces. -[`docs/adr/TEMPLATE.md`](../../docs/adr/TEMPLATE.md) already requires behavioural acceptance criteria +[`docs/adr/TEMPLATE.md`](../../adr/TEMPLATE.md) already requires behavioural acceptance criteria in EARS form, each linked with `→` to the test or fixture that verifies it, and `messagefoundry adr-analyze` already checks that each `→` link resolves to a real file. @@ -1011,7 +1011,7 @@ The extension: **an ADR that lands must also name the master-plan row ID it clos the chapter matrix must gain that row **in the same commit**. Same discipline, same commit boundary, as the ledger gate that already blocks an unallocated ADR or BACKLOG number (`scripts/hooks/ledger_check.py`, wired through `.pre-commit-config.yaml`; see -[`docs/LEDGER-GATE.md`](../../docs/LEDGER-GATE.md)). +[`docs/LEDGER-GATE.md`](../../LEDGER-GATE.md)). Three consequences worth stating plainly: @@ -1055,12 +1055,12 @@ this plan cites them rather than absorbing them: | Artifact | Owns | This plan's relationship | |---|---|---| -| [`FEATURE-COVERAGE-PLAN.md`](../../docs/testing/FEATURE-COVERAGE-PLAN.md) | The subsystem coverage-gap audit across six dimensions, phases `P0`–`P7`, gap IDs in its own space | Chapters cite it and re-grade stale rows **in the same commit** as the re-grading. `SEC`'s open question — extend it, or freeze it with this plan as standing owner — must be answered in E0 | -| [`WIN2025-TEST-PLAN.md`](../../docs/testing/WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../../docs/testing/WIN2025-TEST-MATRIX.md) + [`WIN2025-ACCEPTANCE.md`](../../docs/testing/WIN2025-ACCEPTANCE.md) | Host and service-identity acceptance on the box; the 54-row signed matrix; the `W25:S6.3` sign-off gate | **B4 cites `W25:S6.3` verbatim.** This plan adds only the automated arms and respects `W25:S0.5`'s do-not-re-run list | -| [`VERIFY.md`](../../docs/testing/VERIFY.md) | On-box deployment acceptance across the five `verify` sections | Cited as a per-release and per-backend tool; `CFG-40` covers only its untested report writers | -| [`LOAD-TESTING.md`](../../docs/LOAD-TESTING.md) + `harness/load/` | Throughput, connection-scale, failover-under-load, the profiles, GO/NO-GO verdicts and the falsifier practice | `PERF` adds gates and a filling term **around** the rig; it does not reimplement it | -| [`CI-QUALITY.md`](../../docs/CI-QUALITY.md) + [`docs/quality-gates/`](../../docs/quality-gates/) | The two-checkpoint story and the advisory-gate build handoff | §23 extends the shipped signals (coverage, mutation, liveness); it does not add a new gate framework | -| [`FEATURE-MAP.md`](../../docs/FEATURE-MAP.md) | The capability status catalog and the published posture claims | B6 pins the drifted rows; B8 re-confirms the risk acceptance it records | +| [`FEATURE-COVERAGE-PLAN.md`](../FEATURE-COVERAGE-PLAN.md) | The subsystem coverage-gap audit across six dimensions, phases `P0`–`P7`, gap IDs in its own space | Chapters cite it and re-grade stale rows **in the same commit** as the re-grading. `SEC`'s open question — extend it, or freeze it with this plan as standing owner — must be answered in E0 | +| [`WIN2025-TEST-PLAN.md`](../WIN2025-TEST-PLAN.md) + [`WIN2025-TEST-MATRIX.md`](../WIN2025-TEST-MATRIX.md) + [`WIN2025-ACCEPTANCE.md`](../WIN2025-ACCEPTANCE.md) | Host and service-identity acceptance on the box; the 54-row signed matrix; the `W25:S6.3` sign-off gate | **B4 cites `W25:S6.3` verbatim.** This plan adds only the automated arms and respects `W25:S0.5`'s do-not-re-run list | +| [`VERIFY.md`](../VERIFY.md) | On-box deployment acceptance across the five `verify` sections | Cited as a per-release and per-backend tool; `CFG-40` covers only its untested report writers | +| [`LOAD-TESTING.md`](../../LOAD-TESTING.md) + `harness/load/` | Throughput, connection-scale, failover-under-load, the profiles, GO/NO-GO verdicts and the falsifier practice | `PERF` adds gates and a filling term **around** the rig; it does not reimplement it | +| [`CI-QUALITY.md`](../../CI-QUALITY.md) + [`docs/quality-gates/`](../../quality-gates) | The two-checkpoint story and the advisory-gate build handoff | §23 extends the shipped signals (coverage, mutation, liveness); it does not add a new gate framework | +| [`FEATURE-MAP.md`](../../FEATURE-MAP.md) | The capability status catalog and the published posture claims | B6 pins the drifted rows; B8 re-confirms the risk acceptance it records | ### 25.5 Review cadence for the plan itself