Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 129 additions & 10 deletions docs/BACKLOG.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/adr/0074-adopter-capacity-estimator.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ If the halved rate is the **daily average**, the peak hour lands at 0.5 × 2.7 =
`harness/load/runner.py:59-65` constructs **one** `CorrelationSink` with `ports=tuple(sink_port + i for i in range(sink_ports))` — those are *ports inside one process*, and the ~135–144 msg/s cap is **per sink *process***. In the path §Method names, the cap **cannot be relieved at all**. Multi-*process* sinks exist only in `shardcert.py`'s drive, which this ADR never mentions, and it fails loud when `sink_count > dests` — so the **~36 msg/s single-hospital adopter with 1–2 outbound destinations can provision at most 1–2 sink processes**, hard-capping the *measurable* delivered rate with no remedy.

**B14 — Downstream artifacts carry the stale text and would ship the gate holes even if only this ADR is amended.** — ✅ **RESOLVED: all four now corrected** (see item 13 below).
`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; `docs/BACKLOG.md:4077-4079` (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule).
`docs/adr/README.md:104` (stale status; repeats "poller-zero ⇒ sub-ceiling knee" and "~135–144/s per sink" as settled guidance); `docs/testing/FEATURE-COVERAGE-PLAN.md:298, :1237` — **SCALE-16** directs building *"poller-zero knee **AC-4**"* first, as *"correctness/PHI-safety assertions **independent of any live number**"*: **AC-4 is not independent of a live number — it is circular (B6)**; BACKLOG #96 (the HR-4 source text); `docs/THROUGHPUT.md:260` (the aggregate-is-the-sum rule).

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ drives):
| `harness/load` (`--load` / `--failover` / `--estate`) | Performance, resilience | Reports metrics only; SLO verdicts and exit codes per [`docs/LOAD-TESTING.md`](../../LOAD-TESTING.md) §"Exit codes" |
| `harness/acceptance` (`python -m harness.acceptance`) | Host acceptance | Executes probes + re-runs backing pytest suites; **never fakes green** — unautomatable rows report `MANUAL` |
| `harness/reconcile` (`capture` / `compare`) | Migration / parallel run | Offline comparison core; normalizes engine-non-deterministic fields; exits non-zero on a real diff so it can gate a per-Connection sign-off |
| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](messagefoundry/verify/runner.py:22)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message |
| `messagefoundry verify` | Host + deployment acceptance | Sections `host,store,smoke,manual,federation` ([`verify/runner.py:22`](../../../messagefoundry/verify/runner.py)); `--smoke self` is side-effect-free, `--smoke live` sends **one** synthetic message |
| `messagefoundry check` | Pre-merge config gate | Required checks `validate`, `dryrun`, `posture`, `build-check`, `reference-backend`; `ruff`/`mypy`/`raise-fstring`/`accepts-candidate` are **advisory and never block** |
| CI legs (`ci.yml`) | All automated levels | `test` + `ide` + `ci-gate` on every PR; the heavy legs (`sqlserver-store`, `postgres-store`, `load-test`, `load-test-sqlserver`, `windows-service-smoke`, `docker-smoke`) run **nightly at 03:17 UTC** or on `workflow_dispatch`, plus a PR path-gate arm where `changes` provides one |
| `quality-advisory.yml` | Meta / test-quality | `complexity`, `clone`, `coverage` (diff-cover), `mutation` — all **advisory, never required**; the `liveness` job is the meta-gate that demands proof a signal actually executed |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ bare `HA-nn` in this chapter always means **this chapter's own §3.4 row** — t
| The PG store-side epoch guard is live-proven nowhere | `test_resumed_ex_leader_is_fenced_after_real_handover` exists in both live suites but only the SS one runs. The PG claim guard appears in **three** separate statements (`postgres.py:2686-2688, 2816-2820, 2934-2938`). | One missed append across three claim paths is invisible to fake-pool tests and shows up only as duplicate/out-of-order delivery under a real handover | No | P1 |
| DR readiness is asserted nowhere on the box that needs it | `messagefoundry verify`'s sections are exactly `("host", "store", "smoke", "manual", "federation")` (`messagefoundry/verify/runner.py:22`) — no cluster, HA or DR group. `harness/acceptance/matrix.py:429-470` has G1-G5 with no DR row; `WIN2025-TEST-MATRIX.md` §G likewise. | An operator gets zero signal about DEK availability at the DR site, backup-destination writability, or cluster posture | No | P1 |
| Windows port-rebind lag is a one-off manual number | Captured only as prose in `W25:S4.9`; not in the report JSON, no trend, no threshold. | On the intended production OS this dominates observed recovery; a regression is indistinguishable from normal | Manual only | P2 |
| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. `BACKLOG.md:152` (#100) still reads "P1" though it shipped 2026-07-10 (`BACKLOG.md:4229`); `BACKLOG.md:159` (#101) reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 |
| Catalog drift understates the area | `FEATURE-MAP.md` §6's nine rows omit ADR 0047/0048/0049/0056/0096, `/cluster/nodes`, the H1 fence, on-promotion recovery and the DR run-profile. BACKLOG #100's ranked-backlog Tier cell still reads "P1" though it shipped 2026-07-10 per its own SHIPPED banner; BACKLOG #101's ranked-backlog Tier cell reads "DEMAND-GATE" though ADR 0096 is Accepted and the code is in `settings.py:2896-2910`. | Readers conclude DR standby, turnkey backup and cloud HA packaging do not exist, so they are never scoped for testing or release notes | No | P2 |

### 3.4 Test matrix

Expand Down Expand Up @@ -180,7 +180,7 @@ deliverables that STORE-44 and STORE-46 point at — do not descope them here.
| HA-56 | Config-only `.mfbak` + `DbaDelegatedError` on a live server DB — **pointer** | Cross-backend | — | — | — | T | P1 | Covered by STORE-45; no separate work scoped. (The files themselves reach CI through HA-02's wiring; the assertion is STORE's.) |
| HA-57 | FEATURE-MAP drift guard for Accepted HA/DR ADRs — **pointer** | Compat | — | — | — | T | P1 | Covered by the MIG FEATURE-MAP drift-guard row (one consolidated row extending `tests/test_feature_map_claims.py`); no separate work scoped. HA's contribution is the input list — ADR 0047, 0048, 0049, 0096, 0102 named in `docs/FEATURE-MAP.md` §6, and ADR 0056 carried with a non-shipped marker. |
| HA-58 | Correct the two stale FEATURE-COVERAGE-PLAN rows | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `FCP:HA-20`'s "Web console has no HA surface / coverage: none" (§16, `FEATURE-COVERAGE-PLAN.md:1201`) is corrected to reflect `monitoring.py:471-508` + `routes/status.py:88-97` + `test_webui.py:1437-1532`; `FCP:DEPLOY-26`'s "coverage: none" (§24, line 1552) is reconciled with the same document's P6 closure (line 317). Outcome is a dated doc edit, not a falsifiable assertion — the recurring guard is HA-57's owner row. |
| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: `docs/BACKLOG.md:152` (#100, `MultiSubnetFailover`) changes from "P1" to SHIPPED, consistent with its own detail section at line 4229; `docs/BACKLOG.md:159` (#101, leader preference / non-promotable) changes from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. |
| HA-59 | Correct the two stale BACKLOG status cells | Compat | manual | dev-PC | n/a | C | P2 | Editorial: BACKLOG #100 (`MultiSubnetFailover`) changes its ranked-backlog Tier cell from "P1" to SHIPPED, consistent with its own SHIPPED banner; BACKLOG #101 (leader preference / non-promotable) changes its ranked-backlog Tier cell from "DEMAND-GATE" to SHIPPED, consistent with Accepted ADR 0096 and `settings.py:2896-2910`. |
| HA-60 | Live PG expired-lease reclaim sweep does not steal a live sibling's rows | HA/Resilience | pytest | container-CI | x2 (PG) | T | P1 | On real PostgreSQL with two owners: rows leased by a crashed owner past `lease_ttl_seconds` are re-pended by `reclaim_expired_leases`, while rows leased by a live, renewing owner are untouched. Asserts the multi-node-safety claim in `leader_tasks.py:129-133` against real SQL rather than the single-node path. |
| HA-61 | `[store].lease_ttl_seconds` is inert on SQL Server | Cross-backend | pytest | container-CI | x2 (SS) | T | P2 | On a SQL Server store the setting changes no observable claim/recovery behaviour, and `reclaim_expired_leases` is absent from the SS store surface, so the SS coordinator's `reclaims_inflight()` path never reaches it. Locks in the documented asymmetry (`AOAG-DEPLOYMENT.md:527-531`) that operators otherwise mis-tune. |
| HA-62 | Live PG reference-set and transform-state convergence across nodes | Cross-backend | pytest | container-CI | x2 (PG) | T | P1 | On a real two-node PG cluster: the leader materializes a declared `Reference(...)` set, the follower read-throughs the shared snapshot without re-materializing; a transform-state write on one node bumps the namespace `state_version` and reaches the sibling's cache within one convergence interval. Today only the SS half has live coverage (`test_sqlserver_coordinator.py:183-200`). |
Expand Down
4 changes: 2 additions & 2 deletions docs/testing/master-test-plan/06-parsing-and-codecs.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@
| Codec extras silently uninstall | 41 test files use `importorskip`; there is no skip-counting guard in `tests/conftest.py` or `pyproject.toml` `addopts` | ~100 DICOM/FHIR/X12/XML/WSDL tests vanish and the build is GREEN. `pyproject.toml:119-123` documents exactly this class of breakage (annotated-types 0.8.0 dropping `SLOTS`) already happening once | No — the existing plan flags it for `FCP:X12-6` only | P1 |
| `parse_tree()` has its own separator derivation | `tree.py:65-72` blind-slices `msh[4:8]` and takes `sub_sep = enc[3]`. **Verified divergence:** for a legal 3-character MSH-2 (`^~\`), the tree derives `sub_sep = '\|'` (the *field* separator) while `_extract_separators` correctly returns `'&'`. Leaves are also never unescaped, so the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien` | The operator's only structural view of what actually arrived renders wrong in **both** the web console and the harness Qt pane, while routing stays correct — nothing else signals it | No. `test_parse_tree.py` is 8 tests over one default-separator ADT | P1 |
| ADR 0054 AC-6 (≥6× multi-core, ~14× single-thread on cp314t) has no executable gate | `test_benchmark_parser.py:207-235` is `skipif`-not-freethreaded and asserts only `ratio > 1.0`; `freethread-smoke.yml` runs only `test_parsing.py` + `test_wiring.py`, so the measure never executes anywhere | The entire justification for reimplementing python-hl7 (ADR 0053/0052) is unverified on any automated surface; only the crude `> 200 msg/s` floor would catch an orders-of-magnitude fall | Partially — the floor exists, the scaling gate does not | P1 |
| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `<soap:body parts=…>` selector. Disclosed at `docs/BACKLOG.md:3188` | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 |
| WSDL multi-part message selection | `parsing/xml/wsdl.py:290` `_body_element_for_message` returns `parts[0].element` unconditionally, never reading the binding's `<soap:body parts=…>` selector. Disclosed at BACKLOG #69 | A WS-I-conformant multi-part `wsdl:message` makes validate-against-WSDL check the **wrong** element — a false PASS on a non-conformant body or a false FAIL on a valid one, silently either way | No test in `test_wsdl_import.py` covers it | P1 |
| No malformed/oversized corpus generator | `generators/_core.generate_message` emits only hl7apy-conformant bodies and raises on an unknown trigger; `messagefoundry generate` exposes only `--type/--triggers/--count/--out/--seed/--list/--json` (`__main__.py:349-362`) | The error/dead-letter path — the count-and-log invariant itself — is never exercised at load or repeatably. Both P0s above are exactly the shapes such a generator would emit. Blocks automating `W25:S4.8` (`WIN2025-TEST-PLAN.md:926,1044,1351`) | No — `W25:S4.8` bad input is GUI-injected manually | P1 |
| `ValidationResult.errors` unasserted PHI-redacted | `validate.py:91` appends `str(exc)` from hl7apy verbatim; `wiring_runner.py:3659` does pass it through `safe_text` before persisting, but nothing pins that the **library** result is safe for other callers (dry-run, IDE, harness) | A datatype/table conformance failure can carry the offending field VALUE into any surface that renders `result.errors` directly | Partly — the persisted path is scrubbed; the library return is not asserted | P2 |
| No property-based or coverage-guided fuzzing | `pyproject.toml` carries no hypothesis/atheris; both "fuzz" corpora (`test_parsing.py:228-244`, 15 cases; `test_builtin_hl7_hardening.py`) are hand-enumerated. `docs/quality-gates/HANDOFF-mutation-coverage.md:223` proposes mutmut over `parsing/` but is a handoff doc only | Hand-enumerated corpora find only shapes an author imagined — demonstrated by both P0s, each trivially reachable and each missed. Parsing is the largest attacker-influenced surface in the product | No | P2 |
Expand Down Expand Up @@ -390,6 +390,6 @@
9. **Does the code-set `UnmappedSink` persistence half get built?** If yes it creates a new PHI-at-rest surface needing encryption, audit and retention decisions, and the `_sink is None` pin (PARSE-48) becomes a pin on the *default*, not on the absence of the feature. **Blocks:** PARSE-48's framing.
10. **Should `parse_tree()` unescape leaf values so the operator's view matches the router's?** Today the viewer shows `O\S\Brien` where `Peek.field` returns `O^Brien`. Either answer is defensible; neither is written down or asserted. **Blocks:** PARSE-24, PARSE-51, PARSE-52.
11. **Does `validate(profile=…)` stay a silent no-op (`validate.py:56`), or raise `NotImplementedError` until conformance profiles exist?** A caller passing a profile today gets an `ok=True` that means nothing on a safety-critical strict feed. **Blocks:** PARSE-19.
12. **Is the WSDL `<soap:body parts=…>` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at `docs/BACKLOG.md:3188`. **Blocks:** PARSE-31 — a test written now would encode the defect.
12. **Is the WSDL `<soap:body parts=…>` multi-part defect (`wsdl.py:290`) in scope to fix, or accepted while WSDL import stays demand-gated?** It is disclosed at BACKLOG #69. **Blocks:** PARSE-31 — a test written now would encode the defect.
13. **Should the stale ratings and doc drift be corrected as part of this work?** Concretely: `FEATURE-COVERAGE-PLAN.md` `FCP:PARSE-14` (`:859`) says "partial" (closed by `test_builtin_hl7_hardening.py:595-703`) and `FCP:PARSE-15` (`:860`) says the bench file is "MISSING" (it exists, 235 lines); `docs/HL7-VALIDATION.md:10` still names python-hl7 as the Tier-1 engine though ADR 0054's built-ins parser is the default; ADR 0012 §5 still calls the X12 strict validator deferred though `parsing/x12/validate.py` ships; ADR 0033's header still reads "Proposed … ratified-on-build" though the code sets are built; and `docs/FEATURE-MAP.md` §3 omits ADR 0054, 0122, 0123, 0033, `parsing/xml/`, `consistency.py`, `split.py` and `sniff.py`. **Blocks:** nothing technically — but the acceptance report and this plan both cite those documents as authority.
14. **Is a silent U+FFFD substitution an acceptable ingress outcome?** `normalize()` defaults to `errors="replace"` (`peek.py:152`) precisely so the hot path keeps routing a slightly-off body — but a substitution is an unrecorded, irreversible corruption of every *derived* value while the raw stays byte-faithful, which is count-and-log-clean by construction. Options: keep `replace` and record a warning/`ERROR` disposition when the decoded text differs from a strict decode; switch the ingress decode to `errors="strict"` and let it NAK (PARSE-43's path); or declare it accepted-by-design in `docs/CONNECTIONS.md`. **Blocks:** PARSE-64's unconditional half and the framing of PARSE-43.
Loading
Loading