Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 75 additions & 75 deletions docs/testing/master-test-plan/00-strategy-and-governance.md

Large diffs are not rendered by default.

96 changes: 48 additions & 48 deletions docs/testing/master-test-plan/01-environments-data-and-tooling.md

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions docs/testing/master-test-plan/02-pipeline-reliability.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,20 @@ destination.
In scope:

- The **3 + 1 stage** queue — `ingress → routed → outbound`, plus the optional `Stage.RESPONSE`
re-ingress token ([`store/store.py:330-357`](../../messagefoundry/store/store.py)).
re-ingress token ([`store/store.py:330-357`](../../../messagefoundry/store/store.py)).
- **ACK-on-receipt**: AA is built only after `enqueue_ingress` durably commits
([`pipeline/wiring_runner.py:3717-3745`](../../messagefoundry/pipeline/wiring_runner.py)), and the
([`pipeline/wiring_runner.py:3717-3745`](../../../messagefoundry/pipeline/wiring_runner.py)), and the
four pre-ACK failure classes that still **NAK synchronously** (decode `AR` :3497, NUL `AR` :3523,
strict-validate `AE` :3660, streaming-detach `AE` :3699).
- **Transactional stage handoff** — `route_handoff` / `transform_handoff` / the Step-A combined
`handoff` / `ingress_handoff` ([`store/base.py:289-366`, `:703-726`](../../messagefoundry/store/base.py)) —
`handoff` / `ingress_handoff` ([`store/base.py:289-366`, `:703-726`](../../../messagefoundry/store/base.py)) —
and the at-least-once re-run invariant that rests on them.
- **Crash recovery**: `reset_stale_inflight` across every stage, and its ownership-scoped form
(`OwnedLanes`, ADR 0073) for **engine shards** over one unified store
([`pipeline/engine.py:809-874`](../../messagefoundry/pipeline/engine.py)).
([`pipeline/engine.py:809-874`](../../../messagefoundry/pipeline/engine.py)).
- The **disposition finalizer as sole authority** and the seven-member `MessageStatus` set
(`RECEIVED / ROUTED / UNROUTED / PROCESSED / FILTERED / ERROR / NOT_DEPLOYED`,
[`store/store.py:311-319`](../../messagefoundry/store/store.py)).
[`store/store.py:311-319`](../../../messagefoundry/store/store.py)).
- **Seq-only per-lane FIFO** (ADR 0059), the claim family (`claim_next_fifo`,
`claim_next_fifo_batch` ADR 0058, `claim_fifo_heads` + `list_fifo_lanes` ADR 0066,
`release_claimed` / `reschedule_claimed` ADR 0070), pooled `StageDispatcher` (**the shipped
Expand Down
84 changes: 42 additions & 42 deletions docs/testing/master-test-plan/03-store-and-data-lifecycle.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

This chapter covers the **active-passive HA** model MessageFoundry actually ships: N engine nodes over **one shared server-DB store** (PostgreSQL or SQL Server), exactly one of which holds a self-fencing `leader_lease` row and runs the wired graph. Concretely:

- **Coordinator seam** — `ClusterCoordinator` Protocol, `NullCoordinator` (single-node byte-identical default), `DbCoordinator` (asyncpg), `SqlServerCoordinator` (aioodbc / `MERGE WITH(HOLDLOCK)`), and `build_coordinator` dispatch ([`pipeline/cluster.py:137-311`](messagefoundry/pipeline/cluster.py), [`:326-983`](messagefoundry/pipeline/cluster.py), [`:986-1058`](messagefoundry/pipeline/cluster.py), [`pipeline/cluster_sqlserver.py:78-549`](messagefoundry/pipeline/cluster_sqlserver.py)).
- **Coordinator seam** — `ClusterCoordinator` Protocol, `NullCoordinator` (single-node byte-identical default), `DbCoordinator` (asyncpg), `SqlServerCoordinator` (aioodbc / `MERGE WITH(HOLDLOCK)`), and `build_coordinator` dispatch ([`pipeline/cluster.py:137-311`](../../../messagefoundry/pipeline/cluster.py), [`:326-983`](../../../messagefoundry/pipeline/cluster.py), [`:986-1058`](../../../messagefoundry/pipeline/cluster.py), [`pipeline/cluster_sqlserver.py:78-549`](../../../messagefoundry/pipeline/cluster_sqlserver.py)).
- **Election + fencing** — acquire-or-renew in one statement against the DB clock (`cluster.py:840-900`), the DB-free monotonic fence watchdog (`cluster.py:902-935`), the H1 leader-epoch token and its store-side claim guard (`store/postgres.py:2686-2688, 2816-2820, 2934-2938`; `store/sqlserver.py:796-802, 6003-6007, 6152-6156, 6320`), clean release on stop (`cluster.py:937-962`), ADR 0096 leader preference / non-promotable standby (`cluster.py:871-888`, `config/settings.py:2896-2910`).
- **Leader-gated work** — the graph supervisor (`pipeline/engine.py:1154-1268`), on-promotion in-flight recovery (`pipeline/leader_tasks.py:143-162` on PostgreSQL; `store.reset_stale_inflight()` on SQL Server, `engine.py:1177-1187`), the periodic expired-lease reclaim sweep (`leader_tasks.py:52-141`), leader-gated poll-source intake, and the leader-only WRITE singletons — retention, reclaim, DR backup (`engine.py:953-990`, `pipeline/dr_backup.py:257-259`).
- **Observability** — `GET /cluster/status` and `GET /cluster/nodes` (`api/app.py:4643-4698`), the derived single-live-leader projection (`cluster.py:555-604`), `leadership_acquired` / `leadership_lost` alerts (`cluster.py:966-983`, `pipeline/alert_sinks.py:98-104, 807-824`), and the web console status page's cluster/nodes/DR tables and `leaderless → health=down` derivation (`messagefoundry_webconsole/pages/monitoring.py:471-508`, `routes/status.py:58-97`).
Expand Down
14 changes: 7 additions & 7 deletions docs/testing/master-test-plan/05-connections-and-transports.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,23 @@
## 4. Connections & Transports Matrix

**ID prefix:** `CONN` · **Surface:** engine (`messagefoundry/transports/`) with CI-leg, harness and W2025-box legs
· **Primary risk:** the shared frame decoder treats a start byte **inside an open frame** as ordinary payload ([`framing.py:102-126`](messagefoundry/transports/framing.py)), so a peer that aborts mid-frame and re-syncs by re-sending SB has two clinical messages silently concatenated into one — wrong clinical content downstream **and** a count-and-log record of one received message instead of two, with no error, no `ERROR` disposition and no connection event.
· **Primary risk:** the shared frame decoder treats a start byte **inside an open frame** as ordinary payload ([`framing.py:102-126`](../../../messagefoundry/transports/framing.py)), so a peer that aborts mid-frame and re-syncs by re-sending SB has two clinical messages silently concatenated into one — wrong clinical content downstream **and** a count-and-log record of one received message instead of two, with no error, no `ERROR` disposition and no connection event.

### 4.1 Scope & objectives

This chapter covers **every registered inbound/outbound connector and the connection-level machinery around them**:

- **17 `ConnectorType` values** ([`config/models.py:28-53`](messagefoundry/config/models.py)) across **24 registrations** and **19 `ConnectionSpec` factories** in [`config/wiring.py`](messagefoundry/config/wiring.py) (`MLLP`, `Tcp`, `X12`, `Http`, `File`, `Timer`, `Loopback`, `PassThrough`, `Rest`, `FHIR`, `Email`, `Direct`, `DICOM`, `DICOMweb`, `Database`, `DatabasePoll`, `Soap`, `Sftp`, `Ftp`).
- **Wire-level framing** — the shared delimiter codec ([`transports/framing.py`](messagefoundry/transports/framing.py): `FrameCodec`, `FrameDecoder.feed:102`, `codec_for:143`) and its MLLP binding (`MLLPDecoder`, [`mllp.py:285`](messagefoundry/transports/mllp.py)); partial, coalesced, oversized, truncated and interleaved frames.
- **17 `ConnectorType` values** ([`config/models.py:28-53`](../../../messagefoundry/config/models.py)) across **24 registrations** and **19 `ConnectionSpec` factories** in [`config/wiring.py`](../../../messagefoundry/config/wiring.py) (`MLLP`, `Tcp`, `X12`, `Http`, `File`, `Timer`, `Loopback`, `PassThrough`, `Rest`, `FHIR`, `Email`, `Direct`, `DICOM`, `DICOMweb`, `Database`, `DatabasePoll`, `Soap`, `Sftp`, `Ftp`).
- **Wire-level framing** — the shared delimiter codec ([`transports/framing.py`](../../../messagefoundry/transports/framing.py): `FrameCodec`, `FrameDecoder.feed:102`, `codec_for:143`) and its MLLP binding (`MLLPDecoder`, [`mllp.py:285`](../../../messagefoundry/transports/mllp.py)); partial, coalesced, oversized, truncated and interleaved frames.
- **MLLP** — source (`mllp.py:1299`), destination (`mllp.py:595`), TLS/mTLS (`_mllp_ssl_context:499`), ack modes (`build_ack:329`), ADR 0067 persistent (`mllp.py:643`), ADR 0124 `no_ack` delivery-on-write (`_send_once_no_ack:897`, `_send_persistent_no_ack:1046`), per-outbound encoding override (`:402`/`:427`).
- **Raw TCP** (`tcp.py:570-571`) and **X12 raw-TCP** (`x12.py:596-597`) — both permanently TLS-free by design ([ADR 0153](docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) decision 4 / BACKLOG #311 (above the published #231 baseline), `tcp.py:150-152`, which states the declaration is "PERMANENT, STRUCTURAL … not a transitional one").
- **File** (`file.py:891-892`) incl. ADR 0123 gzip, ADR 0129 `after_read='leave'` + `ProcessedFileLedger` ([`base.py:65-88`](messagefoundry/transports/base.py)), ADR 0132 alternate Windows credential ([`wincred.py`](messagefoundry/transports/wincred.py)); **RemoteFile** SFTP/FTP/FTPS (`remotefile.py:1016-1017`).
- **HTTP family** — `Rest` (`rest.py:1544`), `Soap` (`soap.py:844`, WS-Security/mTLS ADR 0015, WSDL import ADR 0122), `FHIR` (`fhir.py:647`) + SMART Backend Services ([`smart.py`](messagefoundry/transports/smart.py)), `DICOMweb` STOW-RS (`dicomweb.py:382`), generic outbound auth ([`http_auth.py`](messagefoundry/transports/http_auth.py)), ADR 0126 forward proxy.
- **Raw TCP** (`tcp.py:570-571`) and **X12 raw-TCP** (`x12.py:596-597`) — both permanently TLS-free by design ([ADR 0153](../../adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) decision 4 / BACKLOG #311 (above the published #231 baseline), `tcp.py:150-152`, which states the declaration is "PERMANENT, STRUCTURAL … not a transitional one").
- **File** (`file.py:891-892`) incl. ADR 0123 gzip, ADR 0129 `after_read='leave'` + `ProcessedFileLedger` ([`base.py:65-88`](../../../messagefoundry/transports/base.py)), ADR 0132 alternate Windows credential ([`wincred.py`](../../../messagefoundry/transports/wincred.py)); **RemoteFile** SFTP/FTP/FTPS (`remotefile.py:1016-1017`).
- **HTTP family** — `Rest` (`rest.py:1544`), `Soap` (`soap.py:844`, WS-Security/mTLS ADR 0015, WSDL import ADR 0122), `FHIR` (`fhir.py:647`) + SMART Backend Services ([`smart.py`](../../../messagefoundry/transports/smart.py)), `DICOMweb` STOW-RS (`dicomweb.py:382`), generic outbound auth ([`http_auth.py`](../../../messagefoundry/transports/http_auth.py)), ADR 0126 forward proxy.
- **DICOM DIMSE** — C-STORE SCP source (`dicom.py:719`) and C-STORE SCU + C-ECHO destination (`dicom.py:689`/`:720`).
- **DATABASE** destination + DB-IN poll source (`database.py:947-948`) and the read-only `db_lookup` carve-out (ADR 0010); the `fhir_lookup` carve-out (ADR 0043, `fhir.py:991`).
- **Email/SMTP** (`email.py:265`, ADR 0029) and **Direct S/MIME-over-SMTP** (`direct.py:373`, ADR 0085 PR1).
- **Inbound HTTP/1.1 listener** (`http_listener.py:488`, ADR 0023) and the internal/generative sources: `Timer` (`timer.py:341`), `Loopback` (`loopback.py:53`), `PassThrough` (`passthrough.py:68`).
- **Connection lifecycle + operator seams** — the nine `/connections/{name}/…` routes ([`api/app.py:1915,1924,1933,1944,1973,2023,2072,2141,2294`](messagefoundry/api/app.py)), the 17 `test_connection` reachability overrides against the `TestNotSupportedError` default (`base.py:288`/`:327`), the eight-arm fail-closed egress allow-list ([`settings.py` `EgressSettings`](messagefoundry/config/settings.py): `allowed_mllp`, `allowed_tcp`, `allowed_file_dirs`, `allowed_http`, `allowed_db`, `allowed_remote`, `allowed_smtp`, `allowed_direct`, plus `deny_by_default`), `connections.toml` round-trip, connection-level fault isolation, the ADR 0021 §7 connection-event log and ACK capture, and the TLS cert-expiry monitor ([`pipeline/cert_expiry.py:98-130`](messagefoundry/pipeline/cert_expiry.py)).
- **Connection lifecycle + operator seams** — the nine `/connections/{name}/…` routes ([`api/app.py:1915,1924,1933,1944,1973,2023,2072,2141,2294`](../../../messagefoundry/api/app.py)), the 17 `test_connection` reachability overrides against the `TestNotSupportedError` default (`base.py:288`/`:327`), the eight-arm fail-closed egress allow-list ([`settings.py` `EgressSettings`](../../../messagefoundry/config/settings.py): `allowed_mllp`, `allowed_tcp`, `allowed_file_dirs`, `allowed_http`, `allowed_db`, `allowed_remote`, `allowed_smtp`, `allowed_direct`, plus `deny_by_default`), `connections.toml` round-trip, connection-level fault isolation, the ADR 0021 §7 connection-event log and ACK capture, and the TLS cert-expiry monitor ([`pipeline/cert_expiry.py:98-130`](../../../messagefoundry/pipeline/cert_expiry.py)).
- **Partner-side failure injection** — slow peer, accept-and-never-ACK, half-open socket, abortive RST, cert expiry.

**Explicitly NOT in scope here** — cited, never re-planned:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

This chapter covers the **configuration bundle and the command-line surface that reads, writes and gates it**:

- **Code-first wiring loader + `Registry`** — `load_config` ([`messagefoundry/config/wiring.py:3793`](messagefoundry/config/wiring.py)), `validate_config` (`:4213`), `Registry` (`:2779`), the `_*` helper-module skip and the `_SiblingHelperFinder` restricted to `_`-prefixed names (`:3729`, SEC-019 / CWE-427).
- **Code-first wiring loader + `Registry`** — `load_config` ([`messagefoundry/config/wiring.py:3793`](../../../messagefoundry/config/wiring.py)), `validate_config` (`:4213`), `Registry` (`:2779`), the `_*` helper-module skip and the `_SiblingHelperFinder` restricted to `_`-prefixed names (`:3729`, SEC-019 / CWE-427).
- **Config-source trust** — POSIX group/world-writable + foreign-owner refusal (`wiring.py:4147`, `:4124`) and the Windows NTFS-DACL guard (`_evaluate_config_dacl:3889`, `_assert_safe_config_source_windows:3917`, ADR 0036), plus the `MEFOR_ALLOW_INSECURE_CONFIG_SOURCE` dev escape (`settings.py:244/247`).
- **`connections.toml` config-as-data (ADR 0007)** — the read path (`connections_file.py:156/273`, `_TRANSPORTS:70`), the comment-preserving write path (`connections_edit.py:161/202/349/363`), the `connection list|upsert|remove|schema` CLI (`__main__.py:3788`), the `connection schema` form contract (`connection_schema.py:56`), and the console→TOML `flagged` write seam (ADR 0007 amendment).
- **Service settings** — precedence CLI > env(`MEFOR_*`) > toml > default (`settings.py:4128`), the `MEFOR_<SECTION>_<KEY>` parser (`:3777`) against `_SECTIONS` (`:109`), the ADR 0118 `[security]` desugar + relocated-key rejection (`:3861`, `:3848`), secrets-in-file warning (`:3789`), and `extra="ignore"` tolerance (`ServiceSettings:3623`).
Expand Down
26 changes: 13 additions & 13 deletions docs/testing/master-test-plan/08-publishing-and-promotion.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,37 @@
This chapter owns **the whole path by which authored configuration reaches a running engine**, and
specifically the **non-production vs production** distinction. Concretely:

- **The IDE Stage → Promote flow** — [`ide/src/promote.ts`](../../ide/src/promote.ts) (validate →
- **The IDE Stage → Promote flow** — [`ide/src/promote.ts`](../../../ide/src/promote.ts) (validate →
environment pick → engine-shard pick → host policy → env-aware dry-run pre-flight → modal confirm →
apply), [`ide/src/promoteTarget.ts`](../../ide/src/promoteTarget.ts) (pure target resolution),
[`ide/src/engineTarget.ts`](../../ide/src/engineTarget.ts) (SEC-005 host policy).
apply), [`ide/src/promoteTarget.ts`](../../../ide/src/promoteTarget.ts) (pure target resolution),
[`ide/src/engineTarget.ts`](../../../ide/src/engineTarget.ts) (SEC-005 host policy).
- **The engine reload contract** — `POST /config/reload`
([`messagefoundry/api/app.py:2741-2891`](../../messagefoundry/api/app.py)), reload-root confinement
(`Engine._resolve_reload_target`, [`engine.py:1508-1521`](../../messagefoundry/pipeline/engine.py)),
([`messagefoundry/api/app.py:2741-2891`](../../../messagefoundry/api/app.py)), reload-root confinement
(`Engine._resolve_reload_target`, [`engine.py:1508-1521`](../../../messagefoundry/pipeline/engine.py)),
`config:deploy` + `require_step_up`, ADR 0041 D2 dual-control, and the quiesce-and-swap in
`RegistryRunner.reload` ([`wiring_runner.py:3046-3184`](../../messagefoundry/pipeline/wiring_runner.py)).
`RegistryRunner.reload` ([`wiring_runner.py:3046-3184`](../../../messagefoundry/pipeline/wiring_runner.py)).
- **The web console config-deploy page** — `/ui/config` +
[`messagefoundry_webconsole/routes/config.py`](../../messagefoundry_webconsole/routes/config.py)
[`messagefoundry_webconsole/routes/config.py`](../../../messagefoundry_webconsole/routes/config.py)
(fixed `ReloadRequest(config_dir=None, dry_run=False)`) and the provenance badge in
[`pages/config.py`](../../messagefoundry_webconsole/pages/config.py).
[`pages/config.py`](../../../messagefoundry_webconsole/pages/config.py).
- **Attestation & attribution** — ADR 0041 D1 content fingerprint
([`config/fingerprint.py`](../../messagefoundry/config/fingerprint.py)), `GET /config/provenance`,
([`config/fingerprint.py`](../../../messagefoundry/config/fingerprint.py)), `GET /config/provenance`,
the `config_reload*` audit family, and ADR 0041 D3 wheel self-attestation
([`integrity.py`](../../messagefoundry/integrity.py)) where it interacts with a publish.
([`integrity.py`](../../../messagefoundry/integrity.py)) where it interacts with a publish.
- **Environment values and target isolation** — `environments/<env>.toml` + `MEFOR_VALUE_*`
([`config/environments.py`](../../messagefoundry/config/environments.py)), ADR 0050 project-root
([`config/environments.py`](../../../messagefoundry/config/environments.py)), ADR 0050 project-root
anchoring, deferred `env()` resolution *on the target*, and wrong-target / environment-crossing safety.
- **Multi-target publishing** — several named `messagefoundry.environments` entries, engine-shard
sub-targets within one environment, and cluster config-version convergence
([`pipeline/config_convergence.py`](../../messagefoundry/pipeline/config_convergence.py)).
([`pipeline/config_convergence.py`](../../../messagefoundry/pipeline/config_convergence.py)).
- **Atomicity, partial publish, rollback, and behaviour of in-flight messages / open connections across
a publish**, including publishing during a cluster failover.
- **Per-artifact-kind publish semantics** — Router/Handler `*.py`, `connections.toml`, `codesets/*`,
and (critically) the artifacts that are **not** publishable at all: alert rules, `[security]`/auth
config and AI policy in `messagefoundry.toml`.
- **Version control as the delivery mechanism** — [`docs/VERSION-CONTROL.md`](../VERSION-CONTROL.md),
the offline git init + `messagefoundry check` pre-commit hook
([`ide/src/sourceControl.ts`](../../ide/src/sourceControl.ts)), and the air-gapped `git bundle` path.
([`ide/src/sourceControl.ts`](../../../ide/src/sourceControl.ts)), and the air-gapped `git bundle` path.

**Explicitly NOT in scope here (owned elsewhere — cited, not restated):**

Expand Down
4 changes: 2 additions & 2 deletions docs/testing/master-test-plan/09-engine-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ harness at runtime, with no failing test on either side.
### 8.1 Scope & objectives

This chapter covers the single FastAPI application built by
`create_app()` / `create_managed_app()` ([`messagefoundry/api/app.py:1018`](messagefoundry/api/app.py),
`create_app()` / `create_managed_app()` ([`messagefoundry/api/app.py:1018`](../../../messagefoundry/api/app.py),
`:5156`) — verified live at **105 route objects** (104 `APIRoute` + the `/ws/stats`
`APIWebSocketRoute`; 67 declared in `api/app.py`, 38 in `api/auth_routes.py`), **109** with
`expose_docs=True`, **203** with `serve_ui=True`.
Expand All @@ -24,7 +24,7 @@ In scope:
Pydantic models (`api/models.py` 89 + `api/auth_models.py` 32); OpenAPI drift; cross-language
mirrors (`apiclient/client.py`, the IDE's TypeScript DTOs, the tray's `/health` key literal).
- **Deny-by-default authorization plumbing** as it is *enforced on the wire*: the nine factories in
[`api/security.py`](messagefoundry/api/security.py) — `require` (:182), `require_paced` (:250),
[`api/security.py`](../../../messagefoundry/api/security.py) — `require` (:182), `require_paced` (:250),
`require_service_cert` (:430), `require_phi_read` (:504), `require_step_up` (:569),
`require_reauth_only` (:609), `require_step_up_action` (:651), `require_reauth_only_action` (:694),
`authorize_ws` (:768) — plus `enforce_phi_read_hop` (:480), `enforce_phi_read_pacing` (:523),
Expand Down
Loading
Loading