From a02208d68a4f65dd0371afa529398c63a076d84c Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 14:04:50 -0500 Subject: [PATCH] docs: repoint 73 backlog citations at the archive (BACKLOG #1095) Retiring a backlog item moves it verbatim from docs/BACKLOG.md into docs/archive/backlog/BACKLOG-CLOSED.md. Every citation that named the live file keeps pointing at a file the item is no longer in. The link still resolves, so nothing in CI can see it. #1094 fixed two such markers in CLAUDE.md section 12; this is the same defect at repo scale. 73 citations across 34 files, href-only. No prose was rewritten. Visible labels changed ONLY where leaving them would contradict the target -- a label reading `BACKLOG.md` pointing at the archive -- and then only to `BACKLOG-CLOSED.md`. THE TEST IS "DOES THE CITED FILE CONTAIN THE ITEM", NOT "IS THE ITEM CLOSED". Those differ, and keying on closure would corrupt correct citations: #1073 is closed and still legitimately in the live ledger. Item locations came from parse_items imported from scripts/docs/backlog_status_check.py, per CLAUDE.md section 11 -- never a hand-rolled scan of the banner alphabet. DELIBERATELY NOT TOUCHED, each for a stated reason: Both ledger files -- ZERO edits to docs/BACKLOG.md and BACKLOG-CLOSED.md. Only two sites named them and both are excluded, so this change costs no conflict against the merge trains or the pending #1096 filing. The one real site (#322 at BACKLOG.md:2720) is left because the file is the most contended in the repo and the item number is visible in plain text a search away. docs/CONNECTIONS.md:2436 -- the #27 serial/ASTM row. Already fixed on a branch inside merge train #274. Sweeping it from origin/main would re-fix stale text and collide. QUOTATIONS OF THE DEFECT. docs/BACKLOG.md:6319, inside #1094, reads "Two of its markers CITED [`docs/BACKLOG.md`](BACKLOG.md) #26 and #27" -- past tense, describing rot that is already fixed. Repointing it would corrupt a historical record. A regex cannot tell this from a live pointer, which is the reason this was not done with sed. THE WRONG-NUMBER CLASS, which is a different defect and must not be swept into this one. ADR 0068:10 cites #11 and ADR 0113:9 cites #239; both numbers are absent from the live ledger, but the ARCHIVE's #11 ("`check` dry-run cross-products") and #239 ("Re-measure Steps view estate coverage") are unrelated to WebAuthn passkeys and to a Windows tray manager respectively. Repointing would convert a vague reference into a confidently wrong one that lands the reader on the wrong item. Left, and reported. MIXED-LOCATION LINKS, where one link covers items in both files so no single target is correct: docs/AI-OFF-MATRIX.md:50 (six items), docs/adr/0001:13 (#1 archived, #3 live), THROUGHPUT-IMPROVEMENTS.md:215 (#62 live, so its link is already correct). VERIFICATION - Plan applied by literal replacement on the named line only, requiring the quoted string to occur EXACTLY ONCE there; a mismatch aborts rather than fuzzy-matching. Dry run: 73/73 clean, 0 problems, before anything was written. - All 35 distinct anchor fragments introduced match exactly one real "## N." heading in the archive, checked after applying, with a known-bad fragment run through the same check to prove it can report a miss. Fragments were derived with a slugger that does NOT collapse consecutive spaces -- the doubled hyphens are correct, not typos. - All 74 archive hrefs in the changed files resolve to the archive from their own directory depth; the relative prefix differs by depth and was computed per file, not pattern-matched. - Coverage confirmed with a DELIBERATELY LOOSER regex than the one that built the work list: it finds exactly one wrong-file site outside this change set, docs/CONNECTIONS.md:2436, which is the intended exclusion. - backlog_status_check.py: OK, 365 items. No mixed line endings introduced. All 34 changed files are markdown; diff is 67 insertions / 67 deletions, line-for-line. - Staged by explicit path from the plan, cross-checked against git's modified set, so nothing another session is editing was swept in. Not included: the broken-href class (13 sites, mostly (docs/BACKLOG.md) written from inside docs/testing/master-test-plan/), the 12 line anchors past EOF, and the 31 in-range anchors that drifted onto unrelated text. Those are separate classes under #1095 and are catchable by a link checker, which this repo still does not run. --- CHANGELOG.md | 6 +++--- docs/AI-OFF-MATRIX.md | 2 +- docs/CONFIGURATION.md | 6 +++--- docs/DEPLOYMENT.md | 2 +- docs/DICOM.md | 2 +- docs/PHI.md | 2 +- docs/POSITIONING.md | 2 +- docs/REMOTE-CONSOLE.md | 2 +- docs/SECURITY-LOOSENING.md | 2 +- ...4-smart-backend-services-token-provider.md | 2 +- docs/adr/0025-dicom-codec-store-connectors.md | 2 +- docs/adr/0026-off-box-egress-update-check.md | 4 ++-- docs/adr/0028-base64-binary-carriage-codec.md | 2 +- docs/adr/0029-email-smtp-destination.md | 2 +- docs/adr/0032-console-desktop-launch.md | 20 +++++++++---------- ...loud-kubernetes-ha-deployment-packaging.md | 2 +- ...50-single-project-root-config-anchoring.md | 2 +- docs/adr/0065-web-ops-dashboard.md | 12 +++++------ ...8-browser-webauthn-passkeys-offloopback.md | 2 +- ...service-manager-stdlib-ctypes-tokenless.md | 2 +- ...s-phi-at-rest-posture-hl7-aware-compare.md | 2 +- ...type-tree-validate-against-wsdl-no-zeep.md | 4 ++-- ...oss-backend-processed-file-dedup-ledger.md | 2 +- ...s-no-pywin32-no-impersonation-privilege.md | 2 +- ...d-txn-discard-at-the-acquire-chokepoint.md | 2 +- docs/adr/README.md | 2 +- .../throughput/THROUGHPUT-EXECUTION-PLAN.md | 8 ++++---- .../throughput/THROUGHPUT-IMPROVEMENTS.md | 16 +++++++-------- .../throughput/throughput-build-plan.md | 4 ++-- docs/archive/throughput/throughput-roadmap.md | 6 +++--- docs/design/freethread-spike.md | 2 +- docs/message-ordering-design.md | 2 +- .../cloud-deployment-research-2026-06.md | 2 +- docs/research/config-ux-review.md | 2 +- 34 files changed, 67 insertions(+), 67 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 855440cc..d0faffd8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,7 +49,7 @@ All notable changes to MessageFoundry are documented here. The format follows is byte-identical. An **undeclared** proxy (`web_console_public_address` set, no `tls_terminated_upstream`) deliberately still does not refuse — exposure there would be an inference — but it no longer passes in silence: a new warning names single-factor admin directly on a PHI instance - with `require_mfa` off. ([BACKLOG #326](docs/BACKLOG.md), [ADR 0140](docs/adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md) amendment) + with `require_mfa` off. ([BACKLOG #326](docs/archive/backlog/BACKLOG-CLOSED.md#326-mfa-at-exposure-refusal-reads-serve_ui-after-it-is-flipped-off), [ADR 0140](docs/adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md) amendment) - **BREAKING — an `[[alerts.rules]]` block that routes to an unconfigured transport now refuses at startup instead of being silently ignored.** `notifier_from_settings` returned early when **no** transport was configured, *before* the loop that cross-checks each rule's `transports` against the @@ -119,7 +119,7 @@ All notable changes to MessageFoundry are documented here. The format follows displays the body it edits. **Who this would bite:** a deploying org whose admin had minted such a custom role; that role would have exceeded its stated scope (HIPAA minimum-necessary) on first deployment. No built-in role reaches it — `ADMINISTRATOR` and `OPERATOR` grant both permissions — - and every such read was already audited. ([BACKLOG #324](docs/BACKLOG.md)) + and every such read was already audited. ([BACKLOG #324](docs/archive/backlog/BACKLOG-CLOSED.md#324-custom-role-with-messagesedit-alone-reads-raw-phi-via-the-ui-editor)) ### Fixed - **A CR/LF inside an exception message could forge a whole log line on the text sink.** @@ -453,7 +453,7 @@ is additive / opt-in. gate never fired (adopters are pip + IT-covered), and it only ever shipped unsigned. **The desktop console is unaffected** — it stays installable via `pip install messagefoundry[console]` + the ADR 0032 Phase A `gui-script` and shortcut scripts; only the *frozen, zero-Python* conveyance is gone. The zero-install - audience is now served by the browser ops dashboard ([BACKLOG #75](docs/BACKLOG.md)). + audience is now served by the browser ops dashboard ([BACKLOG #75](docs/archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor)). ### Changed - **Server-DB store opens now skip the schema DDL batch when it already ran** ([ADR 0064](docs/adr/0064-schema-init-fastpath.md)). diff --git a/docs/AI-OFF-MATRIX.md b/docs/AI-OFF-MATRIX.md index 310aa32a..cf4c7f4e 100644 --- a/docs/AI-OFF-MATRIX.md +++ b/docs/AI-OFF-MATRIX.md @@ -47,4 +47,4 @@ with boxes. --- *Source deliverable for MULTISESSION-PLAN-7 (L0). Not a decision record — the decisions live in -[`docs/BACKLOG.md`](BACKLOG.md) (#26/#48/#84/#92/#104/#105) and [ADR 0072](adr/0072-traced-dryrun-mode.md).* +[`docs/BACKLOG.md`](BACKLOG.md) (#84/#105), [`archive/backlog/BACKLOG-CLOSED.md`](archive/backlog/BACKLOG-CLOSED.md) (#26/#48/#92/#104), and [ADR 0072](adr/0072-traced-dryrun-mode.md).* diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 56e94ff8..300238c1 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -1010,7 +1010,7 @@ best-effort and runs on a background task, so it never blocks or hangs a deliver | `email_from` | str | _unset_ | sender address (required for email) | | `email_to` | list | _unset_ | recipient(s) (required for email). Via env: comma-separated `MEFOR_ALERTS_EMAIL_TO` | | `email_use_tls` | bool | `true` | issue STARTTLS before sending. Selects TLS vs **cleartext** — it does not by itself decide whether the relay is authenticated; that is `email_tls_verify` | -| `email_tls_verify` | bool | `true` | verify the relay's certificate on that STARTTLS hop — chain + hostname + strict RFC 5280, TLS 1.2 floor ([#323](BACKLOG.md)). `false` keeps the session encrypted but accepts **any** certificate (MITM-able). Both `false` values are **loosenings**: `security_loosenings()` names them, `messagefoundry check`'s `alert-smtp-tls` advisory reports them, and on an enforcing PHI instance `serve` **refuses to start** unless `[security].allow_unverified_alert_smtp_tls` is also set | +| `email_tls_verify` | bool | `true` | verify the relay's certificate on that STARTTLS hop — chain + hostname + strict RFC 5280, TLS 1.2 floor ([#323](archive/backlog/BACKLOG-CLOSED.md#323-smtp-tls-is-unverified-on-all-three-send-paths)). `false` keeps the session encrypted but accepts **any** certificate (MITM-able). Both `false` values are **loosenings**: `security_loosenings()` names them, `messagefoundry check`'s `alert-smtp-tls` advisory reports them, and on an enforcing PHI instance `serve` **refuses to start** unless `[security].allow_unverified_alert_smtp_tls` is also set | | `email_tls_ca_file` | str | *(unset)* | PEM bundle of trust anchors for that hop. Unset = `[tls].internal_ca_file` if configured, else the OS trust store. A path, not a secret | | `email_username` | str | _unset_ | SMTP login user (omit for unauthenticated relays) | | `email_password` | str | _unset_ | **secret** — supply via `MEFOR_ALERTS_EMAIL_PASSWORD`, never the file (or use `email_password_secret`) | @@ -1496,8 +1496,8 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d | `encrypt_stored_data` | bool | `true` | PHI encrypted at rest (key from the environment) | | `allow_unencrypted_phi` | bool | `false` | audited escape: start a PHI instance with **no** key | | `allow_unencrypted_phi_under_strict_enforcement` | bool | `false` | the **second acknowledgment** required to start a PHI instance keyless under strict enforcement ([ADR 0140](adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md)). Under `enforcement = enforce`, `allow_unencrypted_phi = true` on its own is **not** enough — `serve` still refuses to start (exit 2) unless this is also set, so the highest-risk posture (real PHI + strict enforcement) is never one flag away from plaintext at rest. Under `enforcement = warn` the single `allow_unencrypted_phi` flag still governs. With both set the instance starts with PHI bodies, summary/metadata and the error columns **unencrypted at rest**, and the startup AUDIT line names **both** flags. A **loosening** — `security_loosenings()` reports it, so it is never silent | -| `allow_single_factor_admin_when_exposed` | bool | `false` | permit **single-factor admin on an exposed PHI instance** (ADR 0140). With `require_sign_in` on, `require_mfa` explicitly off, and the instance exposed — a **non-loopback bind**, **or** a declared TLS-terminating proxy (`[api].tls_terminated_upstream`) — a PHI instance under `enforcement = enforce` **refuses to start** (exit 2) — the Administrator role would authenticate with a single factor over the network. Setting this permits that start; it is recorded in a WARNING-level AUDIT line and the ordinary exposure warning still prints. A **loosening** — `security_loosenings()` reports it. **The exposure test does not consult the browser console** ([BACKLOG #326](BACKLOG.md); ADR 0140 amendment). It did, and that made the arm miss the topology this document recommends: a loopback bind behind a declared terminator with `serve_web_console` left at its default, where the ADR 0143 auto-degrade clears the console flag in place before the gate reads it. The exposed surface that authenticates with one factor is the **JSON operator API**, which the proxy serves whether or not `/ui` is mounted, so the predicate is the bind-and-proxy posture alone and the refusal fires on at least: an off-loopback bind; a declared proxy with the console left default-on; and a declared proxy with `serve_web_console = false`. **This refusal is the one exception to the "a new refusal fires only on a new opt-in" scoping rule** stated three rows below on `require_memory_encryption_declaration` — by owner ruling of 2026-08-04, recorded in the [ADR 0140](adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md) amendment, which is the single source for why. Nothing new gates it. **One residual is deliberately left open:** an **undeclared** proxy — `web_console_public_address` set with no `tls_terminated_upstream` — does not count as exposed here, because nothing was declared, so exposure would be an *inference*, and an inference must not refuse. It **warns** instead, on its own dedicated arm: on a PHI instance with `require_mfa` explicitly off, startup prints that if that origin is served by an undeclared proxy the Administrator role is single-factor over the network and this refusal cannot see it. Do **not** read the ADR 0068 §8 undeclared-proxy warning as that control — it is about the `/ui` session cookie and HSTS, and it is suppressed entirely when the ADR 0143 auto-degrade clears the console flag, which the same `web_console_public_address` triggers. **Prefer `require_mfa = true` — and know its scope.** Under the shipped `require_mfa_scope = "every_local_account"` it requires a second factor from **every** local account, *not* only Administrators, so a non-interactive **local** bearer-token service account becomes MFA-pending and cannot enrol unattended. **There are two remedies, not three.** Either make it a **directory (AD/Kerberos) principal** — those are out of scope under either value, their factor delegated to the directory — or set `require_mfa_scope = "administrators"` (itself reported as a loosening, and it leaves every local Administrator in scope) — see that row below. **mTLS is *not* the third.** A `[api].tls_client_cert_identities` mapping does grant a cert-identity that never meets the MFA gate, but that plane is admitted on exactly **one** route (`GET /service/identity`, `require_service_cert`) and carries no session, so an account "moved to mTLS" can read back its own identity and nothing else — it cannot replay, purge, poll status, or do any work a service account exists for. The `[api].tls_client_cert_identities` row above is the authority on that reach. Directory identities being out of scope also means an AD-only deployment is safe **for its AD users**; its local bootstrap admin and any local service accounts are still in scope | -| `allow_unverified_alert_smtp_tls` | bool | `false` | the **acknowledgment** required to start an enforcing PHI instance whose `[alerts]` SMTP hop does not authenticate the relay — i.e. `[alerts].email_use_tls = false` (cleartext) or `[alerts].email_tls_verify = false` (encrypted but accepts any certificate) ([#323](BACKLOG.md)). Covers BOTH shapes deliberately: cleartext is strictly worse than unauthenticated TLS, so gating only the second would hand an operator a bypass onto the worse posture. Without it `serve` refuses to start (exit 2); with it the start is permitted and named in a WARNING-level `AUDIT:` line. An **acknowledgment switch rather than the clamped `MEFOR_ALLOW_INSECURE_TLS` escape** the connectors use, because this cell is constructed outside the `active_hop_posture` scope where that clamp would be inert. A **loosening** — `security_loosenings()` reports it, so it is never silent | +| `allow_single_factor_admin_when_exposed` | bool | `false` | permit **single-factor admin on an exposed PHI instance** (ADR 0140). With `require_sign_in` on, `require_mfa` explicitly off, and the instance exposed — a **non-loopback bind**, **or** a declared TLS-terminating proxy (`[api].tls_terminated_upstream`) — a PHI instance under `enforcement = enforce` **refuses to start** (exit 2) — the Administrator role would authenticate with a single factor over the network. Setting this permits that start; it is recorded in a WARNING-level AUDIT line and the ordinary exposure warning still prints. A **loosening** — `security_loosenings()` reports it. **The exposure test does not consult the browser console** ([BACKLOG #326](archive/backlog/BACKLOG-CLOSED.md#326-mfa-at-exposure-refusal-reads-serve_ui-after-it-is-flipped-off); ADR 0140 amendment). It did, and that made the arm miss the topology this document recommends: a loopback bind behind a declared terminator with `serve_web_console` left at its default, where the ADR 0143 auto-degrade clears the console flag in place before the gate reads it. The exposed surface that authenticates with one factor is the **JSON operator API**, which the proxy serves whether or not `/ui` is mounted, so the predicate is the bind-and-proxy posture alone and the refusal fires on at least: an off-loopback bind; a declared proxy with the console left default-on; and a declared proxy with `serve_web_console = false`. **This refusal is the one exception to the "a new refusal fires only on a new opt-in" scoping rule** stated three rows below on `require_memory_encryption_declaration` — by owner ruling of 2026-08-04, recorded in the [ADR 0140](adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md) amendment, which is the single source for why. Nothing new gates it. **One residual is deliberately left open:** an **undeclared** proxy — `web_console_public_address` set with no `tls_terminated_upstream` — does not count as exposed here, because nothing was declared, so exposure would be an *inference*, and an inference must not refuse. It **warns** instead, on its own dedicated arm: on a PHI instance with `require_mfa` explicitly off, startup prints that if that origin is served by an undeclared proxy the Administrator role is single-factor over the network and this refusal cannot see it. Do **not** read the ADR 0068 §8 undeclared-proxy warning as that control — it is about the `/ui` session cookie and HSTS, and it is suppressed entirely when the ADR 0143 auto-degrade clears the console flag, which the same `web_console_public_address` triggers. **Prefer `require_mfa = true` — and know its scope.** Under the shipped `require_mfa_scope = "every_local_account"` it requires a second factor from **every** local account, *not* only Administrators, so a non-interactive **local** bearer-token service account becomes MFA-pending and cannot enrol unattended. **There are two remedies, not three.** Either make it a **directory (AD/Kerberos) principal** — those are out of scope under either value, their factor delegated to the directory — or set `require_mfa_scope = "administrators"` (itself reported as a loosening, and it leaves every local Administrator in scope) — see that row below. **mTLS is *not* the third.** A `[api].tls_client_cert_identities` mapping does grant a cert-identity that never meets the MFA gate, but that plane is admitted on exactly **one** route (`GET /service/identity`, `require_service_cert`) and carries no session, so an account "moved to mTLS" can read back its own identity and nothing else — it cannot replay, purge, poll status, or do any work a service account exists for. The `[api].tls_client_cert_identities` row above is the authority on that reach. Directory identities being out of scope also means an AD-only deployment is safe **for its AD users**; its local bootstrap admin and any local service accounts are still in scope | +| `allow_unverified_alert_smtp_tls` | bool | `false` | the **acknowledgment** required to start an enforcing PHI instance whose `[alerts]` SMTP hop does not authenticate the relay — i.e. `[alerts].email_use_tls = false` (cleartext) or `[alerts].email_tls_verify = false` (encrypted but accepts any certificate) ([#323](archive/backlog/BACKLOG-CLOSED.md#323-smtp-tls-is-unverified-on-all-three-send-paths)). Covers BOTH shapes deliberately: cleartext is strictly worse than unauthenticated TLS, so gating only the second would hand an operator a bypass onto the worse posture. Without it `serve` refuses to start (exit 2); with it the start is permitted and named in a WARNING-level `AUDIT:` line. An **acknowledgment switch rather than the clamped `MEFOR_ALLOW_INSECURE_TLS` escape** the connectors use, because this cell is constructed outside the `active_hop_posture` scope where that clamp would be inert. A **loosening** — `security_loosenings()` reports it, so it is never silent | | `memory_encryption_operator_declared` | bool | `false` | **`[BUILT]` ([ADR 0152](adr/0152-in-use-data-protection-for-phi-platform-memory-encryption-attestation-asvs-11-7-1.md) rung 2, ASVS 11.7.1):** the operator's **declaration** that this host provides hardware memory encryption (AMD SEV-SNP / Intel TDX), so PHI is protected in RAM **while it is being processed**. The engine cannot verify it — a local CPU flag is emitted by the OS whose integrity the requirement protects against — so this records **who took responsibility**, the same discipline as `MEFOR_TLS_REVOCATION_ATTESTED`. It is deliberately **not** called "attested": in confidential computing that word means a CPU-signed quote verified against the silicon vendor's root PKI (ADR 0152 rung 3, **not built**). An **exposed** PHI instance without it **warns and starts** — on every environment, at both `enforcement` settings; it refuses only if `require_memory_encryption_declaration` is also set. A **positive platform read-out does not substitute for it** (a read-out must never relax a control). **Loopback and synthetic instances are byte-identical** (never consulted). If the platform read-out positively contradicts this, the contradiction is **warned at start and reported** as `memory_encryption_readout_contradicts_declaration` on `GET /security/posture` — but **never refused** (the read-out is a self-report, not evidence, and has known false negatives: driver not loaded, container without the device node mapped, Azure CVM paravisor). **Setting this does not make the instance ASVS 11.7.1-compliant** — see the read-out note below the table. Env: `MEFOR_SECURITY_MEMORY_ENCRYPTION_OPERATOR_DECLARED` | | `require_memory_encryption_declaration` | bool | `false` | **`[BUILT]` (ADR 0152 rung 2):** turn the row-12 warning above into a **refusal** — an **exposed** PHI instance with no `memory_encryption_operator_declared` then **refuses to start** under `enforcement=enforce` (and still warns under `warn`). **Opt-in by design, and the default is load-bearing:** the property is a **host** property that no operator can satisfy on Windows (the read-out is always `null` there), and "exposed" includes the recommended loopback-behind-proxy topology, so a refusal by default would stop working dev/staging/prod deployments from booting on upgrade over something they cannot change. Same scoping rule as `[security].allowed_client_networks`' companion refusal (ADR 0151): a new refusal fires only on a new opt-in. **One exception exists, and it is recorded:** the `allow_single_factor_admin_when_exposed` refusal three rows above was corrected under BACKLOG #326 and fires with no new opt-in gating it — see that row and the [ADR 0140](adr/0140-two-acknowledged-production-phi-no-loosen-carve-outs-single-factor-admin-at-exposure-keyless-phi-in-production.md) amendment for the reasoning; do not generalise it. Set it in an estate that has standardized on confidential-computing hosts and wants a missing declaration to be fatal. Env: `MEFOR_SECURITY_REQUIRE_MEMORY_ENCRYPTION_DECLARATION` | | `organization_domains` | list[str] | `[]` | **`[BUILT]` (ASVS 3.7.3):** domains that count as **inside** your organization. The console interposes a "you are leaving this site" page, with a cancel, before any navigation to a destination **not** covered here. ASVS asks about destinations outside the application's **control**, and control is *organisational* rather than topological — your own AD FS is a different host, a different origin, and squarely yours — so this is a declared domain list, **not** a same-origin test. Matched on a **label boundary**: `hospital.example` covers `adfs.hospital.example` and **not** `evilhospital.example` (a bare suffix test would admit the lookalike, which is the failure that makes an interstitial worse than none). **Empty is the STRICT position, not the lax one:** with nothing declared, *every* absolute `http(s)` destination is treated as external and gets the page — including your own IdP. Declaring your domains here is the correct fix for that, **not** `external_link_allowlist`. Entries are bare domains: a URL, scheme or `*` wildcard is refused at config load, because all three look right and match nothing. Env: `MEFOR_SECURITY_ORGANIZATION_DOMAINS` | diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 13782a03..7ea4584c 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -572,7 +572,7 @@ on the shipped posture. | **LDAPS** (`[auth].ad_tls_verify`, default true) | verifying directory bind | | **`[logging]` TLS syslog forwarder** (`forward_tls_verify`, default true) | CA-anchored RFC 5425 hop | | **Webhook alert sink** and the **AI-broker endpoint** | verifying https openers | -| **`[alerts]` SMTP sink** and the **per-user security-event notifier** | `email_tls_verify` defaults **true** ([#323](BACKLOG.md)) — one verifying context, two call sites. Deliberately carries **no** `RevocationHopGuard`: it is constructed outside the `active_hop_posture` scope those guards read, so a guard here could not see the instance posture. Its verify-off / cleartext deviations are gated by `[security].allow_unverified_alert_smtp_tls` at the serve gate instead | +| **`[alerts]` SMTP sink** and the **per-user security-event notifier** | `email_tls_verify` defaults **true** ([#323](archive/backlog/BACKLOG-CLOSED.md#323-smtp-tls-is-unverified-on-all-three-send-paths)) — one verifying context, two call sites. Deliberately carries **no** `RevocationHopGuard`: it is constructed outside the `active_hop_posture` scope those guards read, so a guard here could not see the instance posture. Its verify-off / cleartext deviations are gated by `[security].allow_unverified_alert_smtp_tls` at the serve gate instead | For every hop in that table, revocation is exactly what the ASVS row above calls *delegated* — **your PKI's or your egress proxy's job, written into your runbook**. The engine will not make you say so, and diff --git a/docs/DICOM.md b/docs/DICOM.md index 21f0b72f..17ba0a36 100644 --- a/docs/DICOM.md +++ b/docs/DICOM.md @@ -39,7 +39,7 @@ These are **declined or deferred by design** in [ADR 0025](adr/0025-dicom-codec- | **MPPS** (Modality Performed Procedure Step) | Out of scope | | **Query/Retrieve** — C-FIND / C-MOVE / C-GET | Out of scope (Mirth doesn't have these) | | **DICOMweb QIDO-RS / WADO-RS** (query/retrieve over HTTP) | Out of scope; our DICOMweb is **store/send only** | -| **Inbound DICOMweb (STOW-RS) receiver** | Deferred — needs the inbound HTTP listener (ADR 0023, not yet authored; [backlog](BACKLOG.md) #7) | +| **Inbound DICOMweb (STOW-RS) receiver** | Deferred — needs the inbound HTTP listener (ADR 0023, not yet authored; [backlog](archive/backlog/BACKLOG-CLOSED.md#7-inbound-soaprest-listener--web-service-source-v03) #7) | | **Pixel-data transformation / rendering**, `numpy` | Headers + SR only — also a security boundary (no decompression-bomb surface) | --- diff --git a/docs/PHI.md b/docs/PHI.md index 245602ea..a6c0e114 100644 --- a/docs/PHI.md +++ b/docs/PHI.md @@ -971,7 +971,7 @@ with materially different PHI profiles, so they get their own rows; stream 4 is | **8. `alert_instance` table — default on wherever an `[alerts]` notifier exists** | resolvable operator alerts: `connection_stopped`, `queue_buildup`, `lane_stuck`, `message_stall`, `saturation`, `connection_error`, `content_match`, `storage_threshold`, `cert_expiry`, `secret_rotation`, `bootstrap_admin_expiring` (the UNCLAIMED first-run bootstrap admin nearing its auto-disable deadline — ASVS 6.4.5; its payload carries only the ISO deadline plus whole hours remaining, never the password or any secret), `integrity_drift`, `update_available`, `backup_failed`, `rcsi_off_degraded`, `leadership_acquired`, `dr_activated`, `gcm_invocations` (the per-key AES-GCM invocation bound crossing its 2^31 soft warn — ASVS 11.3.4; its payload carries a one-way `key_id` fingerprint plus counters, never key bytes) The three reachable **inverse** signals — `connection_restored`, `leadership_lost`, `dr_released` — are never rows here: `_record_state` routes an inverse through `_AUTO_RESOLVE` to `resolve_alert_instances_for`, never to `upsert_alert_instance`. (A fourth mapped key, `connection_started`, is emitted by no code path today.) | rows: `event_type`, `connection`, `severity`, `status`, `first_seen`, `last_seen`, `count`, `reason`, `acked_by`, `acked_at`, `resolved_at`, `suspended_until`, `escalation_tier` | the store database, **all three backends** | the operator alert list — acknowledge / resolve / suspend. Durable state is recorded **before** any suppression or throttle return, so a muted alert still leaves a record | `GET /alerts/active` under **`monitoring:diagnose`** (**not** a PHI permission) with the same per-channel scope; ack/resolve/suspend/**resume** are POSTs on the same tier, and the separate read-only `GET /alerts/rules` view sits on its own gate | shares the connection-event window; **only RESOLVED instances are DELETEd**, by `resolved_at` — an open or acknowledged condition is never aged out from under an operator | **`reason` is free text** taken from the event's `detail`/`reason`/`label`: `safe_text(reason)[:200]` then cipher-encrypted (AAD `("alert_instance","reason",event_type,connection)` — the de-dup grain, so one AAD covers both the INSERT and the re-fire UPDATE). `content_match` is **PHI-free by contract**: the sink method takes no value parameter, only the connection, an operator label and an optional rule id | | **9. `response` rows with `kind='ack_sent'` — DEFAULT ON** (`[diagnostics].response_sent = true`) | the ACK/NAK the engine returned to an inbound sender, under a sentinel destination `\x1fack:` | rows: `ack_code` (`AA`/`AE`/`AR`/`CA`/`CE`/`CR`), `ack_phase` (`decode`/`parse`/`strict`/`ingest`), `outcome`, `body`, `detail` | the store database | "what did we actually reply, and why" — the operator's answer to a sender disputing an ACK | `GET /messages/{id}/responses` under `messages:read` + `require_phi_read`; the `body` only for a caller who also holds `messages:view_raw`; every read writes a `response.read` audit row | `body`, `detail` and `resp_headers` are set to `NULL` in place by `purge_message_bodies` on the message-body window, on all three backends | **PHI fail-safe:** the ACK **body** is stored **only when the store cipher is active** — on a keyless store it is `NULL` rather than plaintext — and every NAK passes no body at all, so the offending field value is never persisted. The disposition metadata (`ack_code`/`ack_phase`/`outcome`) is non-PHI and always captured; `detail` is `safe_text`-scrubbed, 200-char bounded and encrypted | | **10. `[alerts]` webhook transport** (off by default — `webhook_url` unset) | one HTTPS POST per alert, carrying every non-underscore event key as JSON | JSON | the operator's webhook endpoint (Slack/Teams/PagerDuty/custom) | operator notification | **`https` only** — a plaintext `http://` webhook URL is refused at construction unless the raw `MEFOR_ALLOW_INSECURE_TLS` escape is set (and then a warning is logged); note this path reads the **unclamped** escape, unlike the connectors. Redirects are refused; an optional `webhook_allowed_hosts` egress allowlist gates the host | the endpoint's | **carries the alert's `detail`/`reason` free text** (`safe_exc()`-scrubbed at the emit sites, but **not** re-run through `safe_text` on this path). Internal `_`-prefixed keys (per-rule recipients, rule id, cooldown) are stripped before send, so recipient addresses never cross the wire | -| **11. `[alerts]` SMTP transport — operator alert list** (off unless `email_smtp_host` + `email_from` + ≥1 `email_to`) | one email per alert; default subject `[MessageFoundry] — `, default body every non-underscore event key as `k: v` | plain text (always kept — never HTML-only); optional HTML alternative | the operators' mailboxes | operator notification | `smtp_allowed_hosts` egress allowlist; the SMTP password comes from `MEFOR_ALERTS_EMAIL_PASSWORD` or a `[secrets]` provider, never the config file; per-send timeout `email_timeout` | the mail system's | carries the same `detail`/`reason` free text as the webhook. #138 operator templates are constrained to a **closed non-PHI variable allowlist** validated fail-closed at config load. **Transport posture:** `send_plain_email` builds an explicit **verifying** context (chain + hostname + strict RFC 5280, TLS 1.2 floor) via `tls_policy.build_smtp_tls_context()` and passes it to `starttls()`, anchored to the OS roots, `[alerts].email_tls_ca_file`, or `[tls].internal_ca_file` — the same factory the EMAIL and DIRECT *message destinations* use, so all three SMTP cells now share one policy ([#323](BACKLOG.md), closed 2026-08-02). Before that this call passed **no** context and Python's stdlib default applied (`ssl._create_stdlib_context` **is** `ssl._create_unverified_context` — `CERT_NONE`, `check_hostname = False`), leaving the hop encrypted but unauthenticated. There is still **no hop gradient or attestation on this path** — unlike the connectors, this cell is constructed outside the `active_hop_posture` scope, so its deviations (`email_use_tls = false`, or `email_tls_verify = false`) are gated by a `[security].allow_unverified_alert_smtp_tls` **acknowledgment switch at the serve gate** rather than by the clamped escape: on an enforcing PHI instance `serve` refuses to start without it, and permits + `AUDIT`-logs the start with it. Both deviations are named by `security_loosenings()` and reported by `messagefoundry check`'s `alert-smtp-tls` advisory | +| **11. `[alerts]` SMTP transport — operator alert list** (off unless `email_smtp_host` + `email_from` + ≥1 `email_to`) | one email per alert; default subject `[MessageFoundry] — `, default body every non-underscore event key as `k: v` | plain text (always kept — never HTML-only); optional HTML alternative | the operators' mailboxes | operator notification | `smtp_allowed_hosts` egress allowlist; the SMTP password comes from `MEFOR_ALERTS_EMAIL_PASSWORD` or a `[secrets]` provider, never the config file; per-send timeout `email_timeout` | the mail system's | carries the same `detail`/`reason` free text as the webhook. #138 operator templates are constrained to a **closed non-PHI variable allowlist** validated fail-closed at config load. **Transport posture:** `send_plain_email` builds an explicit **verifying** context (chain + hostname + strict RFC 5280, TLS 1.2 floor) via `tls_policy.build_smtp_tls_context()` and passes it to `starttls()`, anchored to the OS roots, `[alerts].email_tls_ca_file`, or `[tls].internal_ca_file` — the same factory the EMAIL and DIRECT *message destinations* use, so all three SMTP cells now share one policy ([#323](archive/backlog/BACKLOG-CLOSED.md#323-smtp-tls-is-unverified-on-all-three-send-paths), closed 2026-08-02). Before that this call passed **no** context and Python's stdlib default applied (`ssl._create_stdlib_context` **is** `ssl._create_unverified_context` — `CERT_NONE`, `check_hostname = False`), leaving the hop encrypted but unauthenticated. There is still **no hop gradient or attestation on this path** — unlike the connectors, this cell is constructed outside the `active_hop_posture` scope, so its deviations (`email_use_tls = false`, or `email_tls_verify = false`) are gated by a `[security].allow_unverified_alert_smtp_tls` **acknowledgment switch at the serve gate** rather than by the clamped escape: on an enforcing PHI instance `serve` refuses to start without it, and permits + `AUDIT`-logs the start with it. Both deviations are named by `security_loosenings()` and reported by `messagefoundry check`'s `alert-smtp-tls` advisory | | **12. Per-user security-event SMTP notifier** — **posture-mandatory on a PHI instance** | `account_locked`, `login_after_failures`, `password_changed`, `password_reset`, `email_changed`, `roles_changed`, `account_disabled`, `mfa_enabled`, `mfa_disabled`, `admin_action_new_ip` | plain-text email | the **affected user's own** mailbox | ASVS 6.3.5 / 6.3.7 out-of-band notification of security-relevant account changes | shares stream 11's SMTP transport and therefore its verifying context and its `[alerts].email_tls_*` knobs — note this is a **separate call site** (`pipeline/security_notify.py`), plumbed in its own right rather than inheriting by accident. On a PHI instance with auth enabled `serve` **refuses to start (exit 2) under `[security].enforcement = enforce`** when no effective channel exists; the explicit, **audited** opt-out is `[alerts].security_notifications_required = false` | the mail system's | the body carries the account username, a fixed description, optionally the failed-attempt count or the new email on file, and the source IP — **no message data, no secrets**. Dispatch is a bounded background queue; a failed send is logged, never raised (the event is still in `audit_log`) | | **13. `LoggingAlertSink` fallback** (when no `[alerts]` transport is configured) | every alert **this state-less sink implements**, at `WARNING` — `leadership_lost` / `dr_released` at `INFO`, and `connection_restored` is a **deliberate no-op** (a recovery needs no page and there is no instance to auto-resolve), so a lane recovery produces no record on this stream at all. `content_match` exists only on `NotifierAlertSink` and has no fallback-path record | — | folds into stream 1 | so alerts are never silent | inherits stream 1's | inherits stream 1's | includes the `detail`/`reason` free text, and therefore inherits stream 1's filters, ACL, forwarder and retention | diff --git a/docs/POSITIONING.md b/docs/POSITIONING.md index d6896ad7..5e86496a 100644 --- a/docs/POSITIONING.md +++ b/docs/POSITIONING.md @@ -58,7 +58,7 @@ remote production database. The durable, code-first core is the same at every si - **Not a visual / drag-drop transformer.** Code-first authoring *is* the differentiator — a guided editor that drifts toward declarative *logic* authoring is an anti-goal (see - [BACKLOG.md](BACKLOG.md) #26). + [BACKLOG-CLOSED.md](archive/backlog/BACKLOG-CLOSED.md#26-visual--template-driven-channel-authoring--decision-decline-by-design-no-build) #26). - **Not a broker-coupled architecture.** The staged database queue *is* the durability layer; we don't require Kafka/JMS to be reliable. - **Not chasing native per-core benchmark wins.** The honest trade is interpreted-language flexibility + diff --git a/docs/REMOTE-CONSOLE.md b/docs/REMOTE-CONSOLE.md index 17fd92c3..c80969e1 100644 --- a/docs/REMOTE-CONSOLE.md +++ b/docs/REMOTE-CONSOLE.md @@ -184,7 +184,7 @@ Auth is on by default; remote users sign in with local accounts (± TOTP MFA) or loopback-behind-a-terminator topology in §3 **does** trip it, including when the default-on console auto-degrades to JSON-only, and when `serve_web_console = false` disables the console outright: the single-factor surface being protected is the JSON operator API. (This is a correction — - [BACKLOG #326](BACKLOG.md); the arm used to read the console flag, which the §3 auto-degrade clears + [BACKLOG #326](archive/backlog/BACKLOG-CLOSED.md#326-mfa-at-exposure-refusal-reads-serve_ui-after-it-is-flipped-off); the arm used to read the console flag, which the §3 auto-degrade clears first, and would have missed exactly that topology on first deployment.) An **undeclared** proxy — a set `[security].web_console_public_address` with no `tls_terminated_upstream` — is outside the predicate and does **not** refuse: nothing was declared, so exposure would be an inference. It gets diff --git a/docs/SECURITY-LOOSENING.md b/docs/SECURITY-LOOSENING.md index c7a77878..48124c24 100644 --- a/docs/SECURITY-LOOSENING.md +++ b/docs/SECURITY-LOOSENING.md @@ -223,7 +223,7 @@ trail. factor over the network. This ack **downgrades that refusal to a loud, audited warning** (the same warn-and-start `enforcement = warn` takes, but scoped to this one control), so the instance boots single-factor while staying at `enforce`. -- **Scope correction ([BACKLOG #326](BACKLOG.md)):** "a declared reverse proxy" above means exactly +- **Scope correction ([BACKLOG #326](archive/backlog/BACKLOG-CLOSED.md#326-mfa-at-exposure-refusal-reads-serve_ui-after-it-is-flipped-off)):** "a declared reverse proxy" above means exactly `[api].tls_terminated_upstream` — the bind-and-proxy posture, **independent of the browser console**. The shipped predicate additionally required the console to be *served*, which the ADR 0143 auto-degrade had already turned off, so a loopback-behind-a-declared-proxy instance would not have reached this refusal at diff --git a/docs/adr/0024-smart-backend-services-token-provider.md b/docs/adr/0024-smart-backend-services-token-provider.md index 11dc1282..03b7166a 100644 --- a/docs/adr/0024-smart-backend-services-token-provider.md +++ b/docs/adr/0024-smart-backend-services-token-provider.md @@ -32,7 +32,7 @@ (the at-least-once/purity invariant the per-request mint preserves), ADR 0023 (the **inbound** FHIR server facade — a *separate, deferred, not-yet-written* decision; this ADR is client-only and needs nothing from it), [CLAUDE.md](../../CLAUDE.md) §6/§8/§9 (asyncio-non-blocking, HL7/FHIR version-explicitness, never-log-secrets/PHI), - [FEATURE-MAP.md](../FEATURE-MAP.md) §7, [BACKLOG.md](../BACKLOG.md) #35. + [FEATURE-MAP.md](../FEATURE-MAP.md) §7, [BACKLOG-CLOSED.md](../archive/backlog/BACKLOG-CLOSED.md#35-smart-backend-services-token-provider--fhirrest-client-oauth2-p2--adr-0024) #35. ## Context diff --git a/docs/adr/0025-dicom-codec-store-connectors.md b/docs/adr/0025-dicom-codec-store-connectors.md index 407afe7f..cd94ab74 100644 --- a/docs/adr/0025-dicom-codec-store-connectors.md +++ b/docs/adr/0025-dicom-codec-store-connectors.md @@ -626,7 +626,7 @@ the SR→PowerScribe flow the practice runs on Corepoint's DICOM Gear, authored 5. **A code-first SR→HL7 Handler (CHOSEN) vs a declarative DICOM-mapping surface (the Corepoint-GUI model).** A declarative field-mapping surface is precisely Corepoint DICOM Gear's model — and **declined-by-design** for - MessageFoundry ([CLAUDE.md](../../CLAUDE.md) §12, [BACKLOG.md](../BACKLOG.md) #26: visual/template-driven + MessageFoundry ([CLAUDE.md](../../CLAUDE.md) §12, [BACKLOG-CLOSED.md](../archive/backlog/BACKLOG-CLOSED.md#26-visual--template-driven-channel-authoring--decision-decline-by-design-no-build) #26: visual/template-driven authoring is declined; code-first Routers/Handlers *are* the differentiator). The SR→HL7 mapping is a **pure Python Handler** that calls the codec's mapping helpers; the helpers spare boilerplate but the Handler owns the decisions. **Rejected (declarative mapper), CHOSEN (code-first Handler).** diff --git a/docs/adr/0026-off-box-egress-update-check.md b/docs/adr/0026-off-box-egress-update-check.md index 20cbb75c..66ec86ba 100644 --- a/docs/adr/0026-off-box-egress-update-check.md +++ b/docs/adr/0026-off-box-egress-update-check.md @@ -14,7 +14,7 @@ ## Context -#30 asks for an automatic "newer MessageFoundry version exists" signal surfaced in the console and the VS Code IDE, so an operator sees drift without hand-diffing `pyproject.toml` / `requirements.lock` against PyPI ([BACKLOG #30](../BACKLOG.md)). The feature has a real tension at its heart, and the backlog entry already names it as "the main design constraint": **a version check is an outbound network call, which conflicts with the on-premises-by-default, no-egress posture** ([CLAUDE.md](../../CLAUDE.md) §9; [SECURITY.md](../SECURITY.md) — the API binds `127.0.0.1` by default and no PHI leaves the local environment without explicit, reviewed configuration). +#30 asks for an automatic "newer MessageFoundry version exists" signal surfaced in the console and the VS Code IDE, so an operator sees drift without hand-diffing `pyproject.toml` / `requirements.lock` against PyPI ([BACKLOG #30](../archive/backlog/BACKLOG-CLOSED.md#30-automatic-dependency--messagefoundry-version-update-check-surfaced-in-the-console--ide)). The feature has a real tension at its heart, and the backlog entry already names it as "the main design constraint": **a version check is an outbound network call, which conflicts with the on-premises-by-default, no-egress posture** ([CLAUDE.md](../../CLAUDE.md) §9; [SECURITY.md](../SECURITY.md) — the API binds `127.0.0.1` by default and no PHI leaves the local environment without explicit, reviewed configuration). Why on-prem HL7 integration engines distrust "phone-home": @@ -80,7 +80,7 @@ The event is **best-effort and advisory** (the sink contract — never raises, n ### §5 The §G trim — the auto dependency-vulnerability-scan half is NOT built -#30 originally bundled an auto **dependency**-vulnerability/update scan alongside the MEFOR-version check. That half is **dropped** (MULTISESSION-PLAN-3.md §G; [BACKLOG #30](../BACKLOG.md)): CI's **DEP-1** audit + the **hash-locked `requirements.lock`** already cover dependency staleness/vulnerability out-of-band. Building a **second, runtime** dependency-scan mechanism inside the engine would (a) duplicate the CI gate, (b) require its own always-on egress to a vulnerability index — the exact phone-home this ADR is constraining — and (c) tempt an auto-install/lock-mutating path this ADR forbids (§2.4). So the MVP keeps **only** the off-by-default (prefer no-network) **MEFOR-version** advisory. Dependency posture stays where it belongs: the lock + DEP-1. +#30 originally bundled an auto **dependency**-vulnerability/update scan alongside the MEFOR-version check. That half is **dropped** (MULTISESSION-PLAN-3.md §G; [BACKLOG #30](../archive/backlog/BACKLOG-CLOSED.md#30-automatic-dependency--messagefoundry-version-update-check-surfaced-in-the-console--ide)): CI's **DEP-1** audit + the **hash-locked `requirements.lock`** already cover dependency staleness/vulnerability out-of-band. Building a **second, runtime** dependency-scan mechanism inside the engine would (a) duplicate the CI gate, (b) require its own always-on egress to a vulnerability index — the exact phone-home this ADR is constraining — and (c) tempt an auto-install/lock-mutating path this ADR forbids (§2.4). So the MVP keeps **only** the off-by-default (prefer no-network) **MEFOR-version** advisory. Dependency posture stays where it belongs: the lock + DEP-1. ## Consequences diff --git a/docs/adr/0028-base64-binary-carriage-codec.md b/docs/adr/0028-base64-binary-carriage-codec.md index 1187400c..aa0c2748 100644 --- a/docs/adr/0028-base64-binary-carriage-codec.md +++ b/docs/adr/0028-base64-binary-carriage-codec.md @@ -100,7 +100,7 @@ The existing str accessors are unchanged. (`content_type` stays the **format** t - **NOT a new `content_type`.** Carriage is **orthogonal** to format. `content_type` stays the format tag; `mfb64:` is how bytes ride, whatever the format. - **NOT a pipeline routing-logic edit.** `parsing/` stays pure; the **only** seams are the source-boundary `from_bytes` call site and the `RawMessage` accessors. Routing/filtering logic is untouched. - **NOT a "channel"/"route" element.** No graph-bundling object is added. -- **NOT a declarative/visual transform surface, and NOT a `BLOB`/`bytea` store migration** — encode/decode stays code-first via the `RawMessage` API ([CLAUDE.md](../../CLAUDE.md) §12, [BACKLOG #26](../BACKLOG.md)) and all three backends stay TEXT/NVARCHAR(MAX), preserving the encoded-TEXT cipher seam (both also listed under Consequences → Out of scope). +- **NOT a declarative/visual transform surface, and NOT a `BLOB`/`bytea` store migration** — encode/decode stays code-first via the `RawMessage` API ([CLAUDE.md](../../CLAUDE.md) §12, [BACKLOG #26](../archive/backlog/BACKLOG-CLOSED.md#26-visual--template-driven-channel-authoring--decision-decline-by-design-no-build)) and all three backends stay TEXT/NVARCHAR(MAX), preserving the encoded-TEXT cipher seam (both also listed under Consequences → Out of scope). ### 7. HL7 OBX-5 ED embedding (secondary) diff --git a/docs/adr/0029-email-smtp-destination.md b/docs/adr/0029-email-smtp-destination.md index 1cfafcb4..177f7a3d 100644 --- a/docs/adr/0029-email-smtp-destination.md +++ b/docs/adr/0029-email-smtp-destination.md @@ -40,7 +40,7 @@ written when both cells passed **no** SSL context to `starttls()` — which mean authentication: an on-path attacker presenting any certificate read the Handler payload and the SMTP credential. -[BACKLOG #323](../BACKLOG.md) closed that in two steps, and the sentence is true again — but for a +[BACKLOG #323](../archive/backlog/BACKLOG-CLOSED.md#323-smtp-tls-is-unverified-on-all-three-send-paths) closed that in two steps, and the sentence is true again — but for a different reason than it was written for: - **Layers 1–2 (PR #132)** gave `EmailDestination` / `DirectDestination` an explicit verifying context diff --git a/docs/adr/0032-console-desktop-launch.md b/docs/adr/0032-console-desktop-launch.md index dfe491b4..5539fd13 100644 --- a/docs/adr/0032-console-desktop-launch.md +++ b/docs/adr/0032-console-desktop-launch.md @@ -2,8 +2,8 @@ - **Status:** **RETIRED (2026-07-13) — the PySide6 desktop console is removed.** Superseded by the browser web console (`/ui`, `messagefoundry_webconsole`; [ADR 0065](0065-web-ops-dashboard.md), - [BACKLOG #75](../BACKLOG.md)) as the sole operator UI, and by [ADR 0088](0088-apiclient-service-cli-extraction.md) - (which extracted the Qt-free `apiclient/` + the `messagefoundry service` CLI). [BACKLOG #103](../BACKLOG.md) + [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor)) as the sole operator UI, and by [ADR 0088](0088-apiclient-service-cli-extraction.md) + (which extracted the Qt-free `apiclient/` + the `messagefoundry service` CLI). [BACKLOG #103](../archive/backlog/BACKLOG-CLOSED.md#103-retire-the-pyside6-desktop-console-in-favor-of-the-web-console-p3-owner-decision) completed the deferred remainder — `messagefoundry/console/` deleted, the reusable Qt view widgets rehomed to `harness/`, the `[console]` extra renamed to `[harness]` (keyring dropped), and the `[project.gui-scripts]` windowed launcher + `scripts/console/` shortcut tooling removed. Everything @@ -17,7 +17,7 @@ Accepted (2026-06-28) and built, then **retired (2026-07-01) — superseded**, and its packaging assets + CI leg were removed — see the *Amendment (2026-07-01) — Phase B retired* section below (rationale: zero uptake, and the zero-install audience it targeted is now served by the browser ops dashboard, - [BACKLOG #75](../BACKLOG.md)). The earlier *Amendment (2026-06-28) — Phase B* section is retained + [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor)). The earlier *Amendment (2026-06-28) — Phase B* section is retained below **as the historical record of a now-superseded decision** — it no longer describes shipped code. - **Built:** the `[project.gui-scripts]` `messagefoundry-console` entry point ([pyproject.toml](../../pyproject.toml)); the window/taskbar icon @@ -114,7 +114,7 @@ remains an interactive, per-user desktop app. `tests/test_frozen_launch_smoke.py`); the frozen-launch checks (AC-B4/AC-B5) run **only on the Windows installer CI leg** and skip elsewhere, as noted on each. - **Date:** 2026-06-28 -- **Related:** [ADR 0032](0032-console-desktop-launch.md) (Phase A, Accepted) · [BACKLOG #39](../BACKLOG.md) · [CLAUDE.md §2/§10](../../CLAUDE.md) · [release.yml](../../.github/workflows/release.yml) · [NOTICE](../../NOTICE) · MULTISESSION-PLAN-6 Lane L2 +- **Related:** [ADR 0032](0032-console-desktop-launch.md) (Phase A, Accepted) · [BACKLOG #39](../archive/backlog/BACKLOG-CLOSED.md#39-frozen-zero-python-console-installer-phase-b--p3--adr-0032---retired) · [CLAUDE.md §2/§10](../../CLAUDE.md) · [release.yml](../../.github/workflows/release.yml) · [NOTICE](../../NOTICE) · MULTISESSION-PLAN-6 Lane L2 > This is a **Proposed amendment**, not a status change. Phase A of ADR 0032 stays **Accepted/built** and > nothing in it is discarded — the `[project.gui-scripts]` `messagefoundry-console` entry point is exactly @@ -368,7 +368,7 @@ copy of the LGPL library. We satisfy it as follows: CI leg, and AC-linked tests — has been **removed from the tree**. The *Amendment (2026-06-28) — Phase B* section above is kept as the historical record; it no longer describes shipped code. - **Date:** 2026-07-01 -- **Related:** [BACKLOG #39](../BACKLOG.md) (retired) · [BACKLOG #75](../BACKLOG.md) (browser ops +- **Related:** [BACKLOG #39](../archive/backlog/BACKLOG-CLOSED.md#39-frozen-zero-python-console-installer-phase-b--p3--adr-0032---retired) (retired) · [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) (browser ops dashboard — the successor for the zero-install audience) · [CLAUDE.md §2/§10](../../CLAUDE.md) ### Why retire it @@ -380,7 +380,7 @@ and found the channel is a maintained liability with no evidenced user: (v0.2.11–v0.2.14); exactly one `.exe` ever existed (v0.2.14, attached out-of-band, **0 downloads** on a private repo). It never delivered an artifact in-band. - **Its demand gate never fired.** Phase B was reserved for a site with *no Python and no IT* - (§Context above; [BACKLOG #39](../BACKLOG.md) "Why P3"). No such site materialized; current adopters + (§Context above; [BACKLOG #39](../archive/backlog/BACKLOG-CLOSED.md#39-frozen-zero-python-console-installer-phase-b--p3--adr-0032---retired) "Why P3"). No such site materialized; current adopters are pip + IT-covered (IT already runs the elevated NSSM engine install), and the WIN2025 customer-test plan installs the console via `pip`, never the installer. - **Its value gate was never met.** The OV/EV Authenticode cert was never provisioned, so every built @@ -389,7 +389,7 @@ and found the channel is a maintained liability with no evidenced user: - **Ongoing carrying cost.** Freezing on hosted runners drifts (runner-image Inno version, `signtool`, frozen-exe smoke) — two independent breakage modes across four releases — and the job's failures red the release workflow's run-level signal every time, even though the engine + harness ship fine. -- **The audience moved to the web.** [BACKLOG #75](../BACKLOG.md) (scheduled) serves the exact +- **The audience moved to the web.** [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) (scheduled) serves the exact "viewable without a Python/desktop install" audience from the engine's own FastAPI app, so the installer's strategic rationale transfers to the browser dashboard rather than being lost. @@ -410,7 +410,7 @@ and found the channel is a maintained liability with no evidenced user: ### Reversibility The freeze recipe survives in git history (this ADR + the removed files) and can be restored as a -one-off if a genuine no-Python/no-IT site appears before [#75](../BACKLOG.md) covers its needs. +one-off if a genuine no-Python/no-IT site appears before [#75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) covers its needs. --- @@ -421,10 +421,10 @@ one-off if a genuine no-Python/no-IT site appears before [#75](../BACKLOG.md) co longer describes shipped code. This ADR is kept as the historical record of the desktop console's launch model. - **Date:** 2026-07-13 -- **Related:** [BACKLOG #103](../BACKLOG.md) (the retirement) · [ADR 0065](0065-web-ops-dashboard.md) +- **Related:** [BACKLOG #103](../archive/backlog/BACKLOG-CLOSED.md#103-retire-the-pyside6-desktop-console-in-favor-of-the-web-console-p3-owner-decision) (the retirement) · [ADR 0065](0065-web-ops-dashboard.md) (the browser web console — the successor operator UI) · [ADR 0088](0088-apiclient-service-cli-extraction.md) (the reusable-core extraction: Qt-free `apiclient/` + `messagefoundry service` CLI) · - [BACKLOG #75](../BACKLOG.md) · [CLAUDE.md §2/§10](../../CLAUDE.md) + [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) · [CLAUDE.md §2/§10](../../CLAUDE.md) ### Why retire it diff --git a/docs/adr/0047-cloud-kubernetes-ha-deployment-packaging.md b/docs/adr/0047-cloud-kubernetes-ha-deployment-packaging.md index 55733a88..27e11d5e 100644 --- a/docs/adr/0047-cloud-kubernetes-ha-deployment-packaging.md +++ b/docs/adr/0047-cloud-kubernetes-ha-deployment-packaging.md @@ -8,7 +8,7 @@ packages an **operator-assembled** L4 load balancer whose VIP *follows* failover via a primary-only health check — it does **not** build an engine that *manipulates* a VIP. Any future engine-managed VIP is a distinct, deferred item (see *Out of scope* and *To resolve on acceptance*). -- **Related:** [BACKLOG #41](../BACKLOG.md) (this) · the ratifying research +- **Related:** [BACKLOG #41](../archive/backlog/BACKLOG-CLOSED.md#41-cloud--kubernetes-ha-deployment-packaging-container-fast-follow-follow-ons) (this) · the ratifying research [`research/cloud-deployment-research-2026-06.md`](../research/cloud-deployment-research-2026-06.md) · [ADR 0017](0017-consumer-deployment-model.md) (consumer deployment model, container fast-follow, PR #480) · [ADR 0037](0037-multi-process-sharding-l3.md) (L3 process sharding — the CPU-scaling lever, **not** an HA diff --git a/docs/adr/0050-single-project-root-config-anchoring.md b/docs/adr/0050-single-project-root-config-anchoring.md index b7e48c60..73fef7fe 100644 --- a/docs/adr/0050-single-project-root-config-anchoring.md +++ b/docs/adr/0050-single-project-root-config-anchoring.md @@ -6,7 +6,7 @@ "Path-root caveat" §80-92 + the open **Major** engine-work row §161 this resolves) · [ADR 0007](0007-gui-manageable-connections-toml.md) (connections-as-data, resolved relative to `--config`) · [`docs/research/config-ux-review.md`](../research/config-ux-review.md) (the config-UX review — 31 confirmed findings — this ADR implements its **candidate A**, the split-anchor - fix; covers F3/C1/C3/DD1) · [BACKLOG #33](../BACKLOG.md) (config-UX consolidation, the review's parent; + fix; covers F3/C1/C3/DD1) · [BACKLOG #33](../archive/backlog/BACKLOG-CLOSED.md#33-review-the-end-to-end-configuration-method-across-every-surface-config-ux-consolidation) (config-UX consolidation, the review's parent; this is **#33-A**) · [`docs/CONFIGURATION.md`](../CONFIGURATION.md) (the settings catalog) · [SERVICE.md](../SERVICE.md) (NSSM deployment — the failure case) · CLAUDE.md §1 (on-prem/no-egress default; logic stays code-first), §2 (count-and-log), §9 (PHI never logged at INFO+) diff --git a/docs/adr/0065-web-ops-dashboard.md b/docs/adr/0065-web-ops-dashboard.md index a8a43bbf..0801a574 100644 --- a/docs/adr/0065-web-ops-dashboard.md +++ b/docs/adr/0065-web-ops-dashboard.md @@ -1,6 +1,6 @@ # ADR 0065 — Zero-install same-origin browser ops dashboard (read-only, M1) -- **Status:** **Accepted (2026-07-02) — building (M1).** Implements [BACKLOG #75](../BACKLOG.md) "option +- **Status:** **Accepted (2026-07-02) — building (M1).** Implements [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) "option b" (the scheduled zero-install browser ops dashboard). This ADR settles the M1 (read-only) design; the safe-action + CSRF work is M2 and the full desktop-console port ("option c") stays gated per #75. - **Builds on (must not redesign):** the API is the engine's only external boundary and every route @@ -234,14 +234,14 @@ desktop-console retirement ("option c", gated per #75). `docs/security/ASVS-L3-ASSESSMENT.md` (the "no browser frontend" premise + the flipped cells), `docs/SECURITY.md` (the exposed-gate now covers `/ui`), `docs/PHI.md` (write the missing browser-client -section), and supersedes the CORS + localStorage language in [BACKLOG #75](../BACKLOG.md). +section), and supersedes the CORS + localStorage language in [BACKLOG #75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor). --- ## Amendment (2026-07-19) — Historical trend charts + status-colored data-flow graph (BACKLOG #76) - **Status:** **Accepted (2026-07-19).** Additive to the accepted M1/M2 design — no boundary, PHI, or - CSP change. Implements [BACKLOG #76](../BACKLOG.md) "historical-metrics charting + status-colored + CSP change. Implements [BACKLOG #76](../archive/backlog/BACKLOG-CLOSED.md#76-historical-metrics-charting--status-colored-data-flow-graph) "historical-metrics charting + status-colored data-flow graph" (lane `dg-s8a`, Wave 6). **What it adds.** A read-only **Monitoring → Flow & trends** page (`/ui/monitoring`) with two panels: @@ -252,7 +252,7 @@ section), and supersedes the CORS + localStorage language in [BACKLOG #75](../BA `graph --json` CLI and the IDE CONNECTIONS view use). A new **read-only** JSON endpoint `GET /graph/edges` (`monitoring:read`) returns those edges plus each connection node's **live status** (running/stopped/failed/filtered/not_deployed, read from the `RegistryRunner`). The node colour is - **derived from live status**, never operator-assigned (that is [BACKLOG #79](../BACKLOG.md)'s scope). + **derived from live status**, never operator-assigned (that is [BACKLOG #79](../archive/backlog/BACKLOG-CLOSED.md#79-correlation-object-ux--visual-bidirectional-correlation-editor)'s scope). 2. **Historical trend charts.** A new **read-only** JSON endpoint `GET /metrics/history` (`monitoring:read`) returns a bounded **in-memory ring** of point-in-time samples (queue-by-status counts). The ring is @@ -285,7 +285,7 @@ section), and supersedes the CORS + localStorage language in [BACKLOG #75](../BA ## Amendment (2026-07-19) — Per-message 'Waiting for Reply' outbound display state + cosmetic display delay (BACKLOG #136) -- **Status:** **Accepted (2026-07-19).** Implements [BACKLOG #136](../BACKLOG.md) "'Waiting for Reply' +- **Status:** **Accepted (2026-07-19).** Implements [BACKLOG #136](../archive/backlog/BACKLOG-CLOSED.md#136-waiting-for-reply-per-message-connection-state--display-delay) "'Waiting for Reply' per-message connection state + display delay" (lane `dg-s8a`, Wave 6). A light, additive display note. **What it adds (display only — no delivery-path change).** An outbound MLLP connection already blocks on @@ -307,7 +307,7 @@ display state and a cosmetic knob for when to show it. This adds: **#136 × #117 interaction (do NOT break).** "Waiting for Reply" is **inapplicable in no-ack mode**: the marker is stamped **only around an ACK read that actually happens**, so an outbound that skips the ACK -read ([BACKLOG #117](../BACKLOG.md), a different wave) never sets it and the badge never shows there — the +read ([BACKLOG #117](../archive/backlog/BACKLOG-CLOSED.md#117-sender-no-wait-for-ack-fire-and-forward-option), a different wave) never sets it and the badge never shows there — the state is rendered **only on ACK-waiting outbounds**, by construction. - **AC-10** — WHILE an MLLP outbound is awaiting a reply AND at least `waiting_display_delay` has elapsed diff --git a/docs/adr/0068-browser-webauthn-passkeys-offloopback.md b/docs/adr/0068-browser-webauthn-passkeys-offloopback.md index 223f1269..43d484da 100644 --- a/docs/adr/0068-browser-webauthn-passkeys-offloopback.md +++ b/docs/adr/0068-browser-webauthn-passkeys-offloopback.md @@ -7,7 +7,7 @@ is its design amendment**, superseding the sketch and retiring the `MULTISESSION-PLAN-v0.2.md:437` "WP-14b design amendment authored+Accepted" gate) · [ADR 0065](0065-web-ops-dashboard.md) (web console; its AC-2 cookie boundary and AC-6 off-loopback - refusal are restated and extended here) · [BACKLOG](../BACKLOG.md) #11 / #75 · + refusal are restated and extended here) · [BACKLOG](../BACKLOG.md) #11 / [#75](../archive/backlog/BACKLOG-CLOSED.md#75-browser--web-operator-monitor) · ASVS-L3-ASSESSMENT §2b (both "Deferred (off-loopback / L5)" residuals) · [docs/SECURITY.md](../SECURITY.md) diff --git a/docs/adr/0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md b/docs/adr/0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md index 36819eb2..d829e774 100644 --- a/docs/adr/0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md +++ b/docs/adr/0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md @@ -6,7 +6,7 @@ - **Deciders:** owner (explicit toolkit choice: "No-Qt ctypes spine") + a 25-agent research→design→judge→verify workflow (three competing designs scored by a governance / correctness / security judge panel; the no-Qt design won 251–242, the count-showing signed-in design was eliminated). -- **Related:** BACKLOG [#239](../BACKLOG.md); [#103](../BACKLOG.md) (retired the PySide6 desktop console — +- **Related:** BACKLOG [#239](../BACKLOG.md); [#103](../archive/backlog/BACKLOG-CLOSED.md#103-retire-the-pyside6-desktop-console-in-favor-of-the-web-console-p3-owner-decision) (retired the PySide6 desktop console — and named "a tiny standalone tray/service-manager" as the sanctioned home for out-of-band service control); [ADR 0032](0032-console-desktop-launch.md) (retired); [ADR 0065](0065-web-ops-dashboard.md) (the web console is the sole operator UI); [ADR 0088](0088-apiclient-service-cli-extraction.md) (Qt-free/FastAPI-free diff --git a/docs/adr/0121-test-bench-saved-regression-collections-phi-at-rest-posture-hl7-aware-compare.md b/docs/adr/0121-test-bench-saved-regression-collections-phi-at-rest-posture-hl7-aware-compare.md index 5b9e544f..eac5eb68 100644 --- a/docs/adr/0121-test-bench-saved-regression-collections-phi-at-rest-posture-hl7-aware-compare.md +++ b/docs/adr/0121-test-bench-saved-regression-collections-phi-at-rest-posture-hl7-aware-compare.md @@ -3,7 +3,7 @@ - **Status:** Accepted (2026-07-17) — the DEMAND-GATE-BACKLOG session builds it. IDE-only, no engine change; phased (one coherent commit per layer), pushes/PR owner-approved. - **Date:** 2026-07-17 -- **Related:** BACKLOG [#168](../BACKLOG.md) (Test Bench saved regression collections); [ADR +- **Related:** BACKLOG [#168](../archive/backlog/BACKLOG-CLOSED.md#168-test-bench-saved-regression-collections) (Test Bench saved regression collections); [ADR 0030](0030-anonymization-test-harness-tee.md) (de-identification — the framework authors use to build PHI-free cases); [ADR 0072](0072-traced-dryrun-mode.md) / the Test Bench before/after diff (`hl7diff.ts`, reused here for the compare); CLAUDE.md §9 (PHI rules — this ADR adds a **new diff --git a/docs/adr/0122-wsdl-import-pure-soap-type-tree-validate-against-wsdl-no-zeep.md b/docs/adr/0122-wsdl-import-pure-soap-type-tree-validate-against-wsdl-no-zeep.md index 22fbfced..7117edec 100644 --- a/docs/adr/0122-wsdl-import-pure-soap-type-tree-validate-against-wsdl-no-zeep.md +++ b/docs/adr/0122-wsdl-import-pure-soap-type-tree-validate-against-wsdl-no-zeep.md @@ -12,8 +12,8 @@ **no dependency** (no zeep, no suds), and closing the **distinct `wsdl:import` / `xsd:import` network resolution path** the existing `xmlschema` no-network config does *not* cover. - **Related:** [ADR 0015](0015-ws-soap-outbound-mtls-wssecurity.md) (the WS-* / mTLS SOAP outbound whose - envelopes this validates but does not replace), BACKLOG [#31](../BACKLOG.md) (the pure XML/SOAP codec this - extends — `parsing/xml/`), BACKLOG [#69](../BACKLOG.md), BACKLOG [#70](../BACKLOG.md) + envelopes this validates but does not replace), BACKLOG [#31](../archive/backlog/BACKLOG-CLOSED.md#31-safe-xml-rawmessage-accessor--structured-xml-support-xml--soap--cda-p2) (the pure XML/SOAP codec this + extends — `parsing/xml/`), BACKLOG [#69](../archive/backlog/BACKLOG-CLOSED.md#69-wsdl-import--soap-type-tree--validate-against-wsdl), BACKLOG [#70](../archive/backlog/BACKLOG-CLOSED.md#70-synchronous-in-transform-web-service-call-wscall) (synchronous in-transform WSCall — **declined-by-design**, so a WSDL is a *contract* artifact here, never a live-call generator), [CLAUDE.md](../../CLAUDE.md) §4 (the pure `parsing/` carve-out a client may import), §8 (untrusted XML is attacker-influenceable), §9 (PHI-safe failure reporting). diff --git a/docs/adr/0129-process-in-place-file-disposition-and-cross-backend-processed-file-dedup-ledger.md b/docs/adr/0129-process-in-place-file-disposition-and-cross-backend-processed-file-dedup-ledger.md index e1f87598..31a801a4 100644 --- a/docs/adr/0129-process-in-place-file-disposition-and-cross-backend-processed-file-dedup-ledger.md +++ b/docs/adr/0129-process-in-place-file-disposition-and-cross-backend-processed-file-dedup-ledger.md @@ -9,7 +9,7 @@ **cross-backend `processed_files` store table** that records a **HASHED per-file key** (never a cleartext filename), **after** the file's messages emit successfully, with the **file** (not each split message) as the dedup unit, bounded by an age+count prune. -- **Backlog:** [BACKLOG #142](../BACKLOG.md). Sibling of [BACKLOG #114](../BACKLOG.md) / the +- **Backlog:** [BACKLOG #142](../archive/backlog/BACKLOG-CLOSED.md#142-leave-source-file---process-in-place-fileftp-source-disposition). Sibling of [BACKLOG #114](../BACKLOG.md) / the [ADR 0031](0031-startup-connection-fault-isolation.md) amendment (opt-in startup directory validation), built in the same lane (S3a) and composing with it (a `leave` source validates a read-only share read-only). diff --git a/docs/adr/0132-per-endpoint-alternate-windows-credential-for-file-unc-shares-win32-ctypes-no-pywin32-no-impersonation-privilege.md b/docs/adr/0132-per-endpoint-alternate-windows-credential-for-file-unc-shares-win32-ctypes-no-pywin32-no-impersonation-privilege.md index 4bd28b47..56372eb2 100644 --- a/docs/adr/0132-per-endpoint-alternate-windows-credential-for-file-unc-shares-win32-ctypes-no-pywin32-no-impersonation-privilege.md +++ b/docs/adr/0132-per-endpoint-alternate-windows-credential-for-file-unc-shares-win32-ctypes-no-pywin32-no-impersonation-privilege.md @@ -9,7 +9,7 @@ — established with **win32 ctypes** `LogonUser` + per-thread `ImpersonateLoggedOnUser` (**no pywin32, no privilege**), with the connector's blocking filesystem I/O run on a **dedicated impersonated thread**; **win32-only** (a non-Windows host fails loud, never a silent no-op). -- **Backlog:** [BACKLOG #111](../BACKLOG.md). Corepoint-parity gap: the File connector today reads/writes +- **Backlog:** [BACKLOG #111](../archive/backlog/BACKLOG-CLOSED.md#111-file-endpoint-alternate-windows--network-share-credentials). Corepoint-parity gap: the File connector today reads/writes local/UNC paths **only** under the engine service account's ambient token, and [`transports/remotefile.py`](../../messagefoundry/transports/remotefile.py)'s username/password auth covers **FTP/FTPS/SFTP**, not SMB/UNC Windows-share credentials. diff --git a/docs/adr/0159-cancellation-safe-pooled-connection-release-mid-txn-discard-at-the-acquire-chokepoint.md b/docs/adr/0159-cancellation-safe-pooled-connection-release-mid-txn-discard-at-the-acquire-chokepoint.md index f2857cad..442d788a 100644 --- a/docs/adr/0159-cancellation-safe-pooled-connection-release-mid-txn-discard-at-the-acquire-chokepoint.md +++ b/docs/adr/0159-cancellation-safe-pooled-connection-release-mid-txn-discard-at-the-acquire-chokepoint.md @@ -5,7 +5,7 @@ - **Status:** Accepted (2026-08-02) - **Date:** 2026-08-02 -- **Related:** [BACKLOG #348](../BACKLOG.md) · [ADR 0066](0066-pooled-stage-claimers.md) §9 (the `SET LOCK_TIMEOUT 0` never-block claim, whose 1222→EMPTY translation is what made this silent) · [ADR 0114](0114-phase-4-claim-path-call-complexity-reduction-driver-interface-redesign-ingress-routed-reset-fold.md) §2 (the shielded finally-guard — **explicitly not** a rollback guard; see §3 below) · [ADR 0001](0001-staged-pipeline-architecture.md) (the staged queue whose at-least-once contract bounds the damage) +- **Related:** [BACKLOG #348](../archive/backlog/BACKLOG-CLOSED.md#348-sql-server-a-cancelled-store-call-returns-a-pooled-connection-mid-transaction-holding-x-locks) · [ADR 0066](0066-pooled-stage-claimers.md) §9 (the `SET LOCK_TIMEOUT 0` never-block claim, whose 1222→EMPTY translation is what made this silent) · [ADR 0114](0114-phase-4-claim-path-call-complexity-reduction-driver-interface-redesign-ingress-routed-reset-fold.md) §2 (the shielded finally-guard — **explicitly not** a rollback guard; see §3 below) · [ADR 0001](0001-staged-pipeline-architecture.md) (the staged queue whose at-least-once contract bounds the damage) --- diff --git a/docs/adr/README.md b/docs/adr/README.md index 6f0b4b5d..000fad53 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -144,7 +144,7 @@ what is withheld and what you can request. | [0110](0110-ide-engine-link-doctor-the-status-bar-tells-the-truth-about-the-promote-target.md) | **IDE engine-link doctor — the status bar tells the truth about the promote target** (BACKLOG #232). The `MEFOR: ` item painted a green check whenever *anything* answered `GET /health`: `classifyProbe` folded **every** HTTP status — **including a 401** — into `"reachable"`, and the probe **discarded the body**. `AuthSettings.enabled` defaults True, so a plain `serve` = engine up + IDE holding **no session** + every authenticated call 401ing + **a confident green check** (a live probe of the owner's engine returned `{"status":"ok","version":null}` — `/health` discloses `version` only to an authenticated caller, WP-L3-07). The click's menu diagnosed nothing, and its `Open engine URL in browser` opened the **bare** engine URL — a live **404** (there is no `/` route). **Decision:** green means *"the IDE can USE this engine"*, not *"a socket answered"* — a closed link-state union (`unreachable{code}`/`foreign`/`signedOut`/`unverified`/`blocked{reason}`/`drifted`/`ok`) in the vscode-free model; the tokenless poll reads the `/health` body (`version: null` ⇒ **signedOut**; a version present ⇒ **`unverified`, NEVER green** — `optional_identity` applies no RBAC and no must-change gate, and `/auth/me` is must-change-**exempt**, so neither endpoint can prove a session is usable); a green check is **EARNED** only by a **user-initiated** deep probe of a non-exempt protected route (`GET /config/provenance` — `monitoring:read`, no step-up, yields `drift` free) and it **DECAYS**. **The periodic poll MUST stay TOKENLESS** — `identity_for_token(..., activity=True)` refreshes the session idle clock, so a bearer on a 15s timer would make the engine's 30-min idle timeout unreachable forever (CWE-613). Surface is **native chrome, NOT a webview**: a `MarkdownString` hover (the primary diagnosis — renders at the item, needs no command dispatch), a state-gated QuickPick (`title:`, never a `placeHolder:`), a `$(sync~spin)` flip at the click site, and the extension's **first engine `LogOutputChannel`** (URLs/status/errno/duration/verdict — **never a body, never a token**). **Boundary made executable:** the IDE renders and repairs the **LINK**, never the **WORKLOAD** — container poverty + the model emitting **command ids, never data** + **two frozen CI allowlists** (an `EngineLink` field list with no connection/message/queue/count/rate field, and a probe-endpoint list of only `/health`, `/ai/policy`, `/config/provenance`; `/messages`/`/connections`/`/stats` break the build). Exposes `engineSignIn`/`engineSignOut` (already-written flows that were unreachable; `signIn()` inherits ADR 0035's pre-prompt `assertTargetAllowed()` refusal). **Declined:** a webview panel; a bearer on the poll; a "Reload config" button (`require_step_up` + 300s window while `withAuth` never retries a 403); inline change-password/MFA (deep-link to `/ui/login` — the Console owns credentials); "Start local engine" (a terminal cwd'd at a worktree forks a brand-new store + bootstrap admin). Extends [ADR 0100](0100-ide-native-surface-polish-and-open-to-messagefoundry-startup-experience-backlog-221.md) (supersedes its "opens engine settings" record) / [ADR 0035](0035-ide-extension-workspace-trust-and-scope.md); bounded by [ADR 0065](0065-web-ops-dashboard.md); #26-clean | Accepted (2026-07-14) — owner-directed; built + verified against the live engine (IDE v0.0.28) | | [0111](0111-not-deployed-connections.md) | **Connection present but not deployed** (BACKLOG #233) — a first-class `deployed: bool = True` on **both** connection models so a config can carry a real, reviewed, dark connection (retired partner, superseded duplicate send, a relay pulled from prod) that is **never wired, never started, never queued to, and whose `env()` is never resolved**. Removes DEGRADED-on-**every**-boot, which is indistinguishable from a real regression — a permanent alarm is a disabled alarm. **Key finding:** `auto_start=False` already dodges `resolve_env_settings` on the *cold serve* path (both boot gates return before `_source_config`/`_dest_config`), **but `_build_check_connectors` (`wiring_runner.py:5181`/`:5186`) loops EVERY inbound and EVERY outbound with NO gate** — so `messagefoundry check` (the **required** commit gate), every reload, every promote and every `connection upsert` still explode, and one unresolvable connection blocks edits to *every other* connection in the file. **Honoring the flag there IS the feature.** `deployed=False` **WINS over `auto_start`** (deploying is a *config* change, not a runtime action → `start`/`restart` 409). **Three-way distinction that must never be conflated:** **SIMULATED** (#15 — built, receives rows, suppresses egress, finalizes `PROCESSED`) vs **PARKED** ([ADR 0048](0048-third-tier-disaster-recovery-standby.md)/[ADR 0095](0095-connection-lifecycle-scheduler-and-credential-fault-stop.md) — rows **RETAINED**, queued, retried) vs **NOT DEPLOYED** (no row is ever created). Enforced at the **`transform_one` Send-materialization seam** (`pipeline/dryrun.py:403-416`) — structurally mirroring [ADR 0084](0084-accepts-router-seam.md)'s `accepts=` seam for the router half — which covers the split, [ADR 0057](0057-inline-step-a-fast-path.md) inline and [ADR 0071](0071-cut-executor-round-trips-b5.md) fused paths plus dry-run/`check`/Test Bench in **one** edit; plus a **separate** 409 guard on the [ADR 0090](0090-resend-a-stored-message-to-an-alternate-outbound-connection.md) resend/edit-resend path, which inserts an outbound row directly and bypasses `transform_one` entirely. **Count-and-log (CLAUDE.md §12) preserved** by a per-destination `message_events` row **added to `_AUDIT_FLOOR_EVENTS`** (else the #63 verbosity gate evaporates it at `errors`/`off`, re-creating the accept-and-drop) + a **7th `MessageStatus.NOT_DEPLOYED`** used only when **every** selected destination was declined — because the finalizer decides `FILTERED` **by absence**, merely dropping the target would silently report "the handler filtered this" with **zero code changes and zero test failures** (the trap). **At-least-once ([ADR 0001](0001-staged-pipeline-architecture.md)) preserved:** the decline is keyed on the **REGISTRY FLAG** (a property of the graph), never on live runner state, so a re-run re-derives an identical delivery set. Connection **stays in `Registry.outbound`** (the orphan sweep keys on the registry — removing it would dead-letter already-queued rows). No DDL, no migration, no schema-hash change. **Not built here:** the `ide/` form controls (three TS field enumerations — deferred to avoid colliding with in-flight [ADR 0106](0106-steps-view-add-dropdown-vocabulary-expansion-adr-0076-phase-b.md) work; the flag works via hand-edited TOML + code-first without them) and **BACKLOG #234** (`_SCALAR_FIELDS` is a 12-key whitelist with no passthrough → a GUI save silently strips `priority`/`schedule`/`shard`/`metadata`/`batch`/`stall`/`dead_letter_days`; a **pre-existing** data-loss bug, filed only). Extends [ADR 0007](0007-gui-manageable-connections-toml.md) | Accepted (2026-07-14) — owner-directed; built (#233) | | [0112](0112-ide-engine-lifecycle-from-the-status-bar-pill-guarded-start-stop-restart.md) | **IDE engine lifecycle from the status-bar pill — guarded start / stop / restart** — the `MEFOR: ` pill may now **run** the engine, not just hand over the command. **Supersedes [ADR 0110](0110-ide-engine-link-doctor-the-status-bar-tells-the-truth-about-the-promote-target.md) §5's "Start local engine" decline** (everything else in 0110 stands — the link-state model, the tokenless poll, the earned/decaying green, and the **LINK-not-WORKLOAD boundary §4**, left byte-for-byte intact). §5 declined Start because a `createTerminal` cwd'd at a worktree would fork a **brand-new empty DB + bootstrap admin**; the copy-only v1 also **did not fix the actual failure** — the clipboard's bare `python -m messagefoundry serve …`, pasted into a fresh terminal, resolves `python` to the PATH shim (on Windows the Store shim), which lacks the deps → the reported **`ModuleNotFoundError: No module named 'pydantic'`**. **Decision:** Start runs the **exact blessed command** (`serve --config `, **no `--db`/`--env`/`--port`** — the service TOML stays the sole authority; ADR 0110 rejected `engineEnv` for this reason) but with `python` = `pythonPath()` (the auto-detected workspace `.venv`), launched `createTerminal({shellPath,shellArgs})` = argv, no shell re-parse. The §5 fork is **neutralised, not reintroduced**: Start is gated `canControl` = **loopback** (M-29) + **trusted** (SEC-004/CWE-426) + a workspace, and **never silently creates a store** — a run dir with no service TOML and no `*.db` gets a modal "creates a NEW database + bootstrap admin" confirm (`runDirHasEngine`). **Stop/Restart act ONLY on a terminal this IDE started** (`exitStatus === undefined`), never a port-kill — parallel worktrees can share a port. **"Set up environment"** bootstraps `.venv` + install for the fresh-clone (missing-deps) case. **The ADR 0110 §4 boundary is untouched:** lifecycle actions are **command ids** from the pure `planActions` (new optional `EngineControlContext`, default no-control → every existing call site unchanged); **no `EngineLink` field, no probe endpoint, no setting** added — the two frozen CI allowlists and the settings-scope invariant stay green. Adversarial self-review caught + fixed two shell defects (a post-`dispose` re-probe touching a disposed item; a `startEngine` reentrancy window double-launching). Node-side suite green (348, +20 for this change); tsc + esbuild clean. Supersedes [ADR 0110](0110-ide-engine-link-doctor-the-status-bar-tells-the-truth-about-the-promote-target.md) §5; extends [ADR 0035](0035-ide-extension-workspace-trust-and-scope.md); #26-clean | Accepted (2026-07-15) — owner-directed; built (IDE extension v0.0.29) — Amended 2026-07-16: store-less pill leads with a guided setup page (BACKLOG #238; ratified, built (IDE extension v0.0.32), Plan-12 `ide-238-setup`) | -| [0113](0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md) | **Windows tray service-manager — stdlib ctypes, tokenless** (BACKLOG #239) — a Windows notification-area app for the NSSM engine service: shows engine status at a glance, opens the console (`/ui`) / VS Code / service log, and start/stop/restarts the service via per-action UAC elevation. **No PySide6 — no Qt** (owner-chosen over a `QSystemTrayIcon` design), so §10's "no new PySide6 operator surfaces" is **untouched** — this is the "tiny standalone tray/service-manager" [#103](../BACKLOG.md) sanctioned, an ADR **ratification** not an amendment. **Not a second console:** tokenless `GET /health` + local SCM state **only** — no token ever, no message/queue/count field (a frozen snapshot-field allowlist + a `/health`-stays-tokenless test enforce it, mirroring the IDE `ENGINE_LINK_FIELDS`/`POLL_PLAN` doctrine). Built on stdlib `ctypes` (`Shell_NotifyIcon` + owned message pump) + `apiclient` (**zero** new locked deps). **Elevation writes no new privileged code:** reuse the shipped, injection-hardened `messagefoundry.service.control_service` (System32 `cmd`/`net`, single UAC prompt even for restart), **patched** `ShellExecuteW`→`ShellExecuteExW`+`GetExitCodeProcess` so a UAC-cancel (`ERROR_CANCELLED` 1223) is distinguishable — never elevating the user-writable venv interpreter. Nine-state machine (incl. `WEDGED`/`FOREIGN`/`RUNNING_UNMANAGED`/`UNKNOWN`) over `QueryServiceStatusEx` (`dwCheckPoint`/`dwWaitHint`) with monotonic grace windows; local-single-box scope (**remote** → monitor-only; *amended 2026-07-22* — a **local https** engine is fully managed and its cert verified against the OS trust store, no `verify=False` path; scheme ≠ locality). Ratifies the `messagefoundry.service`/`service_status` client-import carve-out. Extends [ADR 0088](0088-apiclient-service-cli-extraction.md); mirrors [ADR 0110](0110-ide-engine-link-doctor-the-status-bar-tells-the-truth-about-the-promote-target.md)/[ADR 0112](0112-ide-engine-lifecycle-from-the-status-bar-pill-guarded-start-stop-restart.md); bounded by [ADR 0065](0065-web-ops-dashboard.md); #26-clean | Accepted (2026-07-16) — owner chose the no-Qt ctypes spine + accepted the §10 clarification; design + plan complete (25-agent workflow, adversarially verified); build phased, pushes/PR owner-approved | +| [0113](0113-windows-tray-service-manager-stdlib-ctypes-tokenless.md) | **Windows tray service-manager — stdlib ctypes, tokenless** (BACKLOG #239) — a Windows notification-area app for the NSSM engine service: shows engine status at a glance, opens the console (`/ui`) / VS Code / service log, and start/stop/restarts the service via per-action UAC elevation. **No PySide6 — no Qt** (owner-chosen over a `QSystemTrayIcon` design), so §10's "no new PySide6 operator surfaces" is **untouched** — this is the "tiny standalone tray/service-manager" [#103](../archive/backlog/BACKLOG-CLOSED.md#103-retire-the-pyside6-desktop-console-in-favor-of-the-web-console-p3-owner-decision) sanctioned, an ADR **ratification** not an amendment. **Not a second console:** tokenless `GET /health` + local SCM state **only** — no token ever, no message/queue/count field (a frozen snapshot-field allowlist + a `/health`-stays-tokenless test enforce it, mirroring the IDE `ENGINE_LINK_FIELDS`/`POLL_PLAN` doctrine). Built on stdlib `ctypes` (`Shell_NotifyIcon` + owned message pump) + `apiclient` (**zero** new locked deps). **Elevation writes no new privileged code:** reuse the shipped, injection-hardened `messagefoundry.service.control_service` (System32 `cmd`/`net`, single UAC prompt even for restart), **patched** `ShellExecuteW`→`ShellExecuteExW`+`GetExitCodeProcess` so a UAC-cancel (`ERROR_CANCELLED` 1223) is distinguishable — never elevating the user-writable venv interpreter. Nine-state machine (incl. `WEDGED`/`FOREIGN`/`RUNNING_UNMANAGED`/`UNKNOWN`) over `QueryServiceStatusEx` (`dwCheckPoint`/`dwWaitHint`) with monotonic grace windows; local-single-box scope (**remote** → monitor-only; *amended 2026-07-22* — a **local https** engine is fully managed and its cert verified against the OS trust store, no `verify=False` path; scheme ≠ locality). Ratifies the `messagefoundry.service`/`service_status` client-import carve-out. Extends [ADR 0088](0088-apiclient-service-cli-extraction.md); mirrors [ADR 0110](0110-ide-engine-link-doctor-the-status-bar-tells-the-truth-about-the-promote-target.md)/[ADR 0112](0112-ide-engine-lifecycle-from-the-status-bar-pill-guarded-start-stop-restart.md); bounded by [ADR 0065](0065-web-ops-dashboard.md); #26-clean | Accepted (2026-07-16) — owner chose the no-Qt ctypes spine + accepted the §10 clarification; design + plan complete (25-agent workflow, adversarially verified); build phased, pushes/PR owner-approved | | [0114](0114-phase-4-claim-path-call-complexity-reduction-driver-interface-redesign-ingress-routed-reset-fold.md) | **Phase-4 claim-path call-complexity reduction — driver-interface redesign + the INGRESS/ROUTED reset fold** — the frozen D1 verdict locates ~9.7 ms/call of the pooled `claim_fifo_heads` cost (18.0-18.2 ms at the N=4 240-offered pin) **inside the ODBC/TDS driver, per-call, ~2/3 fixed (6.5-7.0 ms), call-complexity-shaped** (the same driver runs a trivial 7-statement batch in 0.335 ms), so the remedy axis is what the call *carries*: three sub-levers, **one default-OFF SqlServerStore-only flag each** (`fifo_claim_fold_reset` / `fifo_claim_proc` / `fifo_claim_prepared`; PG/SQLite provably untouched). **C (fold, measured 1.87-1.91 ms/call, licensed ceiling +8.0% ingress+routed):** the finally-guard's `SET LOCK_TIMEOUT -1` + write-less commit#2 fold into the batch's confirmed-clean success path at INGRESS/ROUTED only (H2-noop code-confirmed + runtime-guarded); the shielded B1/M-6 guard is retained **verbatim on every non-clean exit** (1222, kept≠claimed, cancellation, any error). **A (proc-ification):** TWO lane-family versioned procs (`mefor_claim_fifo_heads_cid_v1`/`_dst_v1` — the lane column is a code literal), fixed-arity 9-param CALL with one JSON lanes parameter, bodies = the batch verbatim (no TRY/CATCH, no txn statements, no reset outside the conditional `@fold_reset` tail — `SET LOCK_TIMEOUT`'s session persistence past proc exit is load-bearing at outbound), guarded DDL that can never break a flag-OFF open, and a startup gate hashing `OBJECT_DEFINITION` + compat ≥ 130 that degrades loudly to the batch. **B (stable text + retained prepared cursor):** the non-DDL fallback, fail-closed-coupled to the fold, on store-owned dedicated connections reconciled with EF-6/STORE-3 — structural feasibility itself is a gate question. FIFO-always semantics preserved exactly; commit-amortization stays D2-scoped-out; **no throughput projections** — pre-registered ms/call bench gates (fixed 240-pin, replicate pairs, invariant battery, total accept/kill rules) are the only forward-looking numbers; the composed 6.35-6.64 ms/call removal (claim 18.8-19.0 → 12.2-12.65) is a target evaluated only at the post-build re-measure; the certification run remains the arbiter | Proposed (2026-07-16) — owner D0-accept + D2-GO (TO-ENGINE-033); design of record, build ships later, all flags default OFF, each flipped only after its own pre-registered bench gate | | [0115](0115-asvs-l3-drive-to-pass-secure-by-default-flips-and-residual-closure.md) | **ASVS L3 drive-to-Pass — secure-by-default flips and residual closure** (BACKLOG #242–#246). The 2026-07-16 ASVS re-score left **50/51 Partials + 2 Fails**; almost every Partial is a *shipped* control scored Partial because it is opt-in / off-by-default / delegated. Owner chose to **drive-to-Pass** rather than leave them all as accepted residuals — but a naïve "flip every default on" would break a valid deployment (a dev box with no SMTP collector, a partner with no JWS verifier, a single-operator loopback install). **Decision:** a **secure-by-default-where-safe, runbook-instructed-otherwise** posture applied per control — (1) flip the global default ON only where it is already gated on the PHI posture so a synthetic/CI box stays byte-identical (bounded retention 14.2.4/14.2.7, egress deny-by-default 13.2.4/13.2.5, cleartext-egress refusal 12.2.1); (2) **instruct** the control in `OFF-LOOPBACK-DEPLOYMENT.md` (+ a fail-closed prod-PHI serve gate where refusing to start is defensible) where a global flip would break a valid install (approvals 2.3.5, JWS signing 4.1.5, off-box forwarding 16.4.3, WebAuthn 6.3.3/6.5.7/6.7.2) — so the *documented deployment* earns Pass while loopback/no-collector/partner-less defaults are unchanged; (3) **build the small last-mile controls** (AEAD context-binding 11.3.3, AES-GCM invocation counter 11.3.4, time-sync enforce 16.2.2, log-all-authz 16.3.2, keyed audit chain default 16.4.2, magic-byte validation 5.2.2, extended pacing 2.4.2, action-bound step-up 7.5.1/7.5.2); (4) **refresh the drifted inventories** (11.1.2, 13.1.1, 13.1.4); (5) **formally accept** the genuinely delegated residuals (WP #246 — proxy-TLS 12.1.x/12.3.x the engine cannot inspect because it terminates no browser TLS, org-delegated backend creds 13.2.1/13.2.2, SMART AS enforcement 10.4.16, AV 5.4.3) into the register. **Out of scope (stay signed residuals):** the runtime sandbox (15.2.5), HSM key custody (13.3.1/13.3.3), the ECH/in-use-memory platform gaps (12.1.5/11.7.2), and the tolerant-HL7 accepted deviations (2.2.1/2.2.3). Each flip/build amends its owning feature ADR (0018 signing, 0080 forwarding, 0068 WebAuthn, 0014 approvals/notify, 0019 AAD/nonce, 0004 magic-byte, 0077 step-up, 0105 served-filename); no secure posture changes without its ADR record updated in the same work. Plan + per-cell mapping: `ASVS-REMEDIATION-2026-07.md`. Drives `ASVS-L3-ASSESSMENT-2026-07-16.md`; residuals owned in `ASVS-L3-RISK-ACCEPTANCE-REGISTER.md`; #26-clean | Accepted (2026-07-16) — owner-directed scope decision (drive-to-Pass); builds phased across BACKLOG #242–#246, pushes/PR owner-approved | | [0118](0118-secure-by-default-security-configuration-section.md) | **Secure-by-default `[security]` configuration section — plain-language, replace scattered keys** — a dedicated `[security]` TOML section as the *canonical, sole* home for the high-value security **posture switches** (network access, at-rest encryption, sign-in/MFA/session, egress deny-by-default, retention, audit) plus the master PHI-posture lever (`handles_real_patient_data` / `production_instance`, was `[ai].data_class`/`.production`), each **named in plain language and defaulting to the secure position**. Fixes config-UX defect **DD2** (the `[security]` heading is a future-only placeholder). Low-level *plumbing* (TLS cert paths, egress allow-list *contents*, DB identity, password policy, rate limits) **stays in its functional section** — CISA "minimize settings". Because **nothing is deployed**, the scattered legacy keys are **replaced outright** (no deprecation/alias); every serve gate + the `checks.py` mirror re-points at `[security]` with **no shipped refusal loosened** (No-loosen rule, [ADR 0092](0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md) §5). Loosening any protection is **deliberate + plain-language-warned** (CISA loosening-guide → new `docs/SECURITY-LOOSENING.md`) and still refuses on production-PHI (ADR 0092 clamp intact). **Editing is IDE-only** (VS Code extension GUI, [ADR 0007](0007-gui-manageable-connections-toml.md) precedent); the **web console stays read-only** (`GET /security/posture`, no settings-write API). Two owner-veto default judgments: `audit_all_authorization_decisions = false` (PHI access is always audited regardless), and preserve-but-surface the PHI-vs-synthetic relaxation split. Research-grounded (CISA *Secure by Design*, NIST SP 800-66r2 §164.312 crosswalk, CIS tiers, Jupyter loopback precedent). #26-clean | Accepted (2026-07-17) — owner signed off; built under BACKLOG #270 | diff --git a/docs/archive/throughput/THROUGHPUT-EXECUTION-PLAN.md b/docs/archive/throughput/THROUGHPUT-EXECUTION-PLAN.md index e5085e19..61621173 100644 --- a/docs/archive/throughput/THROUGHPUT-EXECUTION-PLAN.md +++ b/docs/archive/throughput/THROUGHPUT-EXECUTION-PLAN.md @@ -19,12 +19,12 @@ falsifier for the `2H` thesis, a cannot-claim ledger, and a persistence phase. T > > ⚠️ **ADR 0107's "do not build F2 or F3" uses the Phase-4 plan's labels, NOT this document's §Phase F labels — do > not map them across.** Against **this** plan's lever table: **F1 (`accepts=` seam) SHIPPED** -> ([ADR 0084](../../adr/0084-accepts-router-seam.md), [BACKLOG #213](../../BACKLOG.md)); **F2 (`fifo_claim_batch` flip) is -> owner-closed default-OFF**, priced at ≤ +4.7% against a +8% bar ([BACKLOG #212](../../BACKLOG.md); shipped +> ([ADR 0084](../../adr/0084-accepts-router-seam.md), [BACKLOG #213](../backlog/BACKLOG-CLOSED.md#213-accepts-seam-pure-router-stage-predicate-plus-an-advisory-lint)); **F2 (`fifo_claim_batch` flip) is +> owner-closed default-OFF**, priced at ≤ +4.7% against a +8% bar ([BACKLOG #212](../backlog/BACKLOG-CLOSED.md#212-fifo_claim_batch-decide-the-shipped-default-verification-done--it-is-not-a-no-op); shipped > `default=1`, [`config/settings.py:301`](../../../messagefoundry/config/settings.py)); **F3 (the pooled tempdb claim > rewrite) is ⛔ DECLINED** — but by [ADR 0114](../../adr/0114-phase-4-claim-path-call-complexity-reduction-driver-interface-redesign-ingress-routed-reset-fold.md) -> keeping the table variables as load-bearing for per-lane FIFO, **not** by ADR 0107 ([BACKLOG #210](../../BACKLOG.md)); -> **F4 (group-commit) is ⛔ DECLINED** ([BACKLOG #217](../../BACKLOG.md)); **F5's transform-overlap mechanism MERGED**, +> keeping the table variables as load-bearing for per-lane FIFO, **not** by ADR 0107 ([BACKLOG #210](../backlog/BACKLOG-CLOSED.md#210-remove-the-tempdb-table-variables-from-the-pooled-claim-query)); +> **F4 (group-commit) is ⛔ DECLINED** ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)); **F5's transform-overlap mechanism MERGED**, > its commit-collapse residual unbuilt, owner-deferred and ADR-gated ([BACKLOG #214](../../BACKLOG.md)). > > So **§Phase F is stale as a plan** — every lever it sequences has since shipped, been declined, or been deferred, diff --git a/docs/archive/throughput/THROUGHPUT-IMPROVEMENTS.md b/docs/archive/throughput/THROUGHPUT-IMPROVEMENTS.md index ee955428..75c257a8 100644 --- a/docs/archive/throughput/THROUGHPUT-IMPROVEMENTS.md +++ b/docs/archive/throughput/THROUGHPUT-IMPROVEMENTS.md @@ -17,7 +17,7 @@ as [BACKLOG #64](../../BACKLOG.md).* > [`config/wiring.py:2452`](../../../messagefoundry/config/wiring.py)). > > So **§2's "the database lever we *should* pull", §4 step 2 and §5 step 2 are stale as a plan** — group-commit is -> **⛔ DECLINED** ([BACKLOG #217](../../BACKLOG.md)), dead across ADR 0069 (the commit tier is ~9% utilised) → ADR 0099 +> **⛔ DECLINED** ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)), dead across ADR 0069 (the commit tier is ~9% utilised) → ADR 0099 > (which withdrew ADR 0055) → ADR 0107. Everything else here — the two axes, the core/transform axis, sharding — > is unaffected and is left exactly as written: it is the record of the reasoning at the time. Measured throughput > lives in [`benchmarks/TUNING-BASELINE.md`](../../benchmarks/TUNING-BASELINE.md); 45M/day remains a **target** @@ -156,10 +156,10 @@ peek. High payoff, but only after confirming parsing is the bottleneck. axis per representative feed — now anchored to the **Corepoint target** (§5). 2. ~~**Group-commit** (§2) — the single-node durable-write win; the **#1 unbuilt lever**; stays on existing backends. Lands under its **own ADR** when built (it touches the most invariant-dense code).~~ - **⛔ CLOSED 2026-07-13 — do not build it** (ADR 0107 / [BACKLOG #217](../../BACKLOG.md); see the banner). + **⛔ CLOSED 2026-07-13 — do not build it** (ADR 0107 / [BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path); see the banner). 3. **Lean-writes / carriage** — VARBINARY ciphertext ([#62](../../BACKLOG.md)), the `message_events` verbosity - knob ([#63](../../BACKLOG.md)), embedded-doc pruning ([#47](../../BACKLOG.md) / ADR 0042), retention - ([#34](../../BACKLOG.md)). + knob ([#63](../backlog/BACKLOG-CLOSED.md#63-message_events-verbosity-knob--operator-dial-to-suppress-routine-lifecycle-events-store-size--observability-p3)), embedded-doc pruning ([#47](../backlog/BACKLOG-CLOSED.md#47-embedded-document-base64-attachment-pruning--strip-obx-5--mfb64-blobs-after-a-per-connection-window-mirth-attachment-handler-parity-p2) / ADR 0042), retention + ([#34](../backlog/BACKLOG-CLOSED.md#34-per-connection-retention--pruning-windows-per-channel-message-storage-mirth-parity-p2)). 4. **Multi-process sharding by inbound** (§3a) — **BUILT** (ADR 0037); the multi-core path. For the shared-server backend, a **multi-DB log split** (move event/audit churn off the queue's writer) is a further I/O-isolation step — but the **atomic staged-queue transaction cannot be split**. @@ -196,7 +196,7 @@ against Rhapsody *marketing*, not this spec): denominator makes the store's commit headroom look **larger**, not smaller — it does not rescue the lever.)* And group-commit is no longer merely "unbuilt": it is **⛔ DECLINED** — [ADR 0099](../../adr/0099-phase-4-group-commit-amortize-the-per-event-transaction-cost.md) withdrew it, [ADR 0107](../../adr/0107-phase-4-is-closed-transaction-reduction-is-a-measured-dead-end.md) closed the - whole transaction-reduction class ([BACKLOG #217](../../BACKLOG.md)). + whole transaction-reduction class ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)). - **Storage** — runs higher, but mostly **by construction, not inefficiency**. The "~2× vs Corepoint" was estimate-vs-brochure and is **retracted**. The real, code-confirmed drivers are **carriage** (`NVARCHAR(MAX)` 2 B/char + base64 of the `mfenc` ciphertext → ~2.66·B on SQL Server; VARBINARY ciphertext @@ -207,12 +207,12 @@ against Rhapsody *marketing*, not this spec): **The measure-first path (each step gated on the one before):** 1. **Measure (the gate).** An enterprise-hardware `E_core` + sustained durable-write IOPS run — the local - **Windows Server 2025 + SQL Server 2025 box** ([#40](../../BACKLOG.md)) via the load harness - ([#28](../../BACKLOG.md)/[#29](../../BACKLOG.md)) — against the **9,200-IOPS / ~11 KB-msg / 20 + 16-core** target. + **Windows Server 2025 + SQL Server 2025 box** ([#40](../backlog/BACKLOG-CLOSED.md#40-ci-leg-against-the-local-windows-server-2025--sql-server-2025-box-real-hardware-coverage-p2)) via the load harness + ([#28](../backlog/BACKLOG-CLOSED.md#28-run-a-load-test-execute-the-load-harness-on-the-current-build)/[#29](../backlog/BACKLOG-CLOSED.md#29-run-a-throughput-test-re-measure--refresh-the-tuning-baseline)) — against the **9,200-IOPS / ~11 KB-msg / 20 + 16-core** target. Pins `E_core` and the binding axis. **Nothing builds before it.** 2. ~~**Group-commit** (§2) — *iff* durable-write-bound. Its own ADR.~~ **⛔ CLOSED (ADR 0107); the measurement it was gated on ran and the lever is a measured dead end.** -3. **Lean-writes / carriage** — the [#62](../../BACKLOG.md)/[#63](../../BACKLOG.md)/[#47](../../BACKLOG.md)/[#34](../../BACKLOG.md) +3. **Lean-writes / carriage** — the [#62](../../BACKLOG.md)/[#63](../backlog/BACKLOG-CLOSED.md#63-message_events-verbosity-knob--operator-dial-to-suppress-routine-lifecycle-events-store-size--observability-p3)/[#47](../backlog/BACKLOG-CLOSED.md#47-embedded-document-base64-attachment-pruning--strip-obx-5--mfb64-blobs-after-a-per-connection-window-mirth-attachment-handler-parity-p2)/[#34](../backlog/BACKLOG-CLOSED.md#34-per-connection-retention--pruning-windows-per-channel-message-storage-mirth-parity-p2) cluster. 4. **Multi-DB log split** — shared-server backend only. 5. **Deferred contingencies** — the scoped native engine-service core, free-threading diff --git a/docs/archive/throughput/throughput-build-plan.md b/docs/archive/throughput/throughput-build-plan.md index 725df7a9..eb70ca26 100644 --- a/docs/archive/throughput/throughput-build-plan.md +++ b/docs/archive/throughput/throughput-build-plan.md @@ -21,7 +21,7 @@ parallelism, method, coordination. > permanently (`inline: bool = False`, [`config/wiring.py:2452`](../../../messagefoundry/config/wiring.py)), and **no > throughput claim may be made for it** — the "aggregate-commit lever" note on its row is retained as the > reasoning of the time, not as a measured effect. **B9** (Postgres `commit_delay` / group-commit) is -> **⛔ DECLINED** ([BACKLOG #217](../../BACKLOG.md)). +> **⛔ DECLINED** ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)). > > The rest of this plan — the connection-scale items, the claim-storm program, and the engine-side per-message CPU > frontier — is **unaffected** and is left exactly as written. Measured throughput lives in @@ -176,7 +176,7 @@ re-profile on the production platform before final box sizing; (c) a clean 4-eng | **B6** | shared **ingest sub-pool** / writer path | ⬜ TODO | B11/B13 | Give pre-ACK intake its own commit path so ACK-on-receipt latency doesn't queue behind the worker-claim + idle-poll storm. **Shared sub-pool carved from B13's pool — NOT per-listener** (1,500 dedicated conns is infeasible). | | **B14** | per-listener accept-cap default | ⬜ TODO | B11 | `DEFAULT_MAX_CONNECTIONS=256` × 1,500 listeners = 384k socket ceiling. Lower the default (most HL7 partners hold 1–4), keep the per-connection override. | | **B10** | rename-based FIFO index migration | ✅ **DONE** (ADR 0060, #676) | B3 | renames to `ix_queue_fifo_*_seq` + idempotent on-open DROP-old/CREATE-new on all 3 backends, so B3's cut lands on **upgraded** DBs. | -| **B9** | Postgres `commit_delay` (group-commit) | ⛔ **DECLINED** (was ⏸ DEFERRED) | — | ~~only meaningful once B8 shows durable-write-bound AND the store presents concurrent in-flight txns~~ — B8/ADR 0069 showed the commit tier is **~9% utilised**, ADR 0099 withdrew ADR 0055, and ADR 0107 measured the whole `txn/event` class flat (elasticity **−0.115**). Dead by measurement three times over ([BACKLOG #217](../../BACKLOG.md)). Still unbuilt in code — `commit_delay` is described as a planned PG-only increment at `config/settings.py:281-284`. | +| **B9** | Postgres `commit_delay` (group-commit) | ⛔ **DECLINED** (was ⏸ DEFERRED) | — | ~~only meaningful once B8 shows durable-write-bound AND the store presents concurrent in-flight txns~~ — B8/ADR 0069 showed the commit tier is **~9% utilised**, ADR 0099 withdrew ADR 0055, and ADR 0107 measured the whole `txn/event` class flat (elasticity **−0.115**). Dead by measurement three times over ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)). Still unbuilt in code — `commit_delay` is described as a planned PG-only increment at `config/settings.py:281-284`. | | **T8** | backend-parametric test/measurement | ⬜ TODO | — | run every connection-scale + commit result across SQLite/SQL Server/Postgres. | ## Execution sequence diff --git a/docs/archive/throughput/throughput-roadmap.md b/docs/archive/throughput/throughput-roadmap.md index ecfca1be..7264a33f 100644 --- a/docs/archive/throughput/throughput-roadmap.md +++ b/docs/archive/throughput/throughput-roadmap.md @@ -24,8 +24,8 @@ is NOT the wall; see the B8 result below.** > > So **"cut the chain first, then multiply lanes" and the ranked-lever table are stale as a plan**: lever 1 was > built and buys nothing; lever 2 (`fifo_claim_batch`) is owner-closed **default-OFF**, priced at ≤ +4.7% against -> a +8% bar ([BACKLOG #212](../../BACKLOG.md)); `commit_delay` / group-commit is **⛔ DECLINED** -> ([BACKLOG #217](../../BACKLOG.md)). **Everything below is retained as the record of the diagnosis and of why the work +> a +8% bar ([BACKLOG #212](../backlog/BACKLOG-CLOSED.md#212-fifo_claim_batch-decide-the-shipped-default-verification-done--it-is-not-a-no-op)); `commit_delay` / group-commit is **⛔ DECLINED** +> ([BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path)). **Everything below is retained as the record of the diagnosis and of why the work > was sequenced this way — it is not the live plan.** Measured throughput lives in > [`benchmarks/TUNING-BASELINE.md`](../../benchmarks/TUNING-BASELINE.md); the frontier is **engine-side attribution** > (ADR 0107 §"Out of scope"), and there is no identified throughput lever today. @@ -254,7 +254,7 @@ sharding only *multiplies* the per-lane rate, sub-linearly; it is worth far more | **5** | **Executor sizing / split** (give aioodbc its own pool, or raise the default) | Small secondary **above 4 lanes** (measured ~100→140); SQL-Server-only (asyncpg has no executor) | **build-with-guardrail** | Preserve the EF-6 no-MARS single-active-statement + cursor-close-before-release. | | **6** | **Dedicated ingest lane** (reserved server conn / SQLite priority-gate) | Robustness/no-drop, not throughput — eliminates silent intake socket-shed | **build-with-guardrail** | SQLite = bounded-fair priority-gate on `:799`, never a 2nd writer; `[inbound].max_ingress_depth` TCP-backpressure. | | **7** | **SQLite `synchronous=NORMAL` baseline** (measurement + `db_status()` observability) | Sizes the honest shipped-default SQLite number; not a throughput win | **build-with-guardrail** | Report a band; surface `synchronous`; document the durability relaxation. | -| — | **PG `commit_delay`** | ~1.00× at sustained sharded rates (commits rarely bunch) | ⛔ **DECLINED** (was "defer, gated, off, last") — ADR 0107 / [BACKLOG #217](../../BACKLOG.md) | PG-only GUC; re-test only if batch-claim re-bunches commits. | +| — | **PG `commit_delay`** | ~1.00× at sustained sharded rates (commits rarely bunch) | ⛔ **DECLINED** (was "defer, gated, off, last") — ADR 0107 / [BACKLOG #217](../backlog/BACKLOG-CLOSED.md#217-group-commit--durable-write--sequenced-after-the-claim-path) | PG-only GUC; re-test only if batch-claim re-bunches commits. | | — | `DELAYED_DURABILITY` · extend the SQLite committer to servers · READPAST in the FIFO claim · `shard_key`-as-index-lead · pool tuning | — | **REJECT / N/A** | durability / by-design / per-lane FIFO (#285) / page-latch refuted / pool ran 40. | ### Improvement model (honest) diff --git a/docs/design/freethread-spike.md b/docs/design/freethread-spike.md index 5e385562..618f8973 100644 --- a/docs/design/freethread-spike.md +++ b/docs/design/freethread-spike.md @@ -171,7 +171,7 @@ built-in-type work scales 5.7–7.6×, and a **full HL7 parse into dict/list/str faster single-thread** (158k vs 11k msg/s), while python-hl7 (2.02×) and hl7apy (2.04×) both stall on their `Container(collections.abc.Sequence)` object trees (shared class/type contention under free-threading; not allocation, not GC). **So free-threading is viable IFF the hot-path parse is replaced by a low-allocation -built-ins HL7 parser** ([BACKLOG #88](../BACKLOG.md)) — itself the highest-leverage perf win (helps +built-ins HL7 parser** ([BACKLOG #88](../archive/backlog/BACKLOG-CLOSED.md#88-low-allocation-built-ins-hl7-parser--free-threading-keystone--14-single-thread-peek-speedup-p2)) — itself the highest-leverage perf win (helps single-process + sharding too; a parser ADR should precede the build). **WS4** (invariant preservation under threads) is re-enabled by the conditional GO and runs *after* the parser, *if* free-threading is chosen over ADR 0037 sharding. Full detail: ADR 0053 "WS3 (multi-core scaling)". diff --git a/docs/message-ordering-design.md b/docs/message-ordering-design.md index bc289591..898a8efc 100644 --- a/docs/message-ordering-design.md +++ b/docs/message-ordering-design.md @@ -8,7 +8,7 @@ policy — partner-NAK vs transport vs internal-error split, the **retry-forever [`pipeline/alerts.py`](../messagefoundry/pipeline/alerts.py) defaulting to `WARNING` logs, and the `connection_stopped` emit-point), and Layer 4b (the `queue_buildup` detector + its `buildup_max_depth` / `buildup_max_oldest_seconds` threshold and the `pending_depth` store query). A **real** alerting -framework (routing the `AlertSink` events to notifications) remains [`BACKLOG.md`](BACKLOG.md) item 5; +framework (routing the `AlertSink` events to notifications) remains [`BACKLOG-CLOSED.md`](archive/backlog/BACKLOG-CLOSED.md#5-alerting-framework--fifo-operational-alerts-near-term--the-fifo-defaults-depend-on-it) item 5; the next foundational step is **Phase 2** — per-stage durable queues (ADR-first, top of `BACKLOG.md`). Companion to the engine survey in [`hl7-message-ordering-reference.md`](hl7-message-ordering-reference.md); per-key ordering is the diff --git a/docs/research/cloud-deployment-research-2026-06.md b/docs/research/cloud-deployment-research-2026-06.md index caca92aa..a07997c5 100644 --- a/docs/research/cloud-deployment-research-2026-06.md +++ b/docs/research/cloud-deployment-research-2026-06.md @@ -5,7 +5,7 @@ **Recorded 2026-06-22.** Captured after the container fast-follow (ADR 0017) shipped (PR #480: slim + `-sqlserver` engine images, compose, single-node k8s StatefulSet). This is the saved basis for -[`BACKLOG.md`](../BACKLOG.md) **#41** — read it if/when a cloud or Kubernetes adopter materializes. +[`BACKLOG-CLOSED.md`](../archive/backlog/BACKLOG-CLOSED.md#41-cloud--kubernetes-ha-deployment-packaging-container-fast-follow-follow-ons) **#41** — read it if/when a cloud or Kubernetes adopter materializes. > **Provenance / confidence.** Point-in-time, AI-assisted research synthesis (multi-agent web research + > a codebase assessment) — **not** a vendor benchmark or a committed plan. The dedicated *Mirth* research diff --git a/docs/research/config-ux-review.md b/docs/research/config-ux-review.md index a7138439..17bedbff 100644 --- a/docs/research/config-ux-review.md +++ b/docs/research/config-ux-review.md @@ -3,7 +3,7 @@ **Date:** 2026-06-19 · **Status:** research / findings (no code) · **Owner action:** see *Candidate follow-up items* + *Circulation* below. -This is BACKLOG **[#33](../BACKLOG.md)** — a **review/design** pass over *how* an operator or analyst +This is BACKLOG **[#33](../archive/backlog/BACKLOG-CLOSED.md#33-review-the-end-to-end-configuration-method-across-every-surface-config-ux-consolidation)** — a **review/design** pass over *how* an operator or analyst actually configures a MessageFoundry deployment, before the surfaces multiply further in v0.2+. It **identifies and circulates** findings only; **no code or config is changed here**, and any *acted-on* recommendation is recorded below as a **separate** backlog candidate with its real file contention