Publish Flow 1.31 evidence and enforce maturity boundaries

README.md

@@ -246,16 +246,23 @@ matches the current published release. The current set also has an
 `EVIDENCE_MANIFEST.json` that inventories the verifier scripts, retained
 artifacts, public reports, campaign environments, and task/oracle contracts.
 The 1.28 set remains stale because its run did not retain an exact browser
-revision or installed dependency snapshot. The current 1.30 set records those
+revision or installed dependency snapshot. The current 1.31 set records those
 facts for every campaign. The check runs offline on every pull request and
 against PyPI on a daily schedule (`.github/workflows/evidence-freshness.yml`).
 Re-run the comparison and publish a new evidence set rather than editing an old
 one: superseded reports stay reproducible against the wheel they were measured
 on.
 
-See `docs/eval_results/current_flow_v1_30_0_local_20260806/` for the current
-comparison and its exact retained campaign reports. The older 1.28 set remains
-available as stale historical evidence; it was not relabeled.
+The word `current` means release-fresh. It does not mean production-accepted.
+Each campaign records its evidence class, whether it is production acceptance,
+and whether it counts silent incorrect success and over-halt. The present set
+contains local synthetic and contract-fixture evidence only. See the
+[production-readiness evidence boundary](docs/eval_results/PRODUCTION_READINESS.md)
+for the missing acceptance tracks and their exit conditions.
+
+See `docs/eval_results/current_flow_v1_31_0_local_20260818/` for the current
+comparison and its exact retained campaign reports. The older sets remain
+available as stale historical evidence; they were not relabeled.
 
 ## More workflows
 

docs/eval_results/PRODUCTION_READINESS.md

@@ -0,0 +1,67 @@
+# Production-readiness evidence boundary
+
+OpenAdapt has production-capable mechanisms. It does not have one global
+production-readiness state. Each workflow, application, version, environment,
+identity rule, effect oracle, and deployment boundary needs its own
+qualification.
+
+The `current` label in `PUBLISHED_EVIDENCE.json` has one narrow meaning: the
+evidence set matches the current published `openadapt-flow` release. It does
+not mean the campaign is production acceptance. The evidence manifest records
+that distinction in `campaigns[].evidence_scope.production_acceptance`.
+
+## What the current public Evals set establishes
+
+The Flow 1.31.0 set contains four campaigns. Every condition has three trials.
+
+| Campaign | Environment | What it measures | Production acceptance |
+| --- | --- | --- | --- |
+| Comparison | Synthetic MockMed, local macOS host, headless Chromium | Compiled replay and two Playwright controls under clean, theme, and bounded label drift | No |
+| Independent replication | The same bounded environment in a second complete run | Repeatability of the comparison result | No |
+| Transaction probe | Synthetic MockMed fault server and its independent persistence store | Outcome taxonomy under normal, timeout, optimistic-screen, session, and duplicate faults | No |
+| Remote lease safety | An instrumented fake remote backend | The input-edge refusal contract when a frame lease is absent or changes | No |
+
+The comparison and replication count silent incorrect success and over-halt.
+The transaction and remote-lease probes do not count both metrics as production
+reliability measures. Their manifest entries say `not_counted`. They therefore
+cannot satisfy a production-acceptance gate.
+
+No campaign in this set represents hosted execution, a customer workflow,
+Windows UIA, native macOS Accessibility, Linux AT-SPI, a real RDP session, or a
+real Citrix ICA/HDX session. Other repositories retain their own bounded
+substrate campaigns. Those results remain specific to their exact tasks and
+environments, and no campaign in this repository establishes production
+acceptance.
+
+## Acceptance exit conditions
+
+A production claim for one workflow needs one immutable acceptance record that
+contains all of these facts:
+
+1. The exact task, application version, operating system, display/session
+   properties, OpenAdapt release, runner, and deployment revisions.
+2. At least three trials for each healthy and fault condition in the accepted
+   matrix, with no hidden retry or excluded run.
+3. An independent oracle for the intended effect, wrong-record effects,
+   duplicates, collateral changes, and effect absence or uncertainty.
+4. A complete failure taxonomy that counts silent incorrect success,
+   over-halt, safe halt, uncertain delivery, platform failure, and operator
+   intervention.
+5. The full delivery path for that operating model. A managed-browser result
+   includes authenticated submission, idempotency, polling, one-use delivery,
+   execution, independent verification, a signed receipt, and a signed webhook.
+6. Real configured production dependencies. A mock or simulated success cannot
+   satisfy the gate.
+7. A clean-machine install and first-run check on every operating system that
+   the claim names.
+
+For a real RDP or Citrix claim, the record also includes reconnect, session
+change, stale-frame, resolution/DPI, and compression fault conditions. A fake
+remote backend proves a runtime contract only. It does not replace a real
+session result.
+
+For a generally available Desktop release, distribution evidence is separate
+from workflow qualification. The release needs the intended signing,
+notarization or Authenticode, installer, update, rollback, and key-lifecycle
+evidence. An unsigned or ad-hoc-signed Beta installer can still provide useful
+evaluation evidence, but it is not a generally available signed release.

docs/eval_results/PUBLISHED_EVIDENCE.json

@@ -6,6 +6,8 @@
     "was measured against. scripts/check_published_evidence_freshness.py fails",
     "when a set marked current no longer pins the current published release, or",
     "when a pinned wheel digest does not match that release on PyPI.",
+    "The current label means release-fresh, not production-accepted. Every",
+    "current entry and campaign states its production_acceptance value.",
     "Mark a set superseded (never delete it) when a newer set replaces it: the",
     "old numbers stay reproducible against the wheel they were measured on."
   ],
@@ -49,12 +51,27 @@
     },
     {
       "path": "docs/eval_results/current_flow_v1_30_0_local_20260806",
-      "status": "current",
+      "status": "superseded",
+      "superseded_by": "docs/eval_results/current_flow_v1_31_0_local_20260818",
       "flow_version": "1.30.0",
       "wheel_sha256": "7bf1a7b00388172a79bda666def182688c296ffb5bc9be2fd3281169fc36ae63",
       "sdist_sha256": "3a402610e35f47fd54daaf066b8c3a6006c13483cb4c778740014abea1854ea4",
       "evidence_manifest": "docs/eval_results/current_flow_v1_30_0_local_20260806/EVIDENCE_MANIFEST.json",
-      "measured_on": "2026-08-06"
+      "measured_on": "2026-08-06",
+      "production_acceptance": false,
+      "stale_reason": "These retained results measured Flow 1.30.0. They were not relabeled. A complete Flow 1.31.0 rerun replaced them after the published release changed."
+    },
+    {
+      "path": "docs/eval_results/current_flow_v1_31_0_local_20260818",
+      "status": "current",
+      "flow_version": "1.31.0",
+      "flow_source_commit": "2d225dea9a0ad29ca84ce1b037cc0ac671367e28",
+      "flow_release_tag": "v1.31.0",
+      "wheel_sha256": "81133db1528ad1bb1f26e3fcb6aea61b0651db6d905cf2e4943e8383c1f3d29c",
+      "sdist_sha256": "cf1fc356d14d267df82be188de3e9a3575734f18f46ef91ac8075438cc731540",
+      "evidence_manifest": "docs/eval_results/current_flow_v1_31_0_local_20260818/EVIDENCE_MANIFEST.json",
+      "measured_on": "2026-08-18",
+      "production_acceptance": false
     }
   ]
 }