Skip to content

Publish Flow 1.31 evidence and enforce maturity boundaries - #286

Draft
abrichr wants to merge 1 commit into
mainfrom
codex/production-evidence-contract
Draft

Publish Flow 1.31 evidence and enforce maturity boundaries#286
abrichr wants to merge 1 commit into
mainfrom
codex/production-evidence-contract

Conversation

@abrichr

@abrichr abrichr commented Aug 18, 2026

Copy link
Copy Markdown
Member

What changed

  • publishes a fresh Flow 1.31.0 comparison, independent replication, transaction-fault probe, and remote-lease contract probe
  • binds 93 counted trials to the exact Flow wheel, sdist, release source commit, Evals commit, runners, reports, task/oracle contracts, dependencies, browser version, and Chromium revision
  • marks the prior Flow 1.30.0 evidence as superseded instead of changing or deleting it
  • documents the difference between release-fresh evidence and production acceptance
  • adds machine checks for the minimum trial count, task, oracle or invariants, failure taxonomy, caveats, silent-incorrect-success and over-halt coverage, evidence class, scope booleans, and set-level production-acceptance consistency
  • rejects production-acceptance claims from synthetic or fixture evidence

Why

The daily evidence-freshness workflow had failed after Flow 1.31.0 replaced Flow 1.30.0 on PyPI. The registry also did not express whether a current campaign was production acceptance. This made a release-fresh label easy to read as a broader maturity claim.

Evidence and impact

The comparison and replication each ran 27 trials: three arms, three conditions, and three trials per condition. Compiled replay was correct in 18 of 18 trials across both complete runs, with zero silent incorrect successes, zero wrong actions, zero over-halts, and zero model calls.

The transaction probe ran 30 trials. The remote-lease probe ran 9 trials. They validate bounded safety contracts, but they do not count both required production reliability metrics. Their manifests therefore state production_acceptance: false.

This PR does not claim hosted, customer-workflow, native desktop, RDP, or Citrix production acceptance. The new readiness document names the exit conditions for those tracks.

Validation

  • 31 passed in tests/test_published_evidence_freshness.py
  • Ruff check and format check passed
  • offline evidence binding passed
  • live PyPI check confirmed Flow 1.31.0 and wheel SHA
  • source-boundary check passed
  • release-lock check passed
  • git diff --check passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant