fix: require passing evidence for supported claims by abrichr · Pull Request #371 · OpenAdaptAI/openadapt-flow

abrichr · 2026-08-18T18:25:35Z

Summary

This draft replaces #368. GitHub closed #368 when its merged stack base branch was deleted, and GitHub does not permit a closed pull request to change its base.

A supported claim no longer passes because a cited test file exists. The required CI job must supply real JUnit evidence that each cited file ran and passed.

Root cause

The former gate had four fail-open paths:

Test-file existence was enough to classify evidence as supported.
--junit was optional.
A missing JUnit file produced a warning and passed.
The parser expected a file attribute that Pytest xUnit2 does not emit, so real Pytest output could produce an empty status map.

Change

The existing required test job emits unit JUnit and validates unit-backed claims.
The existing required e2e-browser job emits browser JUnit and validates browser-backed claims.
The separate fast claims workflow remains a structure-only gate.
Missing, malformed, empty, absent, all-skipped, failed, or errored evidence fails closed.
Pytest xUnit2 dotted class names map to canonical repository test paths.
Generated verification output names the required CI binding and does not claim a live pass from its static registry view.
The public artifact inventory contains the workflow, registry, and generated JSON hashes.

This reuses the existing required test executions. It does not add a duplicate test suite or a new expensive CI job.

Independent review fixes

A validating report now requires an explicit evidence-file scope.
Every selected file must contain at least one pass and no failure, error, or all-skipped result.
The report records the exact scope. It does not imply that unselected RDP, Citrix stand-in, or retained OpenEMR evidence ran.
The hosted refresh no longer claims macOS/Parallels evidence from Ubuntu or an unrelated oa-vm endpoint.
The refresh runs only when a repository owner enables a self-hosted macOS ARM64 runner with local Parallels, an exact VM UUID, and a base snapshot binding. A skipped job produces no green report.
The physical job requires the dedicated openadapt-parallels runner label and uses one global concurrency group across branches and dispatches. GitHub can supersede only the read-only pull-request gate, not a physical run during cleanup.
Workflow preflight and both live tests prove the exact configured base snapshot is current before the first VM mutation.
The pixel test runs first. The validating tests no longer create ephemeral snapshots, so no process-local snapshot ID can be lost on termination.
Before the first VM mutation, each test atomically persists a private recovery journal that binds one exact VM UUID to one exact preserved base UUID. Normal cleanup, an unconditional workflow cleanup step, and the next run all use the same recovery function.
Recovery unconditionally reverts the exact base, verifies that it is current, suspends the VM, verifies the final state, and clears the journal last. A failed recovery retains the journal and rejects the evidence.
The live-test step has a shorter timeout than the job. The reserved job time lets unconditional recovery run after a test timeout. Manual cancellation or runner failure can leave the journal, which the next run reconciles before new mutation.
prlctl status and snapshot-list now require a zero exit status. Plausible stdout from a failed command cannot satisfy the final-state or current-snapshot proof.
Each following desktop trial restores and verifies the same base before it starts, so the earlier pixel test and each prior trial cannot supply hidden state.
The operator runbook now documents the actual three-trial Patient Notes proof and its exact snapshot lifecycle.
The inherited Windows Job Object test now polls for lock release for at most the existing five-second process-exit contract. Product cleanup behavior is unchanged, and the synthetic child retains the lock for 30 seconds, so a surviving child still fails the test.

Rebase

Rebased onto the ci: bound qualification and browser setup #370 merge commit 973e21c458de1bccce67ba23b24ce58ce2cbf7ab on main.
Exact head: 357df9324c33263d35a6b13db1b955597e0600da.
The diff contains only the 17 intended workflow, claim, documentation, validator, VM lifecycle, recovery, inventory, and test files.

Local validation

Focused validator, workflow contract, and VM lifecycle suite: 65 passed.
Combined installer, validator, workflow contract, and VM lifecycle suite after the Windows test fix: 86 passed, 4 platform skips.
Opt-in live-test collection: four tests in order (one pixel proof, then three desktop trials).
Claims structure gate: passed.
Claim report regeneration: reproducible with no tracked diff.
Release consistency and public artifact inventory: passed.
actionlint: passed.
Ruff check and format check: passed.
Mypy for the validator and Parallels VM lifecycle: passed.
git diff --check: passed.

Merge order

Do not merge this draft until exact-head CI and independent review pass. The physical Parallels job intentionally skips on pull requests; a passing structure gate is not live Parallels evidence.

…after rebase The claims gate from PR #371 emits ci_job for each cited evidence node, and the browser attach claim entry is new. Regenerate the derived report and the reviewed public artifact inventory so both match the registry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(browser): attach recorder to existing sessions * docs: align generated verification timestamp * fix(browser): harden attached recording boundaries * fix(browser): harden attached recording finalization * fix(browser): close attach evidence gaps * fix(browser): latch attached recording races * fix(browser): close attached finalization gaps * fix(browser): bind same-task secret mutations * fix(browser): retain masks through final capture * docs: keep generated verification report in sync * Harden browser attach secret boundaries * Refresh browser attach verification artifacts * Use deterministic claims report timestamp * fix(browser): make secret classification sticky across DOM replacement Close the four reproduced source-time secret leaks the independent review found in the attach recorder, plus two adjacent races, and repair the finalization lifecycle latch that the fix exposed. Secret boundaries: - A page can replace a declared secret field before its first input event. Attribute records are now applied ahead of rewrite matching, and the removed node's sticky state transfers to its single proven replacement. - A field can lose its name/id while typing. Classification is sticky to the DOM element and its input session; an attribute change cannot remove secrecy. - A multi-node rewrite has no proven field mapping. Every candidate is masked AND its first input refuses the recording, instead of assigning one removed field's identity (and input session) to all of them. - A cached field label can reproduce the current secret value. Trusted labels are scrubbed against that field's live value. - ARIA textboxes read innerText/textContent, so their values enter the scrub set; printable keydowns on a bound secret field no longer emit hotkeys; native non-text controls no longer trigger a false refusal. Screenshot and CDP boundaries: - The closed-shadow inventory runs as a screenshot guard, so a root that appears during screenshot preparation is bound or refused before any bytes exist. Launched mode gets the same initial inventory. - The origin guard reads only location.origin instead of the full URL, so a reflected secret in the query string cannot cross into Python. - The privacy scan scopes its resolved nodes to one protocol object group and releases it after each scan. - Marker cleanup pierces open shadow roots, and a session-keyed cleanup handle survives a page that replaces the global recorder object. Finalization: - finish() runs the privacy guard BEFORE arming the finalization latch. The guard's page round-trips deliver lifecycle events Chromium queued during recording; arming first made an ordinary same-origin history.replaceState from the recording itself alias a post-evidence frame change and refuse every valid recording that reflects state. - _handle_frame_navigation refuses on the armed latch without another page evaluate, which would re-enter dispatch while finalizing. Tests: - Live Chromium regressions for pre-input replacement, same-task identity loss, ambiguous multi-node rewrite, label/value equality, ARIA textbox, AltGr keydown, static and late unbound closed shadow roots, and a replaced page privacy guard. - The five finalization tests use a fake privacy CDP session, since the guard now runs on the finish path. - The live trial sequence reactivates the app tab after each popup or new-tab refusal. Chromium throttles a backgrounded tab, which made the next trial's first evidence screenshot time out on a loaded runner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(claims): regenerate verification report and artifact inventory after rebase The claims gate from PR #371 emits ci_job for each cited evidence node, and the browser attach claim entry is new. Regenerate the derived report and the reviewed public artifact inventory so both match the registry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browser): bind secret redaction to the element, never to keystroke prefixes The page closure retained the value of a declared secret field on EVERY input event, so it kept each intermediate keystroke prefix, then substring-replaced every retained value across later URL, title, label, selector and structural text -- iterating over the already-substituted string, so it also rewrote the inside of its own `[secret]` placeholder. An independent review reproduced three consequences in real Chromium: 1. Silent identity degradation in every mode. With `charlie1` typed into a password field, a later click on `#chart-save` recorded `{"selector": null, "name": "Save [se[secret]ret]t"}`. `uniqueSelector` returned null whenever scrubbing altered an element id, so the DOM identity tier disarmed for any element whose id/name/data-testid contained one character of the prefix. Nothing refused; the bundle compiled and looked healthy. 2. A false abort in attach mode. `emit` wrote the scrubbed `location.href` into every event and the origin guard parsed that as the origin, so on `http://host.test/app` with secret `hunter2` the first keystroke produced `"[secret]ttp://[secret]ost.test/app"` and refused the whole recording with "a browser event came from outside the declared application origin". 3. Corrupted URL evidence on disk in launched mode (`"url_after": "h[secret][secret]p://127.0.0.1:65479/"`). What changed: - Redaction is bound to the ELEMENT, not to a growing value set. The closure keeps the bound element and reads its CURRENT value at scrub time; it never retains a keystroke prefix. Values are committed only where the field can no longer take a keystroke (change, focusout, submit, pagehide), so a value stays scrubbable after the page clears the field. - Redaction makes one left-to-right pass over the original text and never re-reads its own output, so a placeholder can no longer be rewritten. - A URL is redacted one whole token at a time (path segment, query name, query value, fragment). The scheme, host and port are never rewritten. - `emit` sends `location.origin` beside the scrubbed URL and the origin guard reads that, so redaction can never refuse a valid recording. - A value too short to tell a real reflection from a coincidence FAILS CLOSED: Flow withholds the whole URL token, title, label or structural text instead of keeping a partially rewritten copy. A short secret still never persists. - Identity refusal is visible, never a bare null selector: the action carries `identity_withheld`, `meta.json` carries `identity_withheld_events`, and `record` prints how many actions carry no DOM selector and why. - The trusted field-label cache is scrubbed against every declared value, not only the field's own value. Document order meant a field bound before another declared field cached that other field's pre-filled literal, which reached `events.jsonl` as a plain `field_label`. - A declared value no longer survives a same-origin GET form submit. Each document builds its own closure, so a document that does not hold the value cannot scrub it: Flow withholds the URL and title for such a document, notes it in `meta.json`, and reports it. A later document that DOES hold the value scrubs itself and keeps its evidence. - `meta.json` is stamped with the recorded surface BEFORE the atomic publish, so a crash can no longer publish a surface-unbound recording; the CLI stamp is now a no-op when the recorder already stamped it. - The `record` secret line states what Flow actually guarantees. Tests (all fail on the parent commit, pass here): - `test_page_closure_keeps_url_and_identity_evidence_for_a_lowercase_secret` - `test_page_closure_scrubs_a_cached_label_holding_another_declared_secret` - `test_launched_recording_withholds_url_evidence_after_a_get_form_submit` - `test_attached_recorder_reads_the_origin_the_page_reports` - `test_structural_text_is_withheld_after_a_secret_leaves_its_document` - `test_recording_privacy_notices_report_what_flow_withheld` - `test_stamping_a_recorded_surface_does_not_rewrite_a_published_recording` Trial 1 of the live CDP campaign now types a lowercase secret whose characters occur in the attach URL. Every live secret in this file was an uppercase phrase sharing no character with `http://127.0.0.1:<port>/`, and that blind spot hid all three defects. The browser claim no longer says the live campaign covers "URL/title reflection" without qualification: both reflection cases use `history.replaceState`, so the coverage is same-document. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(browser): assert the refused DOM identity states its reason A withheld selector must never read as an element that simply had no stable identity. Lock the reason field on the live case whose button id IS the declared secret value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browser): recognise a keystroke prefix per declared field, not globally A controlled input that swaps its DOM node on every change made every commit point fire in the middle of typing: replacing the focused element fires focusout, so the closure committed `c`, `ch`, `cha`, … as if each were a declared value. Those short values then made every later scrub ambiguous, so the recorder withheld the URL, the title, and the DOM identity of unrelated elements for the rest of the recording — fail closed, but it destroyed evidence a correct recording should keep. Committed values are now filed under their DECLARED FIELD (falling back to the input session when a field has no declared name). At scrub time a value that another value of the SAME field continues is dropped: it is a keystroke prefix of that field's value, not a declared value of its own. The complete value is still redacted, and two declared fields never share a key, so one field's value can never suppress another's. Regression: `test_page_closure_keeps_evidence_when_a_secret_input_swaps_its_node` types a lowercase secret into a field that replaces its node on every keystroke, then clicks an unrelated button and requires the exact selector, the exact accessible name, and the exact URL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(browser): correct the commit-point comment A commit point can fire in the middle of typing, because replacing a focused element fires focusout. Say so, and point at the per-field prefix rule that handles it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browser): never drop a value the field holds or a commit point recorded The prefix rule from 5a871d4 inferred a keystroke prefix from string shape alone, so a longer COMMITTED value suppressed the field's own CURRENT value. An independent re-review reproduced the leak end to end: type `hunter2`, blur (which commits it), return and press Backspace once, and the live value `hunter` was dropped as a "prefix" and never redacted -- `"url_after": "http://127.0.0.1:54721/charts/hunter"`, `"title_after": "session for hunter"`, no withheld marker, and the CLI printing a promise that was false there. The parent commit corrupts that evidence but does not leak it, so the rule was strictly worse in that shape. Retention now tracks WHERE a value came from instead of guessing from its shape: - Committed values are filed under the ELEMENT that held them at the commit point, not under the field. `change` and `focusout` commit only the element they name, so replacing a focused node no longer commits the prefix that its successor happens to hold at that instant. - A value is droppable only when a controlled input REPLACED the node that holds it, the page no longer holds that node, and another value of the same field continues it. `supersededSecretElements` records that replacement where it is proven: the single-node mutation transfer, and the focus transfer to a replacement of a disconnected active element. - A value the field currently holds is never dropped. - Each element keeps the last value it held, and every earlier value long enough to identify. A page that reflects a field as the operator types writes the PREVIOUS value into the title or the URL, and that value must still be redacted. Two more findings from the same review: - A password field with no `name` and no `id` had no stable key: discovery derives a new input session per replacement, so a swapping node looked like a new declared field on every keystroke, no prefix was ever recognised, and every later scrub became ambiguous -- the "fail closed but useless" state. A replacement now inherits the state, and the input session, of the node it replaced, whether the mutation batch or a focus transfer proves the replacement. - A withheld accessible name, role, or clicked-row identity disarmed an identity check with no marker at all; only a withheld identity ATTRIBUTE was marked. `scrubIdentityText` now records the reason, `structuralTarget` reports it in `identity_withheld`, a withheld row identity travels as `sid_withheld`, and the recorder counts the action once for the operator. Also: `scrubSecretUrl` returns the URL the page reports when it redacted nothing, instead of a rebuilt URL with normalised percent-encoding. Regression tests (all four fail on c8f2aa5, pass here): - `test_page_closure_redacts_a_value_the_field_still_holds_after_a_commit` - `test_page_closure_keeps_evidence_when_an_unnamed_password_swaps_its_node` - `test_page_closure_marks_a_withheld_name_and_row_identity` - `test_withheld_row_identity_is_counted_for_the_operator` Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(browser): state the retention rule and every withheld identity Say which values redaction keeps -- everything the field holds and everything a commit point recorded -- and which single case it drops. Extend the never-disarm-silently statement to the accessible name, the role, and the clicked-row identity, which now carry their reason too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browser): replace secret value retention with capture-time withholding Three independent reviews each found a different blocking defect in the recorder's secret-redaction mechanism, and each fix moved the defect into the same retention rule. The literature says why: removing a value from text that was already captured is a known-unsolved problem. Englehardt, Acar and Narayanan measured every major session-replay vendor in 2017 and found that none redacts displayed content automatically and that all of it leaked; PostHog and Sentry still carry open issues for secrets in replay URLs. Production tools solve it at capture time instead, with element-bound, deny-by-default masking. This replaces the retention rule with that model. One rule now governs every text the recorder produces: report it exactly, or withhold it and say why. There is no placeholder substitution left anywhere in the recorder. Root cause. `emit()` sampled `location.href` and `document.title` inside a CAPTURE-phase listener, before the page's own handlers ran, so the values it read described the state before the action. The value history existed only to repair that staleness, and the history produced both open P1s. Python never read those event fields: the recorded URL and title already came from `_read_scrubbed_page_state()` at the settled boundary. The events now carry no reflected text at all, and that boundary is the only sampling point. Deleted: `observedSecretValues`, `committedSecretValues`, `retainedByKey` and its superseded/prefix-drop ladder, `supersededSecretElements`, the `change`/`focusout`/`submit`/`pagehide` commit points, `redactSecretOccurrences` and every scrub-and-rewrite path (`scrubSecretText`, `scrubSecretUrl`, `redactUrlToken`, `SECRET_PLACEHOLDER`). Added: - `liveSecretValues()` -- values that CONNECTED bound elements hold at match time, read live. A node the page detached is not a source of values, which is what a controlled input leaves behind on every keystroke. - `identityTextOrNull()` -- identity evidence (selector, role, accessible name, clicked-row identity, receiving field name) is exact or withheld with a reason. Replay compares identity against the live page, so a rewrite would compare against text the page never held, invisibly. - `safePageState()` -- reflected evidence is reported only while it has not changed since before the document held any declared value, and so cannot be a reflection of it. Anything else is withheld whole: origin-only URL, empty title. The baseline is seeded at install, before the document can reflect anything typed into it. - Every distinct withholding reason reaches `meta.json` and the CLI notice. Regression tests. Each fails at 7acd716 and passes here: - a password beginning with a common word leaves the URL, title, row identity, accessible name and an unrelated button id exact (was: silently rewritten) - a node-swapping input that reflects as you type no longer leaks the prefix the field stopped holding (was: emitted verbatim) - identity holding a declared value is withheld and marked (was: rewritten with no marker, so replay would compare against text the page never showed) - no event carries a URL or a title - the three prior reproductions stay closed - the stated limit -- text that predates the value -- is pinned The ambiguous multi-node replacement refusal, `input[type=password]` auto-detection, `stateFromPriorDeclaration` re-binding, and the screenshot mask are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(browser): re-pin the public artifact inventory after the claims update check_release_consistency.py pins a reviewed SHA-256 of every public artifact. The redaction redesign changed claims.yaml and its generated docs/verification.json, so those two hashes move. Reviewed: exactly those two paths change, none added, none removed. docs/VERIFICATION.md and docs/verification.json are regenerated with the timestamp string the validate-claims workflow uses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browser): prefer the definite withhold reason over the ambiguous one When a document holds both a short declared value and a long one, and a piece of identity text contains both, the operator should be told the text held a declared value, not that it could have held one by chance. Either result withholds the text; only the reason changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(browser): correct comments that still described the removed scrubber Several comments described a mechanism this branch deleted: a scrubbed URL on the event, values retained at scrub time, and a later document 'scrubbing' text. Nothing is scrubbed any more -- text is reported exactly or withheld. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browser): close three round-4 blockers and redact URLs by structure Round 4 judged the retention replacement sound and found three bounded defects, two of them pre-existing rather than regressions. It also found the whole-URL refusal too broad: a single-page application that routes after a login lost its URL evidence for the whole session. P1-A, pre-existing. A same-origin GET submit carries the declared value under the field's own NAME, because that is how an HTML form works. The results document seeded its baseline from the URL it loaded with -- which already held the value -- and Python stopped applying the cross-document rule as soon as that document held a declared value of its own, so the URL was reported. Closed by STRUCTURE, not by matching. The URL is now parsed instead of treated as one opaque string: * The origin and the PATH are reported. A path change is the single-page application case and is app structure, not operator input. This is the evidence the old refusal destroyed. * Every parameter NAME survives. The VALUE of a parameter whose name is a declared secret field name, or the name or id of any bound field (which includes every auto-detected input[type=password]), is dropped -- always, deterministically, with no reference to the value. Sentry and Datadog redact URLs the same way and for the same reason. * A parameter Flow cannot prove predates the moment the document first held a declared value loses only ITS value. The whole URL is not withheld for it. * Nothing is invented. A dropped value becomes empty. Flow removes characters; it never adds characters the page did not show. * A document born after some earlier document received a declared value has no trustworthy baseline, so it proves no parameter value. Python supplies that recording-wide fact, which the closure cannot know. The net, a DETECTION and never a rewrite: if the URL Flow is about to report still holds a value it can see, the whole URL is withheld and the operator is warned that the application put a declared secret into it -- an application defect that exists with or without Flow, as OWASP notes. The net runs on the path in both containment directions, so a page that writes the field into its path as the operator types cannot leave a segment behind. Matching is sound here and was not sound before: this function now runs only from Python at the settled boundary, so it needs no history. The title has no structure to exploit, so it keeps the unchanged-or-withhold rule plus the same net. P1-B, a regression against 7acd716. Deleting the commit points lost the ability to match a value after the page removes the field: an SPA wizard that replaces its form with a summary row leaked the value into the clicked row's identity. Commit-point retention is restored for ONE purpose -- deciding whether to WITHHOLD identity text -- and never for the URL, the title, or any rewrite. A spurious match can only withhold, which costs evidence and cannot corrupt or leak, so the rule that failed three reviews is not re-created. A commit is decided at the microtask checkpoint, so a controlled input that fires focusout on the node it just replaced commits no keystroke prefix. Across documents the same case is closed by name: a document recovers the value of any inbound query or fragment parameter whose NAME is declared, long enough to identify, and uses it to withhold identity text. P2-A, pre-existing. targetName() returned null for a bound secret field with no reason, so a declared field with an aria-label produced a silent null. It now states secret-field-name-not-read and is counted. Tests: nine new cases, each failing at ff5d71e and passing here. The four reviewer reproduction scripts all pass. tests/test_browser_attach.py: 74 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(browser): state the structured URL rule and what it still costs The reflected-evidence contract changed from a whole-URL refusal to a structural reduction, so README, docs/BROWSER_RECORDING.md and claims.yaml no longer described what the code does. They now state: origin and path reported, parameter names kept, declared-parameter values dropped by name, unproven values dropped one at a time, nothing invented, and the detection net with its OWASP warning. The costs and the stated residual are written down beside them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browser): apply the same proof to a bare URL fragment A fragment that names nothing -- `#section` rather than `#a=1` -- has no structure to reduce, so it was passed through unreduced and could carry a value the proof rule would have dropped from a named parameter. Treat the whole fragment as one unnamed value and apply the same proof. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browser): withhold a later document's URL; the path is not structural Round 5 reproduced a declared secret reaching events.jsonl through the URL PATH. An ordinary REST redirect answers a GET submit with `/results/<value>`. The new document is a fresh closure: no bound element holds the value, nothing was committed there, and no parameter NAME identifies a path segment, so the detection net is empty. `requireProof` never applied to the path, and round 4 had removed the Python backstop, so the literal was reported while meta.json told the operator the URL was protected. Regression against ff5d71e. Restore the cross-document rule. Only the FIRST document to hold a declared value reports its own reflected text; every later document's URL and title are withheld. A document that receives a value of its own is NOT exempt, because holding a value says nothing about whether it loaded with an earlier document's value in its path. This does not cost the single-page-application evidence the structural rule recovered, and there is now a test for that claim rather than an argument: history.pushState and history.replaceState do not build a new document, so the closure that held the value is the closure being sampled, and its route change is still reported exactly. The rule bites only on a real navigation, which is where the leak lives. Cross-document URL evidence IS lost again, and the docs say so. Second reproduction, same document: a scanner input that writes the value into the path and CLEARS its own field inside its own `input` handler. Nothing in the DOM holds the value by the time Python samples, and `change`/`focusout` then fire on an empty field, so every value source was empty. The capture-phase `input` listener runs BEFORE the page's handler, which is the last moment the DOM still holds the value, so it now records the last non-empty value per bound element -- ONE value, REPLACED on every keystroke, never a ladder, never used to rewrite. It applies only when nothing in the document holds a value any more, so a live field's value still wins and a detached node's keystroke prefix never withholds unrelated evidence. This also closes the reviewer's P3, where the page removes the field rather than clearing it. Also, one call: identityRefusal used liveSecretValues() while identityTextOrNull and labelTextOrNull used identityMatchValues(), so a results document refused the accessible name and the row identity but emitted `#row-<value>`. All identity paths now use the same set, which can only withhold more. Value comparison is now case-insensitive. Upper-casing an identifier before showing it is normalisation, not an application-defined transform, and widening a withhold test cannot leak. A dropped parameter is recorded only for a URL Flow actually reports, so meta.json never names a parameter of a URL that was withheld whole. Two dead bindings removed. Tests: four new cases failing at 07372d5 and passing here, plus one that must pass at BOTH heads because it proves the SPA evidence was not lost. All twelve reviewer round-5 cases pass; six of them fail at 07372d5. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browser): treat a page that consumes its own field as having held a value Round 6 reproduced four P1 leaks to events.jsonl, all silent, all present at 07372d5 as well. A scanner input that writes the badge into the URL and then clears its own field inside its own `input` handler holds nothing at any moment Python samples, and two rules keyed off what the DOM holds at that instant. CAUSE A -- the last-value cache was consulted only when NOTHING in the document held a value, while committed values were unioned unconditionally. There was no reason for the asymmetry. The test is now per element: a bound element that holds nothing right now contributes its cached value. It must stay per element rather than unconditional. A controlled input that swaps its node on every keystroke leaves detached elements still reporting `c`, `ch`, `cha`; unioning those in withheld any text containing a one-letter match and nulled `#chart-save` in three existing cases. A detached element is added only when no connected bound field holds anything, which covers a page that REMOVED its field while excluding the swap trail. That alone did not close F3: the cache holds ONE value per element, and a second scan displaced the first badge while the first was still in the URL. A value the next one does not CONTINUE was not edited away by the operator -- the page took it and started the field over -- so it is promoted into the withhold-only committed set, which is not per element. While the operator types, each value continues the one before it, so no keystroke prefix can be promoted. It is checked at the next input event, not at a microtask checkpoint after the current one: a checkpoint runs BETWEEN listeners, so it observes the field before the page's own handler has cleared it. CAUSE B -- `documentHeldSecretValue` was derived from live values, so it stayed false forever for a consuming page. The title net sits inside that flag while the URL net does not, and `state.secret` never became true, so Python never learned the document had received a value. It is now armed in the capture-phase `input` handler, the moment the document provably held one, which is what the documentation already said: "once a declared secret field RECEIVES INPUT". Python had the same asymmetry: `_track_secret_document` added to `_secret_doc_ids` from the input event but set `_first_secret_doc_id` only from the settled read, and the cross-document rule keys off the second. Both markers now move together through `_mark_secret_document`. Also corrected the cache comment: it is literally the last `input` event's value, not "the value the operator stopped on". Tests: four new cases, each failing at d1a762e and passing here. All round-4 and round-5 reviewer files pass (31). Eleven of thirteen round-6 reviewer cases pass; two assert the pre-fix behaviour and contradict F3/F4 -- see the PR body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

abrichr added 4 commits August 18, 2026 15:56

fix: require passing claim evidence

872fe84

fix: bind validating evidence to its substrate

1da2735

fix: make validating VM lifecycle fail closed

460cdb1

fix: recover interrupted Parallels validation

bf7ccd5

abrichr force-pushed the codex/claims-passing-evidence branch from 0da7b34 to bf7ccd5 Compare August 18, 2026 19:57

test: bound Windows lock release check

357df93

abrichr marked this pull request as ready for review August 18, 2026 20:29

abrichr merged commit e1a236c into main Aug 18, 2026
15 checks passed

abrichr deleted the codex/claims-passing-evidence branch August 18, 2026 20:30

This was referenced Aug 18, 2026

Harden remote display qualification evidence #365

Merged

fix: require Flow production release evidence #366

Merged

Navigation Menu

Uh oh!

fix: require passing evidence for supported claims - #371

fix: require passing evidence for supported claims#371
abrichr merged 5 commits into
mainfrom
codex/claims-passing-evidence

abrichr commented Aug 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

abrichr commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root cause

Change

Independent review fixes

Rebase

Local validation

Merge order

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

abrichr commented Aug 18, 2026 •

edited

Loading