[major] Exported code has no wait before post-action extractions — author-green tests fail deterministically on fast executors

[tlemmonds]

Summary

testmd export (Playwright/Python) emits no wait between a triggering action and the extraction that follows it. At author time the agent's own think-time masks the app's response latency, so no wait is ever captured in the recording. On a fast executor (HyperExecute VM) the exported code clicks, then reads the DOM in the same second — before the app has rendered — the extraction returns None, and the assertion fails.

The tests are author-green and TMS-execution-green; only the exported code fails, deterministically, on fast infrastructure.

Evidence

CLI 0.6.4, 2026-07-23. Four auth tests exported and run on HyperExecute (job aa584d5f-5917-487c-9299-c30a2facdaad): 3 of 4 failed, all the same way.

Example (wrong-password login test) — click and extraction in the same second, extraction returns None:

23:08:10   [STEP 8] Clicking the Log in button
23:08:10   [STEP 9] Extract the visible error alert text ...
23:08:10     [set_var] name='error_element_text' value=None
23:08:10   [STEP 10 FAIL] Assertion failed: Assert the error element (data-testid `error`) contains the text `Invalid`

The correct recorded value is sitting right in the exported assertion payload ('variable_refs': {'{{error_element_text}}': 'Invalid username or password.'}) — the app behaved correctly; the export just read too early.

The exported extraction is an instant JS evaluation:

async with testmu.step('Clicking the Log in button', ...):
    await _loc_1.click()

async with testmu.step('Extract the visible error alert text ...'):
    set_var('error_element_text', await testmu.textual_analyzer(page, wrapped_js="(els) => {...}", locators=['internal:role=main'], ...))

The one test of the four that passed did so only because its recording happened to contain a fixed wait_for_timeout(500) — luck, not determinism.

Related codegen issues observed in the same exports

1. Extraction downgraded to vision_query — one status read (data-testid "account-status-C3003", named explicitly in the prose) was exported as a screenshot vision_query instead of a DOM read. Same open-vocabulary re-read flake class as cloud re-evaluation of recorded checkpoints.
2. Brittle positional locator — another extraction used tr:nth-child(17) > td:nth-child(4) even though the prose named a data-testid. Any row-count change breaks it.
3. Dead pre-action until-loops — the export contained three recorded until-loops polling for a 'Disabled' status before the Disable button is clicked (author-time planner-reordering artifact). They can never succeed and burn ~25 check_until_condition AI calls and ~50s per run.

Workaround (verified)

Hand-patching the exports with testid-anchored auto-waiting locators fixed everything — the re-run (job 83010776-e3bc-445c-8b16-54fa76ededd1) passed 4/4:

_err_el = page.locator("internal:testid=[data-testid=\"error\"s]")
await _err_el.wait_for(state="visible", timeout=10000)
set_var('error_element_text', (await _err_el.inner_text()).strip())

But patches are destroyed by the next re-export, so this doesn't scale.

Ask

1. Codegen should anchor extractions to their target element with Playwright's auto-waiting locator API (locator(...).inner_text() / wait_for) instead of instant wrapped_js evaluation over pre-resolved elements — especially when the prose names a data-testid.
2. When the prose names a data-testid, never emit vision_query or positional (nth-child) locators for it.
3. Strip until-loops that poll for a state produced by a later action (they are provably dead code in the recording's own ordering).