Skip to content

[major] Export codegen emits select_option() with no arguments — recorded option value silently dropped (false-fails and false-passes) #151

Description

@tlemmonds

Summary

testmd export (Playwright/Python) generates select_option() with no arguments for a recorded dropdown selection, even though the recording carries the selected value. The generated line selects nothing, and the test silently derails from that point on.

Evidence

CLI 0.6.4, 2026-07-24. Recorded action (from actions.ndjson — note action_params.value is present and the selector is a clean testid):

{"action_type": "select",
 "action_instruction": "Selecting the sample applicant persona dropdown option",
 "selectors": [{"selector": "internal:testid=[data-testid=\"persona-select\"s]", "score": 100}],
 "action_params": {"value": "Dana Ortiz · Decline", "label": null, "index": null}}

Generated code for that action:

element_1 = page.locator("internal:testid=[data-testid=\"persona-select\"s]")
await element_1.select_option()   # <-- no value/label/index — selects nothing

Suspect: the non-ASCII · (U+00B7) in the option value trips the argument serialization.

Impact — silent cascade, both false-fails AND false-passes

In our lending AUT the dropdown loads an applicant persona (prefills every form field). With the selection dropped:

  • persona never loads → all fields stay empty → bank verification can't complete → the decision engine's first rule returns RFAI
  • a test expecting Decline fails with RFAI (reproduced on TMS cloud execution AND a local run of the same export — deterministic)
  • worse: tests that expect RFAI false-pass on the empty form — green results that validated nothing

Compounding issue (same export)

The decision extraction in the same generated file is re-derived as a brittle text-glob with no expected value:

locators=['internal:text="DecisionDeclineYour"i'], expected_value=''

even though the freshly authored recording carries internal:testid=[data-testid="decision-outcome"s] with expected_value: "Decline". Codegen discards the recorded anchors it should trust — same family as #150.

Ask

  1. Emit the recorded value/label/index in select_option(...) — and fail codegen loudly if none survives serialization, rather than emitting a no-op call.
  2. Add a codegen test with non-ASCII characters in option values.
  3. Prefer recorded testid locators and recorded expected values over re-derived text globs (see [major] Exported code has no wait before post-action extractions — author-green tests fail deterministically on fast executors #150).

Workaround

We added a /apply?persona=<id> deep-link testability hook to the application so TMS-executed tests avoid the <select> entirely — workable for us, not for teams who can't modify the AUT.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions