Secret (secret: true) leaks in cleartext to --agent NDJSON stdout and on-disk session artifacts

[komal-lt]

Summary

A variable marked secret: true is written in cleartext to both the --agent NDJSON stream (stdout) and on-disk session artifacts after a successful run, despite the documentation promising secrets are "masked in displayed output and logs." Only Result.md and the replayable output-<stem>/ cache are correctly redacted.

Severity

S2 — credential exposure. In CI, the --agent stdout is captured into build logs / uploaded artifacts, so a secret passed to a test leaks into CI logs.

kane-cli version

0.4.0

Repro

1. Define a secret in the auto-loaded store, e.g. ~/.testmuai/kaneai/variables/x.json:

   { "MY_PASSWORD": { "value": "<secret>", "secret": true } }

2. Reference it in a *_test.md step: ... type {{MY_PASSWORD}} into the Password field ...
3. Run to a successful completion: kane-cli testmd run login_test.md --agent --headless

Actual

The cleartext secret value appears in:

• stdout — the run_end NDJSON event's variables block:

  "variables": { "secrets.user.MY_PASSWORD": { "value": "<CLEARTEXT_SECRET>", "secret": true, ... } }

• ~/.testmuai/kaneai/sessions/<id>/execution.json (×1)
• ~/.testmuai/kaneai/sessions/<id>/runs/<n>/run-test/step_<NNN>.json (×2)

(Verified by grepping the artifacts for the literal value; reproduced consistently.)

Expected

A secret: true value must be masked/redacted everywhere it is emitted — the --agent NDJSON stream and all on-disk session artifacts — consistent with Result.md, which already redacts it to the {{MY_PASSWORD}} placeholder. The secret: true flag is present on the leaked entries, so the masking layer has the information it needs and is simply not applying it to these sinks.

Correctly redacted (for contrast)

• output-<stem>/Result.md — shows {{MY_PASSWORD}}, no cleartext. ✅
• output-<stem>/.internal/ replay cache — no cleartext. ✅ (safe to commit)

Impact

The documented CI workflow (--agent + secrets from CI store) leaks those secrets into CI logs and any uploaded NDJSON/session artifacts. Anyone running a credentialed test in CI is exposed. Related: #75, #76.

[komal-lt]

Adding a severity data point: this is much worse under a per-test self-login pattern.

When tests don't use a persisted chrome_profile (so they're stateless / CI-parallelizable), every test logs in on every run, so the cleartext secret: true password is written to ~/.testmuai/kaneai/sessions/* on every test of every run, not just a one-time seed.

Measured locally after one suite-development session: 132 session dirs, 406 MB, and 110 of them contain the plaintext password. On a CI runner this lands in the workspace and would be captured by any actions/upload-artifact of the sessions dir (the LambdaTest reference workflow uploads exactly that).

Mitigations a consumer is forced into today: dedicated throwaway account, rotate frequently, never upload session artifacts, and periodically purge ~/.testmuai/kaneai/sessions/*. A real fix (redact secret: true values in on-disk step artifacts, not just "displayed output") would remove all of that.