Prefer the real mutation tool, and make a hand-rolled runner prove it ran

skills/old-coder/SKILL.md

@@ -109,7 +109,7 @@ or a tool is unavailable, record that in the evidence report with the reason.
 | Static types | whole classes of bugs | tsc / mypy / etc., zero new errors |
 | Lint + format | latent bugs, drift | project's linter, zero new warnings |
 | Coverage on changed lines | untested code paths | every changed/added line executed by a test; branch coverage where the tool supports it. Global % is vanity — changed-line coverage is the constraint. **This layer must exit nonzero when its threshold is missed** (`--cov-fail-under`, `diff-cover --fail-under`, equivalent): a layer that prints a percentage and exits 0 is a report, not a gauntlet layer, and it will sit there green while coverage falls |
-| Mutation testing | tests that assert nothing | see `references/gauntlet.md`. No mutation tool? Do manual mutation: introduce 3–5 plausible bugs into the new code one at a time (flip a comparison, off-by-one a bound, drop a condition, return early); the suite must kill every one. Restore after |
+| Mutation testing | tests that assert nothing | **prefer the project's mutation tool** (mutmut, cosmic-ray, Stryker, PIT…), which generates mutants from the syntax tree and cannot silently skip one. No tool available? Manual mutation, per `references/gauntlet.md` — introduce 3–5 plausible bugs one at a time; the suite must kill every one; restore after. A hand-rolled runner must **prove it executed each mutant**: a runner that can report a kill it never ran inflates the score and no red gauntlet will ever surface it |
 | Property-based tests | edge cases you didn't imagine | for parsing, math, serialization, anything with invariants (round-trip, idempotence, ordering) — add hypothesis/fast-check properties |
 | Complexity budget | unmaintainable output | new functions small and single-purpose; if a function needs a paragraph to explain, split it |
 | Real execution | "passes tests, doesn't run" | actually run the app/CLI/endpoint once on a realistic input, not only the test harness |

skills/old-coder/references/gauntlet.md

@@ -120,6 +120,23 @@ dependency diff so the human can see exactly what the agent pulled in.
 
 ## Manual mutation procedure (any language, no tool)
 
+**Reach for the project's mutation tool first.** A real tool generates mutants
+from the syntax tree, so it cannot apply a mutant to code that has moved and it
+cannot report a mutant it did not run. A hand-written mutant list matched
+against source text is a second copy of the code: it goes stale on every
+refactor of the thing it guards, and it fails in the one direction no gauntlet
+can catch. Use the procedure below when no tool exists for the language, not as
+a default.
+
+**A hand-rolled runner must prove it executed each mutant.** This is the sharp
+edge, and this repo's own demo found it: two same-size mutants written in the
+same second shared a bytecode cache, so the runner reported kills for mutants it
+never executed. That class of defect can *only* inflate the score, which means
+it can never surface as a red gauntlet — the layer stays green precisely because
+it is broken. `tools/mutants.py` now guards against it (mtime pinning, a cache
+check that aborts the run); any runner written from this procedure needs the
+equivalent, and EVIDENCE should say which check proves execution.
+
 Script this rather than hand-editing, and **persist the script in the repo**
 (e.g. `tools/mutants.py`): it holds the original source, applies each mutant by
 unique string replacement, runs the suite, and restores. Hand-editing N times