feat(episode): stop an episode when the game reports it is over

CHANGELOG.md

@@ -2,6 +2,32 @@
 
 All notable changes to Playproof are documented here.
 
+## Unreleased
+
+### An episode can end because the game ended
+
+- **The measurement.** A consumer running `ale-breakout` at `maxTurns: 300` counted **163 of 300 decisions (54.3%) taken after lives reached 0**, with the engine's own `terminal` flag set. The ALE worker breaks out of its action-repeat loop once that flag holds, so none of those inputs reached the emulator: the decisions were inert, not merely unproductive. The episode still reported 300 of 300 answered and looked healthy.
+- **Why the consumer could not fix it.** `playEpisode` had two stop conditions, the turn limit and the dollar budget, and no terminal concept in the published API. The only other exit was an abort through `signal`, which throws inside the decision loop before `finalizeRecord` and destroys the attestation the grade is made of.
+- `Game<S>` gains an OPTIONAL `over(state): boolean`. A game that omits it is never over, so every existing adapter keeps its behaviour with no edit. It must be pure like `step`, because a verifier recomputes the final state from the seed and the input log and asks again. There is no shared spelling of the terminal flag across substrates — ALE writes `terminal`, Gymnasium `terminated` and `truncated`, stable-retro `episodeDone`, the 2048 core `gameOver` — so the mapping belongs to the adapter, not to a guess the harness makes over field names. `adapters/ale`, `adapters/gymnasium`, `adapters/stable-retro`, `adapters/native-2048`, and the `screen-puzzle` fixture implement it over evidence they already published.
+- `playEpisode`, `runCampaign`, and `executeBenchmark` take `stopAtGameOver`. **It is off by default**: episode length is the denominator a study divides by, and rounds compare only while every round played to the same turn limit. A default that shortened episodes would retroactively break a running comparison, so arming the stop is the caller's decision, taken once for a series.
+- The stop is an exit from the decision loop, never a thrown abort. `finalizeRecord` runs, and the record verifies by replay exactly as a turn-limited record does.
+- `EpisodeRecord` gains `stoppedBy` and `gameOver`, so a reader of an artifact never infers why a run ended. `stoppedBy` is `maxTurns`, `budget`, `gameOver`, or — for a campaign — `steering` or `analyst`. `gameOver` is `true`, `false`, or `null` when the game declares no terminal state at all. Game over outranks the limits: a run that reaches its last allowed turn and a finished game at the same instant reports `gameOver`.
+- The two fields together state which mode produced a record. `stoppedBy: 'maxTurns'` next to `gameOver: true` can only come from a run played past the end, so the stop was not armed. Where the game never ended, the two modes produce the same length and the same record.
+- `CampaignStop` gains `gameOver`, and a campaign segment records it. A campaign resumed from a ledger whose game already ended plays no further segment and writes no empty one. A ledger written by 0.6.0 still loads; the ledger schema is unchanged.
+
+### Measured
+
+ALE Breakout, ale-py 0.12.1, seed 0, `maxTurns: 300`, one scripted policy that opens four milestones and then loses every life. Both runs are gates in `pnpm test:ale`.
+
+| Run | Decisions | `stoppedBy` | `gameOver` | Milestones | Replay |
+|---|---|---|---|---|---|
+| turn limit (today's behaviour) | 300 | `maxTurns` | `true` | 4 of 6 | clean |
+| `stopAtGameOver: true` | 150 | `gameOver` | `true` | 4 of 6 | clean |
+
+The 150 dropped decisions are inert: every evidence channel — screen hash, save-state hash, and engine state — is byte-identical from decision 150 to decision 300, while decision 149 to 150 did move the emulator.
+
+Gymnasium FrozenLake-v1, same shape on a second real substrate: 6 decisions instead of 26, 3 of 3 milestones either way, replay clean. It is a gate in `pnpm test:gym`.
+
 ## 0.6.0
 
 ### A hash check identifies a state, not a trajectory

README.md

@@ -121,6 +121,66 @@ const driver = createCliAgentDriver({
 
 The CLI driver spawns without a shell and bounds time and output. It can invoke Claude Code, Codex CLI, OpenCode, Pi, a local executable, or a container entrypoint. See [Agent drivers](docs/agent-drivers.md) and `examples/` for complete integrations, including a caller-supplied Tangle Agent Runtime backend.
 
+## When an episode ends
+
+An episode has three stop conditions, and the record names the one that fired.
+
+| `record.stoppedBy` | What happened |
+|---|---|
+| `maxTurns` | The turn limit was reached. |
+| `budget` | The dollar budget was reached. |
+| `gameOver` | The game reported that it is finished. |
+
+The game-over stop is opt-in.
+
+```ts
+const { record } = await playEpisode(game, contract, driver, budgetUsd, maxTurns, seed, signal, {
+  stopAtGameOver: true,
+})
+// record.stoppedBy === 'gameOver'
+// record.gameOver === true
+```
+
+`runCampaign` and `executeBenchmark` take the same `stopAtGameOver` option.
+
+It is off by default because episode length is a denominator.
+Rounds of one study compare only while every round played to the same turn limit, so shortening an episode is the caller's decision, taken once for a whole series.
+
+Every record also carries `gameOver`, armed or not.
+It is `true` when the game was finished at the last state of the run, `false` when it was not, and `null` when the game declares no terminal state at all.
+A record that reports `stoppedBy: 'maxTurns'` next to `gameOver: true` is a run that kept paying for decisions after the game ended.
+
+A game declares the end of play with the optional `over(state)` member.
+
+```ts
+const game: Game<MyState> = {
+  // ...
+  evidence: (s) => s.evidence,
+  over: (s) => s.evidence.engineState?.terminal === 1,
+}
+```
+
+`over` must be pure, like `step`, because a verifier recomputes the final state from the seed and the input log and asks again.
+A game that omits it is never over, so every adapter written before this member keeps its behaviour.
+`adapters/ale`, `adapters/gymnasium`, `adapters/stable-retro`, and `adapters/native-2048` implement it over the terminal flag their worker already publishes.
+
+The stop is an exit from the decision loop, not an abort.
+The attestation runs, and the record verifies by replay exactly as a turn-limited one does.
+Aborting through `signal` is a different thing: it throws inside the loop, before the record is built, and leaves nothing to grade.
+
+### Measured
+
+ALE Breakout, ale-py 0.12.1, seed 0, 300 turns, one scripted policy that opens four milestones and then loses every life:
+
+| Run | Decisions | `stoppedBy` | `gameOver` | Milestones |
+|---|---|---|---|---|
+| turn limit | 300 | `maxTurns` | `true` | 4 of 6 |
+| game-over stop | 150 | `gameOver` | `true` | 4 of 6 |
+
+The 150 dropped decisions are inert, not merely unproductive.
+The ALE worker breaks out of its action-repeat loop once the game is over, so every evidence channel is byte-identical from decision 150 to decision 300.
+Gymnasium FrozenLake behaves the same way: 6 decisions instead of 26, with 3 of 3 milestones either way.
+
 ## The observation channel: text always, pixels optional
 
 A game declares what the agent perceives. Text is always there. Images are opt-in.

adapters/ale.ts

@@ -298,6 +298,11 @@ export function makeAle(options: AleOptions): Ale {
       ? { text: s.frameText }
       : { text: s.frameText, images: [s.frameImage] }),
     evidence: (s) => s.evidence,
+    // The worker sets `terminal` when `ale.game_over()` first holds, and from
+    // that instant its step loop breaks out before it acts, so no further
+    // input reaches the emulator. Reading the flag the worker already
+    // publishes keeps one definition of the end of the game.
+    over: (s) => s.evidence.engineState?.terminal === 1,
   }
 
   try {

adapters/gymnasium.ts

@@ -240,6 +240,11 @@ export function makeGymnasium(options: GymnasiumOptions): Gymnasium {
     },
     frame: (s) => s.frameText,
     evidence: (s) => s.evidence,
+    // Gymnasium ends an episode two ways and the worker publishes both:
+    // `terminated` is the environment's own goal or failure, `truncated` is
+    // its time limit. The worker freezes the environment on either, so both
+    // mean no further input changes anything.
+    over: (s) => s.evidence.engineState?.terminated === 1 || s.evidence.engineState?.truncated === 1,
   }
 
   try {

adapters/native-2048.ts

@@ -184,6 +184,9 @@ export function makeNative2048(seed = 0): Native2048Adapter {
     },
     frame: (state) => state.frameText,
     evidence: (state) => state.evidence,
+    // The core publishes `gameOver` when no move can change the board, which
+    // is exactly the end of a 2048 game.
+    over: (state) => state.evidence.engineState?.gameOver === 1,
   }
   const reference = [...NATIVE_2048_REFERENCE]
   const contract = deriveContract(game, seed, reference, marks())

adapters/screen-puzzle.ts

@@ -34,6 +34,9 @@ export const screenPuzzle: Game<PuzzleState> = {
     engineState: { x: s.x },
     frameHash: hashString(render(s)),
   }),
+  // The far gate is the end of the puzzle: at the last column no input moves
+  // the piece any more, so the state machine is finished.
+  over: (s) => s.x >= PUZZLE_WIDTH - 1,
 }
 
 export const screenPuzzleContract = (): MilestoneContract =>

adapters/stable-retro.ts

@@ -287,6 +287,9 @@ export function makeStableRetro(options: StableRetroOptions): StableRetro {
       ? { text: s.frameText }
       : { text: s.frameText, images: [s.frameImage] }),
     evidence: (s) => s.evidence,
+    // The worker publishes `episodeDone` when the environment reported
+    // terminated or truncated, and stops acting from that point.
+    over: (s) => s.evidence.engineState?.episodeDone === 1,
   }
 
   try {

ale.test.mts

@@ -18,6 +18,7 @@ import { spawnSync } from 'node:child_process'
 import { createHash } from 'node:crypto'
 import { attestRun } from './attestation'
 import { assertContractSeparates, assertOpaqueChecksDeclared, calibrateContract } from './calibration'
+import { playEpisode, scriptedDriver } from './episode'
 import { logFrom, observationOf } from './runtime'
 import { decodePng, unscale } from './test-png.mts'
 import { contractLegibility, formatMilestoneScore, scoreMilestones, validateContract } from './schema'
@@ -313,6 +314,59 @@ if (!pythonHasAle()) {
       vision.dispose()
     }
 
+    // The game-over stop, on the real emulator.
+    //
+    // Motivation, measured by a consumer at 300 turns on this ROM: 163 of 300
+    // decisions (54.3%) were taken after lives reached 0 with `terminal` set.
+    // The worker breaks out of its action-repeat loop once that flag holds, so
+    // none of those inputs reached the emulator.
+    //
+    // The script reproduces the shape deterministically — 40 inputs of the
+    // reference, which open four milestones, then FIRE until the last life is
+    // gone. A regression in these turn counts means ale-py, the ROM, or the
+    // reference moved.
+    const GAME_OVER_TURNS = 300
+    const REFERENCE_PREFIX = 40
+    const dying = () => scriptedDriver([
+      ...adapter.reference.slice(0, REFERENCE_PREFIX),
+      ...Array.from({ length: GAME_OVER_TURNS - REFERENCE_PREFIX }, () => 'FIRE'),
+    ])
+    const fullLength = await playEpisode(adapter.game, adapter.contract, dying(), 1, GAME_OVER_TURNS, adapter.seed)
+    const atGameOver = await playEpisode(
+      adapter.game, adapter.contract, dying(), 1, GAME_OVER_TURNS, adapter.seed, undefined, { stopAtGameOver: true },
+    )
+    assert.equal(fullLength.record.turns, GAME_OVER_TURNS)
+    assert.equal(fullLength.record.stoppedBy, 'maxTurns')
+    assert.equal(fullLength.record.gameOver, true, 'the full-length run ended past a finished game')
+    assert.equal(atGameOver.record.turns, 150)
+    assert.equal(atGameOver.record.stoppedBy, 'gameOver')
+    assert.equal(atGameOver.record.gameOver, true)
+    // The milestone verdict is unchanged: the 150 dropped decisions bought
+    // nothing, on either the score or the two pinned hashes.
+    assert.deepEqual(atGameOver.record.verified, fullLength.record.verified)
+    assert.deepEqual(atGameOver.record.score, fullLength.record.score)
+    assert.equal(atGameOver.record.verified.length > 0, true)
+    // The short record verifies by replay exactly as the full-length one does.
+    for (const run of [atGameOver, fullLength]) {
+      assert.equal(run.record.verdict, 'clean')
+      assert.equal(run.record.replayDivergence, false)
+      const recomputed = attestRun(adapter.game, adapter.contract, adapter.seed, run.log, [...run.record.verified])
+      assert.equal(recomputed.verdict, 'clean', recomputed.reasons.join('; '))
+      assert.deepEqual(recomputed.verified, run.record.verified)
+    }
+    // The dropped decisions were inert, not merely unproductive: every
+    // evidence channel is byte-identical from the game-over snapshot to the
+    // 300th, while the decision before it did move the emulator.
+    const dyingTrace = trace(adapter, [...fullLength.log.inputs()])
+    assert.equal(dyingTrace[atGameOver.record.turns], dyingTrace[GAME_OVER_TURNS])
+    assert.notEqual(dyingTrace[atGameOver.record.turns - 1], dyingTrace[atGameOver.record.turns])
+    console.log(
+      `ale: game-over stop — ${atGameOver.record.turns} of ${GAME_OVER_TURNS} decisions played, ` +
+      `${fullLength.record.turns - atGameOver.record.turns} dropped as inert ` +
+      `(${Math.round((1 - atGameOver.record.turns / GAME_OVER_TURNS) * 1000) / 10}% of the episode); ` +
+      `milestones ${formatMilestoneScore(atGameOver.record.score)}, unchanged from the full-length run`,
+    )
+
     // Teardown: dispose kills the worker and every later call fails loudly
     // instead of silently reading a dead transport.
     second.dispose()

campaign.ts

@@ -26,9 +26,10 @@ import type {
   AgentDriver,
   AgentHistoryEntry,
   EpisodeRecord,
+  EpisodeStop,
   MilestoneCostRow,
 } from './episode'
-import { observationTextOf, type Game, type InputLog } from './runtime'
+import { isGameOver, observationTextOf, type Game, type InputLog } from './runtime'
 import { contractHash, scoreMilestones, type MilestoneContract, type MilestoneScore } from './schema'
 
 /**
@@ -47,6 +48,7 @@ export type CampaignStop =
   | 'segmentLimit'
   | 'maxTurns'
   | 'budget'
+  | 'gameOver'
   | 'steering'
   | 'analyst'
   | 'abort'
@@ -160,6 +162,14 @@ export interface CampaignOptions {
   steer?: (report: SegmentReport, analysis: Analysis | null) => Promise<Steering | null>
   /** Persistence hook. Save the ledger here so a killed process can resume. */
   onLedger?: (ledger: CampaignLedger) => void | Promise<void>
+  /**
+   * End the campaign as soon as the game declares itself over. Off by default,
+   * for the reason `EpisodeOptions.stopAtGameOver` states: episode length is a
+   * denominator, and shortening it is the caller's decision.
+   *
+   * A resumed campaign whose game already ended plays no further segment.
+   */
+  stopAtGameOver?: boolean
   signal?: AbortSignal
 }
 
@@ -207,8 +217,18 @@ export async function runCampaign<S>(
 
   let guidance = resumeGuidance(ledger)
   let stopped = false
+  // Why the whole campaign ended, once it is known. Null while it is still the
+  // limits that decide, which the tail of this function reads off the rollout.
+  let endedBy: EpisodeStop | null = null
   while (!stopped && rollout.turns < maxTurns && rollout.spent < budgetUsd) {
     options.signal?.throwIfAborted()
+    // A campaign resumed from a ledger whose game already ended starts no new
+    // segment and records none. An empty segment would report a boundary the
+    // run never played to.
+    if (options.stopAtGameOver === true && isGameOver(game, rollout.state)) {
+      endedBy = 'gameOver'
+      break
+    }
     const segment = ledger.segments.length
     const startTurn = rollout.turns + 1
     const guidanceInEffect = guidance
@@ -225,6 +245,7 @@ export async function runCampaign<S>(
         maxTurns,
         maxDecisions: segmentTurns,
         ...(guidance === undefined ? {} : { guidance }),
+        ...(options.stopAtGameOver === undefined ? {} : { stopAtGameOver: options.stopAtGameOver }),
         ...(options.signal === undefined ? {} : { signal: options.signal }),
       })
     } catch (error) {
@@ -301,12 +322,19 @@ export async function runCampaign<S>(
         }
         // Explicit steering outranks the analyst; a stop from either ends the run.
         guidance = nextGuidance(guidance, steering, analysis)
-        stopped = steering?.stop === true || analysis?.recommendation === 'stop'
+        const hookStop = steering?.stop === true
+          ? 'steering'
+          : analysis?.recommendation === 'stop' ? 'analyst' : null
         // A hard limit that already ended the segment keeps its own reason.
-        if (stopped && stoppedBy === 'segmentLimit') {
-          stoppedBy = steering?.stop === true ? 'steering' : 'analyst'
-          segmentRecord.stoppedBy = stoppedBy
+        if (hookStop !== null && stoppedBy === 'segmentLimit') {
+          stoppedBy = hookStop
+          segmentRecord.stoppedBy = hookStop
         }
+        // A finished game ends the campaign whatever a hook advises: there is
+        // nothing left to play, and no note can restart it.
+        if (stoppedBy === 'gameOver') endedBy = 'gameOver'
+        else if (hookStop !== null) endedBy = hookStop
+        stopped = endedBy !== null
       }
     } finally {
       ledger.updatedAt = new Date().toISOString()
@@ -315,7 +343,9 @@ export async function runCampaign<S>(
     if (abortError !== undefined) throw abortError
   }
 
-  const record = finalizeRecord(game, contract, seed, rollout, budgetUsd, started)
+  // The record states why the CAMPAIGN ended, not why its last segment paused.
+  const runStop: EpisodeStop = endedBy ?? (rollout.turns >= maxTurns ? 'maxTurns' : 'budget')
+  const record = finalizeRecord(game, contract, seed, rollout, budgetUsd, started, runStop)
   syncLedger(ledger, rollout)
   // Replay-verified progress supersedes the live tracker once the run is over.
   ledger.verified = [...record.verified]
@@ -479,6 +509,7 @@ function isStop(value: unknown): value is CampaignStop {
   return value === 'segmentLimit'
     || value === 'maxTurns'
     || value === 'budget'
+    || value === 'gameOver'
     || value === 'steering'
     || value === 'analyst'
     || value === 'abort'

docs/adapters.md

@@ -24,6 +24,30 @@ Read the mode column strictly.
 `trusted-recorder` means a named recorder signed what it captured and nothing was independently reproduced.
 `platform-attested` means a signed recorder read normalized progress from a platform API; it is not a signature from that platform.
 
+## Which adapters declare the end of play
+
+`Game.over(state)` is optional, and it is what an episode reads when the caller
+passes `stopAtGameOver`. A game that omits it is never over.
+
+| Adapter | `over(state)` reads |
+|---|---|
+| `adapters/ale` | `engineState.terminal`, set when `ale.game_over()` first holds |
+| `adapters/gymnasium` | `engineState.terminated` or `engineState.truncated` |
+| `adapters/stable-retro` | `engineState.episodeDone` |
+| `adapters/native-2048` | `engineState.gameOver`, set when no move changes the board |
+| `adapters/pyboy-generic`, `adapters/pyboy-tetris`, `adapters/retroarch` | Nothing; these workers publish no terminal flag, so the episode runs to a limit |
+| `platforms/steam`, `platforms/xbox` | Nothing; the title runs elsewhere |
+
+Each adapter maps its own substrate, because there is no shared spelling of the
+flag. It must be pure, like `step`: a verifier recomputes the final state from
+the seed and the input log and asks the same question.
+
+Every emulator worker here already stops acting once its own terminal flag is
+set, so the decisions an episode takes after that point change nothing at all.
+Measured on ALE Breakout at 300 turns: from the game-over decision onward the
+screen hash, the save-state hash, and the engine state are byte-identical to
+the end of the run.
+
 ## The observation image channel
 
 Every emulator adapter here was already capturing the screen for evidence and then showing the agent an ASCII downsample of it.