feat(calibration): separate earnable milestones from replay-identity checks

CHANGELOG.md

@@ -2,6 +2,41 @@
 
 All notable changes to Playproof are documented here.
 
+## 0.6.0
+
+### Earnable milestones are separated from replay-identity checks
+
+- **The defect.** `deriveContract` samples frame hashes and save hashes from the reference trajectory and emits them as milestones. On ALE Breakout the derived contract has six milestones, and two of them — `frame-at-first-score` and `save-at-first-score` — pin the exact bytes the reference produced. No independent policy earns them. An authored policy that verified `score-opened`, `score-tier-2`, and `life-lost` read as 3 of 6; a hand-written ball tracker reached the same three. A third of that contract's points were reachable only by a replay of the reference, and every score quoted from it carried that denominator.
+- A milestone's role is now derived from its check kind. `save-hash` and `frame-hash` are replay-identity; `state-path`, `save-path`, `frame-path`, and `log-contains` are achievements. Nothing is stored on the milestone, so every existing contract keeps its bytes and its hash, and no author can forget to set it.
+- `contractEarnability(contract)` follows `requires` as well. An achievement gated behind a hash is unreachable too, because `MilestoneTracker` admits a milestone only after every prerequisite has passed. The packaged `save-levels` contract is the case in the repo: its `log-contains` milestone is semantic and still unearnable, through its save-hash prerequisite.
+- `Attestation` and `EpisodeRecord` keep `verified` unchanged and gain `earned` and `score`. `MilestoneScore` is `{ verified, earned, earnable, total }`, and `formatMilestoneScore` writes it as `3 of 4 earnable (3 of 6 verified, 2 replay-identity)`. A campaign segment report gains `scoreSoFar`, so an analyst reads progress against the earnable denominator mid-run.
+- The campaign ledger is unchanged. Its `verified` list and the contract it pins by hash reproduce the score through `scoreMilestones`, so a ledger written by 0.5.0 still loads.
+
+### Calibration refuses a contract with points no policy can score
+
+- **A second defect, in the gate itself.** A replay-identity milestone is never earned by a trivial baseline, so it landed in `separating` and the separation test read it as the contract's strongest evidence. Measured on 0.5.0: a contract whose two milestones are hashes over the whole input chain reports `separates: true` with every baseline earning nothing. Unreachable read as hard.
+- `CalibrationReport.separating` now holds earnable milestones only, and `separates` compares earnable counts through the new `bestBaselineEarnedCount`. A baseline that beats the reference on real progress can no longer be outvoted by hashes the reference reproduces by construction.
+- The report gains `earnable`, `unearnable`, `unearnableReasons`, `unearnableReproduced`, and `referenceScore`.
+- `assertContractSeparates(report, { identityChecks })` takes an exact declaration of the identity checks the author accepts. An undeclared one, a stale id, and an identity hash that a trivial baseline reproduced all fail the gate, so a hash milestone added by a later derivation cannot enter a published contract unnoticed. `assertMilestonesEarnable` runs the same check alone, for a target that is not meant to separate.
+- Identity checks are not removed and not discouraged. Replay attestation is exactly the claim that one run reproduced another's bytes. The fix is to stop counting them as achievements.
+
+### Measured
+
+| Contract | Milestones | Earnable | Reference | Best trivial baseline |
+|---|---|---|---|---|
+| ALE Breakout, 210 turns, seed 0 | 6 | 4 | 4 of 4 earnable (6 of 6 verified) | 0 earnable |
+| Libbet through `pyboy-generic`, 70 turns, seed 0 | 6 | 4 | 3 of 4 earnable (3 of 6 verified) | 3 earnable |
+| `save-levels` toy | 2 | 0 | 0 of 0 earnable (2 of 2 verified) | 0 earnable |
+| `screen-puzzle` toy | 2 | 0 | 0 of 0 earnable (2 of 2 verified) | 2 verified |
+
+- Breakout separates on its earnable milestones and its scores were quoted out of the wrong denominator. Libbet still does not separate, and its four earnable milestones are exactly the ones a constant `a` press already earns.
+- `screen-puzzle` renders from one coordinate, so `constant:r` walks to the same square and reproduces both pinned frames. A hash another trajectory reproduces identifies no run, and the gate now says so.
+
+### Replay attestation is unaffected
+
+- `verified` keeps its meaning and its contents. The three packaged toy contract hashes are byte-identical across the change, and `calibration.test.mts` pins them, together with the serialized milestone key set, so a contract that gains a field fails the build.
+- `ale.test.mts` and `pyboy-libbet.test.mts` verify the same milestone ids on the same runs as before, and now also report the split.
+
 ## 0.5.0
 
 ### The observation channel

README.md

@@ -270,6 +270,33 @@ Use semantic checks such as `score >= 10` for progression. Exact hashes identify
 
 Dependencies between milestones form a declared partial order. A later achievement cannot verify before its prerequisites, even when its raw condition already holds.
 
+### Earnable milestones and replay-identity checks
+
+A contract holds two kinds of statement, and one score cannot carry both.
+
+- An **achievement** is a threshold, a normalized field, or an event. Two different valid trajectories both satisfy it, so an independent policy earns it by playing.
+- A **replay-identity check** is a hash over the exact bytes one recorded run produced. It proves that a replay reproduced that run, and no independent policy earns it.
+
+The role is derived from the check kind, so no contract changes and no author has to remember to set it.
+`save-hash` and `frame-hash` are identity; `state-path`, `save-path`, `frame-path`, and `log-contains` are achievements.
+`requires` is followed: an achievement gated behind a hash is unreachable too, because a milestone verifies only after every prerequisite has.
+
+```ts
+import { contractEarnability, formatMilestoneScore } from '@tangle-network/playproof'
+
+contractEarnability(contract)
+// { earnable: ['score-opened', 'score-tier-2', 'score-tier-4', 'life-lost'],
+//   unearnable: ['frame-at-first-score', 'save-at-first-score'],
+//   reasons: { 'frame-at-first-score': "its frame-hash check pins the reference run's exact bytes", … } }
+
+formatMilestoneScore(record.score) // '3 of 4 earnable (3 of 6 verified, 2 replay-identity)'
+```
+
+`Attestation` and `EpisodeRecord` carry `verified` unchanged, plus `earned` and `score`.
+A campaign segment report carries `scoreSoFar`.
+Report a run as earned over earnable.
+Identity checks stay in the contract and stay in `verified`; they are what replay attestation proves.
+
 ## Calibration: does the contract separate?
 
 A milestone contract says which progressions count.
@@ -292,10 +319,38 @@ assertContractSeparates(report)
 `calibrateContract` replays the reference and a suite of trivial policies through the same attestation path: one constant policy per input word, a word the game cannot interpret, a round-robin cycle over the vocabulary, and a seeded pseudo-random walk over it.
 Every policy is deterministic in the seed, so a report reproduces from one number.
 
-The report names `separating` (milestones no baseline earned), `trivial` (milestones at least one baseline earned), and `bestBaselineCount`.
-`separates` is true only when something is out of reach of every baseline **and** the reference verifies strictly more milestones than the strongest baseline.
+The report names `separating` (earnable milestones no baseline earned), `trivial` (milestones at least one baseline earned), `earnable` and `unearnable`, and both baseline counts.
+`separates` is true only when an **earnable** milestone is out of reach of every baseline **and** the reference earns strictly more earnable milestones than the strongest baseline.
 `assertContractSeparates` throws otherwise, and the message names every trivial milestone with the baseline that earned it.
 
+### The gate also refuses points no policy can score
+
+A replay-identity check is never earned by a baseline, so the separation test alone reads it as the contract's strongest evidence.
+It is the opposite: a milestone out of reach of every policy measures nothing, and it inflates the denominator of every score quoted from the contract.
+
+Measured on ALE Breakout: the derived contract has six milestones and two of them are hashes of the screen and the save state at the first point.
+An authored policy that verified `score-opened`, `score-tier-2`, and `life-lost` read as 3 of 6.
+It could never have reached 6.
+The honest number is 3 of 4.
+
+```ts
+assertContractSeparates(report, {
+  identityChecks: ['frame-at-first-score', 'save-at-first-score'],
+})
+```
+
+The declaration is an exact set, not a switch: an undeclared hash milestone, a stale id, and an identity hash that a trivial baseline reproduced all fail the gate.
+A hash milestone that a later derivation adds therefore cannot enter a published contract unnoticed.
+`assertMilestonesEarnable` runs the same check alone, for a demonstration target that is not meant to separate.
+
+| Contract | Milestones | Earnable | Reference score | Best trivial baseline |
+|---|---|---|---|---|
+| ALE Breakout | 6 | 4 | 4 of 4 earnable | 0 earnable |
+| Libbet through `pyboy-generic` | 6 | 4 | 3 of 4 earnable | 3 earnable |
+
+Breakout separates and its score was quoted out of the wrong denominator.
+Libbet does not separate, and its earnable milestones are exactly the ones a constant button press already earns.
+
 ### The measurement that made this exist
 
 A live agent campaign ran 70 turns on Libbet and the Magic Floor through `adapters/pyboy-generic` and the packaged `pyboy/discovery-libbet.json` blind-discovery document.

ale.test.mts

@@ -7,18 +7,19 @@
  * into a loud failure (that is how CI proves the job really executed).
  *
  * Battery: contract derivation across three evidence tiers, known-good
- * attestation, garbage rejection, graded partial credit, cross-process
- * determinism including the save-state hash, checkpoint round-trip,
- * unknown-input no-op, the observation image channel, and worker teardown.
- * Zero model spend.
+ * attestation, garbage rejection, graded partial credit, calibration with the
+ * earnable split, cross-process determinism including the save-state hash,
+ * checkpoint round-trip, unknown-input no-op, the observation image channel,
+ * and worker teardown. Zero model spend.
  */
 import { strict as assert } from 'node:assert'
 import { spawnSync } from 'node:child_process'
 import { createHash } from 'node:crypto'
 import { attestRun } from './attestation'
+import { assertContractSeparates, assertMilestonesEarnable, calibrateContract } from './calibration'
 import { logFrom, observationOf } from './runtime'
 import { decodePng, unscale } from './test-png.mts'
-import { validateContract } from './schema'
+import { contractEarnability, formatMilestoneScore, scoreMilestones, validateContract } from './schema'
 import { AleRpc } from './adapters/ale-rpc'
 import { bundledReference, makeAle, type Ale, type AleState } from './adapters/ale'
 
@@ -109,6 +110,66 @@ if (!pythonHasAle()) {
     assert.deepEqual(partial.verified, ['score-opened', 'frame-at-first-score', 'save-at-first-score', 'score-tier-2'])
     assert.deepEqual(partial.reasons, ['claimed-not-reproduced:score-tier-4,life-lost'])
 
+    // The reference verified all six milestones, and only four of them are
+    // progress. `frame-at-first-score` and `save-at-first-score` hash the exact
+    // screen and save state this trajectory produced, so reaching them means
+    // reproducing this run rather than playing Breakout.
+    assert.deepEqual(contractEarnability(adapter.contract), {
+      earnable: ['score-opened', 'score-tier-2', 'score-tier-4', 'life-lost'],
+      unearnable: ['frame-at-first-score', 'save-at-first-score'],
+      reasons: {
+        'frame-at-first-score': "its frame-hash check pins the reference run's exact bytes",
+        'save-at-first-score': "its save-hash check pins the reference run's exact bytes",
+      },
+    })
+    assert.deepEqual(good.earned, ['score-opened', 'score-tier-2', 'score-tier-4', 'life-lost'])
+    assert.deepEqual(good.score, { verified: 6, earned: 4, earnable: 4, total: 6 })
+    assert.equal(formatMilestoneScore(partial.score), '2 of 4 earnable (4 of 6 verified, 2 replay-identity)')
+
+    // The measured defect this split exists for. An authored policy and a
+    // hand-written ball tracker each verified score-opened, score-tier-2 and
+    // life-lost on this contract, and neither reproduced either hash. Both
+    // read as three of six. Three of four is the honest statement, and it is
+    // the whole of what an independent policy can score.
+    const played = ['score-opened', 'score-tier-2', 'life-lost']
+    assert.deepEqual(scoreMilestones(adapter.contract, played), { verified: 3, earned: 3, earnable: 4, total: 6 })
+    assert.equal(formatMilestoneScore(scoreMilestones(adapter.contract, played)),
+      '3 of 4 earnable (3 of 6 verified, 2 replay-identity)')
+
+    // Calibration on the real emulator. Every baseline plays the reference's
+    // 210 turns, so the comparison is length-matched.
+    const calibration = calibrateContract(adapter.game, adapter.contract, {
+      reference: adapter.reference,
+      vocabulary: adapter.inputs,
+      seed: adapter.seed,
+    })
+    for (const outcome of [calibration.reference, ...calibration.baselines]) {
+      console.log(`  ${outcome.id.padEnd(32)} ${String(outcome.verified.length).padStart(2)}  ${outcome.verdict}  ${outcome.verified.join(',') || '-'}`)
+    }
+    assert.deepEqual(calibration.earnable, ['score-opened', 'score-tier-2', 'score-tier-4', 'life-lost'])
+    assert.deepEqual(calibration.unearnable, ['frame-at-first-score', 'save-at-first-score'])
+    assert.deepEqual(calibration.referenceScore, { verified: 6, earned: 4, earnable: 4, total: 6 })
+    // No trivial policy reproduced either hash, so both really are identity
+    // checks on this substrate and not low-entropy channels.
+    assert.deepEqual(calibration.unearnableReproduced, [])
+    // Undeclared, the contract cannot ship, whatever the separation verdict.
+    assert.throws(() => assertMilestonesEarnable(calibration), /2 of 6 milestone\(s\) no policy can earn/u)
+    const declared = { identityChecks: ['frame-at-first-score', 'save-at-first-score'] }
+    assertMilestonesEarnable(calibration, declared)
+    // Whether Breakout separates is a fact about the ROM, not about this
+    // change: assert only that the earnable comparison is the one being made.
+    assert.equal(
+      calibration.separates,
+      calibration.separating.length > 0 && calibration.referenceScore.earned > calibration.bestBaselineEarnedCount,
+    )
+    if (calibration.separates) assertContractSeparates(calibration, declared)
+    else assert.throws(() => assertContractSeparates(calibration, declared), /does not separate/u)
+    console.log(
+      `ale: calibration — reference ${formatMilestoneScore(calibration.referenceScore)}, ` +
+      `best trivial baseline ${calibration.bestBaselineEarnedCount} earnable over ${calibration.turns} turns, ` +
+      `separating=${calibration.separating.join(',') || 'nothing'}, separates=${calibration.separates}`,
+    )
+
     // Determinism: two replays in this worker and one in a freshly spawned
     // worker must agree on every frame hash, every save-state hash, and every
     // privileged variable. Cross-process is the load-bearing case, because a

attestation.ts

@@ -8,14 +8,24 @@
  */
 import { logFrom } from './runtime'
 import type { Evidence, Game, Input, InputLog } from './runtime'
-import { contractHash } from './schema'
-import type { Milestone, MilestoneContract, NumericOperator } from './schema'
+import { contractHash, earnedMilestones, scoreMilestones } from './schema'
+import type { Milestone, MilestoneContract, MilestoneScore, NumericOperator } from './schema'
 
 export interface Attestation {
   gameId: string
   verdict: 'clean' | 'rejected'
   reasons: string[]
+  /** Every milestone the replay reproduced, replay-identity checks included. */
   verified: string[]
+  /**
+   * The subset of `verified` an independent policy can earn by playing.
+   *
+   * A hash check proves a replay reproduced the recorded run. Counting it as
+   * progress inflates the denominator of every reported score, so the two sets
+   * are reported apart. `verified` stays the attestation statement.
+   */
+  earned: string[]
+  score: MilestoneScore
   checks: { name: string; passed: boolean }[]
 }
 
@@ -109,6 +119,8 @@ export function attestRun<S>(
     verdict: reasons.length === 0 ? 'clean' : 'rejected',
     reasons,
     verified,
+    earned: earnedMilestones(contract, verified),
+    score: scoreMilestones(contract, verified),
     checks: [
       { name: 'input-log-chain', passed: chainOk },
       { name: 'claimed-milestones-reproduced', passed: notReproduced.length === 0 && unknownClaims.length === 0 },