Skip to content

Fail closed on unsafe incremental resumes - #57

Open
davanstrien wants to merge 1 commit into
fix/split-end-to-endfrom
fix/safe-resume-provenance
Open

Fail closed on unsafe incremental resumes#57
davanstrien wants to merge 1 commit into
fix/split-end-to-endfrom
fix/safe-resume-provenance

Conversation

@davanstrien

Copy link
Copy Markdown
Owner

Summary

  • Fingerprint the sampled pages and each OCR column, then stamp every comparison and checkpoint with per-pair provenance.
  • Include source repo, config and split, raw judge backend identity, sampling, prompt, preprocessing, token cap, and minimum text threshold in resume identity.
  • Reuse unchanged old-model pairs when a model is added, but require --full-rejudge for legacy unstamped or mismatched current-grid rows.
  • Make stamped checkpoint rows authoritative over stale completed-run metadata after an interrupted full rejudge.
  • Publish only one-way judge-spec hashes so endpoint credentials and private URLs are not exposed.

Compatibility

Legacy result repositories fail closed once and can be upgraded with --full-rejudge. Historical comparisons outside the current rankable grid remain preserved.

Stack

This is stacked on #56 because source split is part of the provenance contract. Change the base to main after #56 merges.

Validation

  • 632 tests passed
  • Ruff passed on src and tests
  • ty passed on src
  • git diff --check passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant