Backfill: re-run PCA publish for both SOTUs with structured provenance#40
Merged
Conversation
Regenerates site-pca via a fresh live `truthbot publish --engine pca` over Trump 2026 + Biden 2022, so the published reports carry the VerdictProvenance + reconciled-judge render shipped in PR-E (6850da1) — the 7/16 site predated persistence and was provenance-less. Adds the two replay artifacts (metrics/pca_runs/<run_id>.json) holding the raw per-seat panel votes for all 276 claims — the vote data that was previously unrecoverable, which unblocks the strength_from_votes recalibration. Results (dev roster, open-book Brave+FactCheck, CRM-114): - Trump 2026: 166/722 check-worthy — True66/Misleading53/False18/Unverifiable23, 6 non-converged; 49 panel-split claims now surfaced (were collapsed before). - Biden 2022: 110/480 check-worthy — True90/Misleading7/Unverifiable11, 2 non-converged. (Biden still skews True — known severity-softening, separate.) Note: pipeline self-reported cost ($0.163 + $0.062) is a ~7-8x undercount of real proxy spend (~$1.83 total this run). Cost-telemetry fix is a follow-up. Data-only change; suite 944 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
This was referenced Jul 19, 2026
aRealGem
pushed a commit
that referenced
this pull request
Jul 19, 2026
…ong to PR #40 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aRealGem
added a commit
that referenced
this pull request
Jul 19, 2026
… HOLD for jackie (#45) * data(gold): +14 FALSE/MISLEADING boundary rows, 21 -> 35 (P67 Phase 2) Targeted expansion to make the severity-softening fix measurable: FALSE 4 -> 11, MISLEADING 6 -> 11, TRUE 10 -> 11, UNVERIFIABLE 1 -> 2. Rows selected on the FALSE/MISLEADING boundary (absolute quantifiers, superlatives, exaggerated numbers) per the Phase-1 diagnosis; every decidable row is anchored to published fact-checks / primary statistics researched live (AP, FactCheck.org, PolitiFact, Poynter, WGBH, BEA, DIA via FactCheck.org, Lead Stories, The Assembly NC). Claim text verbatim from claim_set.train.jsonl; TRAIN-only (I6-safe); suite 944 green. All new rows needs_review=true (annotator claude-expand-p67) — verdict judgments held for jackie's adjudication per the no-auto-merge rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: drop metrics/pca_runs artifacts accidentally staged — they belong to PR #40 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cass (Claude Code agent) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Regenerates
site-pcavia a fresh livetruthbot publish --engine pcaover Trump 2026 + Biden 2022, so the published reports carry theVerdictProvenance+ reconciled-judge render shipped in PR-E (6850da1). The 7/16 site predated persistence and was provenance-less; this backfills it and generates real vote data.What changed
site-pca/regenerated — both reports now render the provenance strip (Layer A → PCA panel: Misleading ×2 → CRM-114: MISLEADING→FALSE), and genuinely-unconverged split claims showNo single verdictinstead of rendering blank.metrics/pca_runs/<run_id>.json×2 — replay artifacts holding raw per-seat panel votes for all 276 claims. This is the vote data that was previously unrecoverable, and it unblocks thestrength_from_votesrecalibration.Results (dev roster, open-book Brave+FactCheck, CRM-114)
The 49 Trump panel-splits are now surfaced (were collapsed to ~5–7 before) — the core provenance win.
Notes / follow-ups (not in scope here)
$0.163 + $0.062but real proxy spend was ~$1.83 (~7–8×). Worth a fix to thecost_usdfold so operators can budget accurately.Data-only change; suite 944 green.
🤖 Generated with Claude Code