Say why a Claude review execution failed - #1140
Merged
Merged
Conversation
Every Claude Review run since 2026-07-08 ~06:00 UTC has errored on its first API call (is_error: true, one turn, ~250ms, $0), so no PR has been AI-reviewed since. The last real review ran at 05:34 that day; the workflow did not change in between, so the cause is outside the repo. But the verifier from #1137 prints the same line for every failure, so the log cannot tell a revoked key from an empty balance or a retired model. On an execution that reports is_error: true, print the terminal result's error text: one line, capped at 300 characters, with anything key-shaped redacted before truncating. A finished review's result is the review itself, so nothing is printed unless is_error is true; #1137's rule against printing transcript contents still holds and is still tested. This PR runs its own copy of the verifier, so its own review job will name the cause.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI review hasn't run on any PR since 2026-07-08. This PR makes the failure name its cause.
What I found
Every Claude Review run errors on its first API call:
is_error: true, one turn, ~250ms, $0. Bisecting ~900 runs:claude-review.ymldidn't change between those runs — the last edit was #756 two days earlier, and reviews kept working after it. So the cause is outside the repo: most likely theANTHROPIC_API_KEYsecret or the account's balance, possibly the pinned modelclaude-sonnet-4-6.Until #1137 these failures reported success, so every green
reviewcheck since July 8 was false.Change
#1137's verifier prints one identical line for every failure, and the action's log strips the error text. So nothing on record distinguishes a revoked key from an empty balance or a retired model. This prints the terminal result's error on failure:
is_error: true. A finished review'sresultis the review itself, so Report failed Claude review executions honestly #1137's rule against printing transcript contents still holds and is still tested.sk-ant-…) redacted before truncation, so a key can't be cut in half and leak the other half.This PR runs its own copy of the verifier (the workflow uses
pull_requestand checks out the PR head), so its ownreviewjob will print the real cause.Verification
node --test scripts/verify-claude-review.spec.mjs— 16/16 (3 new: the reason is printed; it's one bounded, redacted line; no reason is printed withoutis_error)node --test scripts/ci-workflow.spec.mjs(the suite CI runs) — 61/61🤖 Generated with Claude Code