Report failed Claude review executions honestly - #1137
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
Contributor
blove
enabled auto-merge (squash)
September 23, 2026 14:59
This was referenced Sep 23, 2026
blove
added a commit
that referenced
this pull request
Sep 23, 2026
Every Claude Review run since 2026-07-08 ~06:00 UTC has errored on its first API call (is_error: true, one turn, ~250ms, $0), so no PR has been AI-reviewed since. The last real review ran at 05:34 that day; the workflow did not change in between, so the cause is outside the repo. But the verifier from #1137 prints the same line for every failure, so the log cannot tell a revoked key from an empty balance or a retired model. On an execution that reports is_error: true, print the terminal result's error text: one line, capped at 300 characters, with anything key-shaped redacted before truncating. A finished review's result is the review itself, so nothing is printed unless is_error is true; #1137's rule against printing transcript contents still holds and is still tested. This PR runs its own copy of the verifier, so its own review job will name the cause.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claude review runs returned
subtype: successwithis_error: trueand produced no review, but their checks stayed green. The pinned action determines success from subtype alone; the workflow also swallowed action failures.Check the action's documented execution file for a terminal success with
is_error: false, and let execution failures fail the advisory job. Missing, malformed, and incomplete output also fails without printing transcript contents. Fork PRs skip this secret-dependent review. Required CI and branch protection are unchanged.Validation: 164 workflow and CI-scope tests pass, including 13 new process/workflow regressions. The existing workflow failed the new regression before the fix. The full
nx test scriptssuite also passes after registering the new Node suite in Vitest's exclusions. Formatting and diff checks pass. Independent source review found no actionable issues.Hosted run 35877008975 reproduces the actual
is_error: trueresponse and the new verifier exits 1, correctly making the advisory job fail. This corrects execution reporting; it does not claim to resolve the provider error or establish the quality of a completed review.