chore(ci): emit claude:review:passed/failed label from automated PR review [PYSDK-105]#593
Conversation
…eview [PYSDK-105] Adds a machine-readable verdict from the automated Claude PR review so branch-protection rules can gate `auto-merge` on the review outcome without human intervention. Changes: * `.github/labels.yml`: two new mutually-exclusive labels - `claude:review:passed` (green) — no blocking findings on current head - `claude:review:failed` (red) — blocking findings on current head * `.github/workflows/claude-code-automation-pr-review.yml`: extend prompt with a mandatory "Machine-Readable Verdict" final step. Claude must emit PASS or FAIL based on the existing CRITICAL CHECKS criteria (test markers, coverage ≥85%, lint clean, conventional commits, architecture/security) and apply the corresponding label via `gh pr edit --add-label ... --remove-label ...`. The opposite label is always removed, so the labels stay mutually exclusive across re-reviews on subsequent pushes. Re-emission: every push to a PR re-runs the review (existing `synchronize` trigger), so the verdict is always tied to the current head commit — stale verdicts cannot persist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
helmut-hoffer-von-ankershoffen
added
sop:cc-sop-01
|
There was a problem hiding this comment.
Pull request overview
Adds a machine-readable outcome for the automated Claude PR review by introducing two mutually exclusive labels that can be used by branch protection / auto-merge gating.
Changes:
- Extend the Claude PR review workflow prompt to require emitting a PASS/FAIL verdict and applying a corresponding
claude:review:*label. - Add two new labels (
claude:review:passed,claude:review:failed) to the repo’s label sync configuration.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| .github/workflows/claude-code-automation-pr-review.yml | Updates the automation prompt to require applying a deterministic PASS/FAIL label verdict after the review. |
| .github/labels.yml | Defines the new claude:review:passed/failed labels for label-sync creation. |
| ## Machine-Readable Verdict (MANDATORY) | ||
|
|
||
| After posting your review comment, you MUST emit a single-label verdict on the PR. This label is consumed by branch-protection rules to gate auto-merge — it is the only deterministic signal of your review outcome. | ||
|
|
||
| **Verdict criteria** (all must hold for PASS): | ||
|
|
||
| - No blocking findings under "CRITICAL CHECKS" — i.e. no missing test markers, no coverage drop below 85%, no `make lint` failures, no conventional-commit violations. | ||
| - No blocking architecture or security violations under "Repository-Specific Review Areas". | ||
| - Suggestions / nice-to-haves do NOT block the verdict. | ||
|
|
||
| If any blocking finding remains: verdict is **FAIL**. | ||
| Otherwise: verdict is **PASS**. | ||
|
|
||
| **Apply the label** (the two labels are mutually exclusive — always remove the opposite one): | ||
|
|
||
| ```bash | ||
| # PASS: | ||
| gh pr edit ${{ github.event.pull_request.number }} \ | ||
| --add-label "claude:review:passed" \ | ||
| --remove-label "claude:review:failed" | ||
|
|
||
| # FAIL: | ||
| gh pr edit ${{ github.event.pull_request.number }} \ | ||
| --add-label "claude:review:failed" \ | ||
| --remove-label "claude:review:passed" | ||
| ``` |
There was a problem hiding this comment.
Applying claude:review:* labels from within this workflow can retrigger the workflow because it listens to pull_request labeled events and the job condition stays true as long as the PR has the claude trigger label. That can cause at least one redundant extra Claude run per review (and potentially a loop if GitHub emits labeled even when --add-label is a no-op). Consider gating the job so it only runs on labeled events when github.event.label.name == 'claude', or remove labeled from the triggers and switch to a different trigger for starting the review.
| ```bash | ||
| # PASS: | ||
| gh pr edit ${{ github.event.pull_request.number }} \ | ||
| --add-label "claude:review:passed" \ | ||
| --remove-label "claude:review:failed" | ||
|
|
||
| # FAIL: | ||
| gh pr edit ${{ github.event.pull_request.number }} \ | ||
| --add-label "claude:review:failed" \ | ||
| --remove-label "claude:review:passed" |
There was a problem hiding this comment.
gh pr edit --add-label fails if the label does not already exist in the repo. Since labels-sync.yml only syncs labels on push to main, this PR’s own review run (and any runs before labels are created in repo settings) can error when trying to apply claude:review:passed/failed. To make the automation robust, add guidance/logic to ensure the labels exist first (e.g., create them if missing, or handle the “label not found” failure gracefully and still post the review).
|
ok, stupid idea, obsolete |
Pull request was closed
Codecov Report✅ All modified and coverable lines are covered by tests. |
🛡️ Implements PYSDK-105 — Governed by CC-SOP-01, part of our ISO 13485-certified QMS.
Summary
Adds a machine-readable verdict from the automated Claude PR review so branch-protection rules can gate
auto-mergeon the review outcome without human intervention.claude:review:passed(green) andclaude:review:failed(red).claude-code-automation-pr-review.yml) extended with a mandatory "Machine-Readable Verdict" final step. Claude emits PASS or FAIL based on the existing CRITICAL CHECKS criteria (test markers, coverage ≥85%, lint clean, conventional commits, architecture/security) and applies the corresponding label viagh pr edit --add-label … --remove-label …. The opposite label is always removed, so the labels stay mutually exclusive.Why
Today the automated review leaves a sticky comment but no structured verdict — a human still has to skim the comment to decide whether blocking findings remain. With the new labels, branch-protection can require both green CI and
claude:review:passedbefore GitHub-native auto-merge fires. This unblocks fully unattended auto-merge for low-risk PRs (Renovate/Dependabot bumps, the upcoming daily audit-vulnerabilities routine remediation PRs, and otherauto-merge-labelled changes).Re-emission
Every push to a PR re-runs the review (existing
synchronizetrigger), so the verdict is always tied to the current head commit. A staleclaude:review:passedcannot persist past a new commit — the workflow flips the label on the next push.Risk
make lint/make test/make auditremain primary defences, (c) the verdict is re-emitted on every push.claude:review:passed).Out of scope
Test plan
make lintclean.gh label sync(vialabels-sync.yml) creates the two new labels.claude:review:passed.claude:review:failed.claude:review:passedon next push.🤖 Generated with Claude Code