docs(#3510): add AGENTS.md guidance for review conventions - #3511
Conversation
Add two sections to AGENTS.md to prevent false-positive review findings: 1. Go file header convention: documents the repo-wide pattern of placing SPDX license headers before //go:build tags, so review agents do not flag this as a style violation. 2. Security fix review expectations: instructs review agents to flag documentation gaps as informational comments rather than blocking change requests on security-fix PRs. Both patterns were identified from PR #3489 where review agents issued miscalibrated findings. This follows the established pattern of adding AGENTS.md guidance to improve agent review accuracy (similar to #3503, #3508, #3451, #3389, #3458). Note: pre-commit could not run in sandbox (network restriction). The post-script runs it authoritatively. Closes #3510
|
🤖 Finished Review · ✅ Success · Started 3:03 PM UTC · Completed 3:13 PM UTC Commit: |
ReviewFindingsMedium
Labels: PR modifies agent guidance documentation (AGENTS.md) |
fullsend-ai-review
Bot
added
requires-manual-review
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
🤖 Finished Retro · ✅ Success · Started 2:06 PM UTC · Completed 2:24 PM UTC Commit: |
Retro: PR #3511 — AGENTS.md guidance for review conventionsWhat happenedPR #3511 added two sections to Pipeline executionThe end-to-end agent pipeline executed cleanly with zero rework cycles:
The code agent produced a well-scoped 18-line addition on the first try. The review agent correctly identified Improvement opportunities — all covered by existing issuesThe underlying problems from PR #3489 (repeated review cycles, miscalibrated severity, re-raised findings) are well-covered by existing open issues. No novel proposals are warranted. Evidence for existing issues:
Broader patternThe conforma/cli repo has accumulated 7 retro-generated AGENTS.md issues in ~2 months (#3389, #3451, #3458, #3502, #3503, #3508, #3510), with only #3510 fully resolved (this PR). Each encodes domain knowledge that review agents lack, following a consistent pattern of preventing a specific false-positive class. The pipeline is working as designed — the acceleration reflects the retro agent finding genuine review calibration gaps. |
1 participant
What:
Add two new sections to AGENTS.md to prevent false-positive review findings:
Go file header convention — Documents the repo-wide pattern of placing SPDX license headers before
//go:buildtags. Review agents (fullsend and qodo) were flagging this as a style violation on PR Use ValidateVSAAndComparePolicy for ec validate image VSA skip #3489, but it is an intentional convention across all Go files.Security fix review expectations — Instructs review agents to flag documentation gaps (e.g., missing
THREAT_MODEL.mdorDESIGN.mdupdates) as informational comments rather than blocking change requests when the PR's primary purpose is a security fix. On PR Use ValidateVSAAndComparePolicy for ec validate image VSA skip #3489, the review agent issued REQUEST_CHANGES verdicts in 6 of 10 review cycles for doc gaps on a security fix that both human reviewers considered non-blocking.Why:
Both findings from PR #3489 were technically correct but miscalibrated for this repo's conventions. The build-tag false positive is a straightforward convention gap. The documentation severity miscalibration creates friction and teaches authors to ignore agent verdicts. Adding AGENTS.md guidance follows the established pattern in this repo (#3503, #3508, #3451, #3389, #3458).
Tickets:
Closes #3510
Post-script verification
agent/3510-agents-md-review-guidance)a18338416237ca7c5154a0af499f81d4b821ea46..HEAD)