Implements #7980. Depends on #7982 (shared calibration module).
Problem
The existing self-correction system only detects a systematically-wrong rule via precision-over-time (auto-tune.ts), which needs a real, decided sample (≥10) accumulated over however long that takes — too slow for a bug that can mis-close 4 PRs within hours, as happened 2026-07-21/22. A much cheaper, ground-truth-free signal already exists in the data: the SAME deterministic rule/blocker code rejecting several DIFFERENT targets in a short window in the same repo/cohort is itself a strong "something's broken" signal, independent of whether any of those rejections ever gets corrected by a human.
Area
Built on the shared module (#7982). ORB-side: src/rules/advisory.ts finding codes / src/review/content-lane's deterministic codes. AMS-side: whatever policy-check codes AMS's own deterministic layer produces.
Proposal
- A rolling counter, keyed by (deployment/repo-or-cohort, rule/finding code), over a short window (e.g. 1-24h, tunable).
- When the counter crosses a threshold (e.g. ≥3 within 24h), emit an operator-visible alert (existing Discord/Slack notification path, or a new dedicated channel — reuse what's there) naming the rule code and the affected PR/attempt numbers, WITHOUT taking any autonomous action (detection + alert only, mirrors the existing
gamingPatternFlags precedent in src/orb/analytics.ts: "Detection only — never an automatic action").
- Applies uniformly to both ORB gate blockers and AMS policy rejections via the shared module's interface.
Deliverables
Resources
Boundaries
Detection + alert only — no autonomous hold/disable action (that's sub-issues 4-6 of #7980's job, and even those stay tightening-only per auto-tune.ts's existing design principle).
maintainer-only.
Implements #7980. Depends on #7982 (shared calibration module).
Problem
The existing self-correction system only detects a systematically-wrong rule via precision-over-time (
auto-tune.ts), which needs a real, decided sample (≥10) accumulated over however long that takes — too slow for a bug that can mis-close 4 PRs within hours, as happened 2026-07-21/22. A much cheaper, ground-truth-free signal already exists in the data: the SAME deterministic rule/blocker code rejecting several DIFFERENT targets in a short window in the same repo/cohort is itself a strong "something's broken" signal, independent of whether any of those rejections ever gets corrected by a human.Area
Built on the shared module (#7982). ORB-side:
src/rules/advisory.tsfinding codes /src/review/content-lane's deterministic codes. AMS-side: whatever policy-check codes AMS's own deterministic layer produces.Proposal
gamingPatternFlagsprecedent insrc/orb/analytics.ts: "Detection only — never an automatic action").Deliverables
Resources
src/orb/analytics.ts(gamingPatternFlags— detection-only precedent)src/services/notify-discord.ts/ notify-slack (existing alert channels)Boundaries
Detection + alert only — no autonomous hold/disable action (that's sub-issues 4-6 of #7980's job, and even those stay tightening-only per
auto-tune.ts's existing design principle).maintainer-only.