[ci-scan-feedback] ci-scan: add Hard Rule 10 for early exit on no scannable build

[github-actions]

Triggering signals

• (run #27518931725, rubric finding: 2.4M effective tokens consumed before concluding with no follow-up build yet, defer to next run skip reason — same outcome as a run that used only 249K ET)
• Same pattern observed in 4 of 27 ci-scan runs since the window opened (2026-06-08): runs #27280572825, #27419354921, #27467511572, #27518931725 each burned ≥2M ET before emitting the same noop the low-ET runs reached in one step.

Proposed edits

• .github/workflows/ci-scan.agent.md lines 81–92: Add Hard Rule 10 that elevates the no-scannable-build exit to a first-class invariant. The existing Step 1 sentence "and stop" was not preventing the agent from continuing to fetch timelines and logs. Hard Rule 10 names the exact operations that are forbidden (fetch a timeline, download any log) and gives the tally row literal so the agent never needs to compute it. Also update the Step 1 trailing sentence to reference Hard Rule 10 instead of restating it.

Expected behavior change

On any run where Step 1 yields no follow-up build yet, defer to next run (or either other selection-time skip reason), the scanner will append the reason to the coverage file, print the tally, call noop, and stop — without fetching any AzDO timeline, Helix work item, or task log. This eliminates the 10× token variance observed between low-ET (correct) and high-ET (incorrect) runs on identical pipeline state.

Generated by CI Failure Scanner - Feedback (machinelearning) · ● 4.4M · ◷

---

Note

This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available in the agent artifact in the workflow run linked above.

To create a pull request with the changes:

# Download the artifact from the workflow run
gh run download 27523314204 -n agent -D /tmp/agent-27523314204

# Create a new branch
git checkout -b fix/ci-scan-early-exit-hard-rule-69a7197adfd7572e

# Apply the patch (--3way handles cross-repo patches where files may already exist)
git am --3way /tmp/agent-27523314204/aw-fix-ci-scan-early-exit-hard-rule.patch

# Push the branch to origin
git push origin fix/ci-scan-early-exit-hard-rule-69a7197adfd7572e

# Create the pull request
gh pr create --title '[ci-scan-feedback] ci-scan: add Hard Rule 10 for early exit on no scannable build' --base main --head fix/ci-scan-early-exit-hard-rule-69a7197adfd7572e --repo dotnet/machinelearning

Show patch preview (47 of 47 lines)

From 00b5788b65a2eee096ba792b34de2140aef1fb2c Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Mon, 15 Jun 2026 04:13:43 +0000
Subject: [PATCH] ci-scan: add Hard Rule 10 to force early exit on no scannable
 build

4 of 27 ci-scan runs (15%) consumed 2.4M+ effective tokens before
concluding with the same skip reason ("no follow-up build yet, defer
to next run") that should have stopped the run immediately after Step 1.
The existing "and stop" sentence in Step 1 was insufficient as a hard
constraint.

Add Hard Rule 10 which elevates the no-scannable-build exit to the same
level as the issue-cap and label rules: append the skip reason, print
the tally, call noop, and stop -- without fetching any timeline, log, or
Helix data.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 .github/workflows/ci-scan.agent.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/ci-scan.agent.md b/.github/workflows/ci-scan.agent.md
index 0937c5f..41b5588 100644
--- a/.github/workflows/ci-scan.agent.md
+++ b/.github/workflows/ci-scan.agent.md
@@ -78,6 +78,7 @@ These invariants are not delegated to the shared file. Honor them even if a shar
 7. **All state under `/tmp/gh-aw/agent/`;** each bash call is a fresh subshell.
 8. **AzDO REST is anonymous;** stay on `https://dev.azure.com/dnceng-public/public/_apis/build/...`. Follow every rule in [Environment constraints](shared/ci-scan.instructions.md#environment-constraints) (pre-bind URLs, `%24top`, no redirection).
 9. **Sanitize every embedded log excerpt** per [Sanitization](shared/ci-scan.instructions.md#sanitization).
+10. **Exit at Step 1 on no scannable build.** If Step 1 yields any skip reason (`stale build window (>14d)`, `no follow-up build yet, defer to next run`, or `no failed build in 7d`): append the reason to `/tmp/gh-aw/agent/coverage/MachineLearning-CI.txt`, print the Step 7 tally row (`| 0 | 0 | 0 | 1 |`) to t
... (truncated)