Component
Agent (Python runtime)
Describe the bug
The delivery gate (_apply_delivery_gate, agent/src/pipeline.py:665-728) cannot distinguish "the agent's work was lost" from "the agent correctly concluded there was nothing to do". Both produce zero commits and no PR, so a correct no-op on a new_task is reported to the user as deliverable=lost — and the error classifier then tells them it was a transient infrastructure fault they should retry.
The docstring already enumerates the sanctioned no-ops it exempts (read-only workflows, artifact workflows, push_resolve/resolve strategies, needs_input clarify-and-hold). A conditional request whose condition is already satisfied — "do X if the file lacks Y" where it already has Y — is a fifth sanctioned no-op that is not exempted, and the agent has no way to signal it.
The downstream classification (cdk/src/handlers/shared/error-classifier.ts:495-512) then says:
The change was not saved — "…This is usually a transient workspace fault (e.g. the clone ended up in an unexpected directory), not a problem with your request."
Remedy: "Reply here to try again — a fresh run normally saves the work correctly."
with retryable: true and errorClass: TRANSIENT. For a correct no-op that advice is actively wrong: retrying can never succeed, because the condition will still be satisfied. The user is directed into an unbounded retry loop against a task that already did the right thing.
Expected behavior
An agent that finished with agent_status=success, made no changes, and can articulate why no change was needed should terminate as a non-failure outcome (COMPLETED with a "no change required" note, in the spirit of the existing code_changed=False "answered" path for pr_iteration), or at minimum as a non-retryable failure whose copy says "the agent determined no change was needed" rather than "transient workspace fault, try again".
The deliverable=lost copy and retryable: true should be reserved for the case it was written for: work that was actually produced and then lost.
Current behavior
Task 01KZS7NFB480DGF2A6FZ5N3CZD on aws-samples/sample-semantic-layer-structured:
- Request: "Add a module-level docstring to
agents/shared/advisory.py if it lacks one. Do not change any logic or tests."
agents/shared/advisory.py already has a module-level docstring, so the correct action was to change nothing — and the agent did exactly that, reporting success after 7 turns.- Result:
Task 01KZS7NFB480DGF2A6FZ5N3CZD — FAILED (7m 21s total)
Last milestone: agent_execution_complete
Reason: agent: The change was not saved — Task did not succeed
(agent_status=success, deliverable=lost): the coding task reported
success but no commit reached the branch and no PR was opened —
the agent's changes did not land in the task's repository.
Presented as FAILED / transient / retryable, with $0.19 spent and a remedy that cannot work.
Reproduction steps
- Onboard any repo and submit a
new_task whose instruction is conditional on a state the repo already satisfies, e.g. "Add a module-level docstring to <file> if it lacks one" against a file that already has one.
- The agent correctly makes no change and reports success.
- The task is marked FAILED with
deliverable=lost, classified TRANSIENT / retryable: true, and the user is told to reply to retry.
Possible solution
Two parts, and the first is the substantive one:
- Give a correct no-op a way to be expressed. The agent needs a terminal outcome for "success, no change required, here is why" that
_apply_delivery_gate can treat as a sanctioned no-op — analogous to how needs_input is exempted today and forced to success right after the gate. Without such a signal the gate cannot tell this case from real lost work, because the observable state (no commit, no PR, agent success) is identical.
- Soften the fallback copy. Until (1) exists,
deliverable=lost is ambiguous, and the classifier states a specific cause ("the clone ended up in an unexpected directory") with more confidence than the evidence supports. Wording that admits both possibilities — work lost, or the agent judged no change necessary — would stop pointing users at a retry that cannot help. Keeping retryable: true for the genuinely-lost case is fine; asserting TRANSIENT for both is not.
Found empirically while validating build-regression gating end to end; the task above was a deliberate smoke test, but the misclassification would hit any real conditional request.
Component
Agent (Python runtime)
Describe the bug
The delivery gate (
_apply_delivery_gate,agent/src/pipeline.py:665-728) cannot distinguish "the agent's work was lost" from "the agent correctly concluded there was nothing to do". Both produce zero commits and no PR, so a correct no-op on anew_taskis reported to the user asdeliverable=lost— and the error classifier then tells them it was a transient infrastructure fault they should retry.The docstring already enumerates the sanctioned no-ops it exempts (read-only workflows, artifact workflows,
push_resolve/resolvestrategies,needs_inputclarify-and-hold). A conditional request whose condition is already satisfied — "do X if the file lacks Y" where it already has Y — is a fifth sanctioned no-op that is not exempted, and the agent has no way to signal it.The downstream classification (
cdk/src/handlers/shared/error-classifier.ts:495-512) then says:with
retryable: trueanderrorClass: TRANSIENT. For a correct no-op that advice is actively wrong: retrying can never succeed, because the condition will still be satisfied. The user is directed into an unbounded retry loop against a task that already did the right thing.Expected behavior
An agent that finished with
agent_status=success, made no changes, and can articulate why no change was needed should terminate as a non-failure outcome (COMPLETED with a "no change required" note, in the spirit of the existingcode_changed=False"answered" path for pr_iteration), or at minimum as a non-retryable failure whose copy says "the agent determined no change was needed" rather than "transient workspace fault, try again".The
deliverable=lostcopy andretryable: trueshould be reserved for the case it was written for: work that was actually produced and then lost.Current behavior
Task
01KZS7NFB480DGF2A6FZ5N3CZDonaws-samples/sample-semantic-layer-structured:agents/shared/advisory.pyif it lacks one. Do not change any logic or tests."agents/shared/advisory.pyalready has a module-level docstring, so the correct action was to change nothing — and the agent did exactly that, reporting success after 7 turns.Presented as FAILED / transient / retryable, with $0.19 spent and a remedy that cannot work.
Reproduction steps
new_taskwhose instruction is conditional on a state the repo already satisfies, e.g. "Add a module-level docstring to<file>if it lacks one" against a file that already has one.deliverable=lost, classified TRANSIENT /retryable: true, and the user is told to reply to retry.Possible solution
Two parts, and the first is the substantive one:
_apply_delivery_gatecan treat as a sanctioned no-op — analogous to howneeds_inputis exempted today and forced to success right after the gate. Without such a signal the gate cannot tell this case from real lost work, because the observable state (no commit, no PR, agent success) is identical.deliverable=lostis ambiguous, and the classifier states a specific cause ("the clone ended up in an unexpected directory") with more confidence than the evidence supports. Wording that admits both possibilities — work lost, or the agent judged no change necessary — would stop pointing users at a retry that cannot help. Keepingretryable: truefor the genuinely-lost case is fine; asserting TRANSIENT for both is not.Found empirically while validating build-regression gating end to end; the task above was a deliberate smoke test, but the misclassification would hit any real conditional request.