story-084: A rerun keeps the output that explains it - #82
Merged
Conversation
Implemented by the l5 harness story workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Story
A rerun keeps the output that explains it.
A self-route caused by a red suite cited
suite-run-result.jsonand its output file. The rerun it triggered then wrote both over — so by the time a human read the record, the artifact it named held a different run, usually the green one that ended the story, and nothing said so. The coordinator owning the suite run is what makes the exit code trustworthy; the cost was that it kept one slot for an unbounded number of runs.Change
suite_run_checkwrites two pairs instead of one.suite-run-result.jsonand the output beside it hold the most recent run — so every reader that already knows those names, the verifier's rendered context among them, reads what it read before.retained_suite_result_file, in the shapeself_route_result_fileandprompt_filealready establish. The retained output name is derived by passing that result name back throughsuite_output_file, so the output filename is shaped once rather than twice.The try number is the invocation's own
self_route_count— the same number the prompt file for the invocation this run judges is keyed by — so the prompt a stage was given and the suite run judging that turn share a key by construction rather than by two derivations that happen to agree.One record, written twice, with each copy's
output_pathrepointed at the file beside it — the pointer-rewriting shape_archive_companion_outputalready uses — so following either record's own pointer reaches that record's run.suite_run_checkreturns the retained pointer, so the self-route'sartifactsarray and the statement composed from it name the pair that survives. The reason text is untouched: still the truncated summary, since what needed widening was what it points at, not the sentence.attemptandtry_numberare required and keyword-only for the reasonstage_nameis — a defaulted argument would let a call site write an unkeyed run by omission, which is the overwrite the keying exists to remove.Retention is unconditional: a story with a single suite run writes the retained pair too, and the
OSErrorpath writes both records with nooutput_pathon either. The only path writing neither is the one that wrote neither before — a target configuring notest_command.Bound, and what reads these
Retention is unbounded and the bound is structural. Nothing prunes or expires a retained file and no configured retention key was added; the ceiling is arithmetic on budgets the workflow already declares — one pair per invocation, invocations per attempt no more than
max_self_routesplus one, attempts no more than the retry budget. The files are small and run directories are gitignored.Nothing routes on a retained file. Every occurrence of the identifiers under
orchestration/is a write position except two discovery calls that move names and decide nothing:retained_suite_artifactsis written besideself_route_artifactsand built the same way, globbing the run directory because the try is live state a resume has already zeroed, taking the artifact offsuite_run_declarationand the names off the writing functions with the try wildcarded — so discovery cannot drift from writing and no artifact name is spelled inorchestration/.interrupted_attempt_artifactscalls it so a resume archives an interrupted attempt's retained pairs before the resumed stage's first suite run lands on those names;entry_artifactscalls it with the attempt wildcarded too.Evidence
suite-run-result.json, exit 0)clean-clone-result.json, exit 0)verification-result.json)tests/test_coordinator_runs_the_suite.py::test_the_statement_names_both_the_record_and_the_path_to_the_output— exit 1 reverted, exit 0 appliedtests/test_retained_suite_runs.pytest_the_fixture_really_makes_two_suite_runs), repaired in placeNote on the run
This run's own directory holds no retained pair, and that is expected rather than a finding: the coordinator process driving it imported
story_coordinator.pybefore the change existed, so it ran the pre-story code. It is the recurring shape of any story moving work into orchestration. The retention is verified instead throughtests/test_retained_suite_runs.py, which drivesrun_storyagainst temporary targets with a faked agent runner — evidence that does not depend on the running process having loaded the code under test. The first run to demonstrate it on itself is the next one.story-083's
scopefield appears in all three of this run's records, which is its own first live confirmation.🤖 Generated with Claude Code
https://claude.ai/code/session_01HkujNrgMcsRBFWNEKWYnmA