Skip to content

fix: restore provider refresh and preserve recall re-ask outcomes - #1656

Draft
santoshkumarradha wants to merge 89 commits into
devfrom
Santosh/dev3
Draft

santoshkumarradha wants to merge 89 commits into
devfrom
Santosh/dev3

Conversation

@santoshkumarradha

@santoshkumarradha santoshkumarradha commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

What changed

Ordinary local chat now refreshes connected providers through the same catalog lifecycle as in-process chat. Failed refreshes retain selectable cached models and show the error even when filtering or sorting; retry clears the error.

Recall now publishes cancellation and its re-ask outcome together. Previously, a fast cancelled generation could finish before that outcome was recorded. Session regression fixtures now synchronize with actual readings and shutdown, and retain the inherited Go build cache when isolating their home directory. Assertions and real audit commands remain; no timeouts were increased.

Fixes #1508. Fixes #1655. The provider correction follows the recorded acceptance failure.

This PR is stacked on frozen #1632 (Santosh/dev2); retarget to dev after that dependency lands. The separately identified settings mouse-row mismatch is outside this batch.

Validation

Current combined revision: 70babe7d9936f92a3503dd8ce2dcd5eb14a5f160. Independent integration review passed. Combined canonical build, changelog validation and full make pr-ready BASE=458d1be73 passed: fresh full application tests (165.753 seconds), all eight session-engine shards (87 seconds), and all eight TUI shards (22 seconds); exact-head CI passed. Final combined real-workflow verification passed on this same revision: provider failure/retry, invoice CSV aging, and a customer follow-up summary were built and used. All 27 generated artifact tests and 17 independent checks passed. All 70 requests across nine roles used DeepSeek v4.1 Flash and returned HTTP 200. Continuous three-minute GIF, full video, five screenshots and exact-revision audit. Fresh-chat preference retrieval also searched retained history, so this does not claim isolated automatic recall or live proof of the rare re-ask race; deterministic tests cover that ordering.

Component evidence, which does not replace final combined acceptance:

  • Provider source eb92b814e: canonical build and full gate passed, including fresh full command package tests and eight TUI shards. Actual terminal acceptance passed cold loading, refresh, visible 503 beside filtered cached choices, retry recovery, provider-menu refresh, and closing a delayed picker without reopening. Six catalog GET requests, zero inference calls. The selected deepseek/deepseek-v4.1-flash model stayed unchanged. Closing the picker does not establish transport cancellation.
  • Recall source 73662adb5: full gate passed, including eight session shards. Early/late recall and revised-direction controls passed 300 repetitions. Earlier failed gates were diagnosed and corrected; no failures were waived.
  • Genuine recall workflow on 73662adb5 saved a preference, opened fresh conversations, and generated and ran two Go programs. Request tracing verified late memory injection; independent artifact checks passed. All 27 completed requests used OpenRouter deepseek/deepseek-v4.1-flash (26 HTTP 200, one cancelled auxiliary caption, no fallback). This did not exercise the rare live re-ask ordering; unchanged early-recall regression tests cover that race. Captioned source checkpoints and coverage limitations are attached.

Automatic CI filters do not match the case-sensitive stacked base Santosh/dev2, so CI was explicitly dispatched. Dispatch tests the immediate-parent delta; the combined local full gate compares the entire batch against frozen #1632. The earlier provider change entry initially failed two sentence checks, was corrected, and then passed; the aggregate entry also passed validation.

Recorded provider behavior

These genuine terminal stills show executable revision eb92b814e, cropped only to exclude private footer/settings rows. See the full evidence caption and limits. They establish catalog interaction, not paid model-answer quality or transport cancellation.

Failed refresh remains visible beside cached choices under the model filter:

Filtered picker retains cached model and displays HTTP 503

Successful retry clears the error and adds the new model:

Recovered picker shows both catalog models

The connected provider menu exposes refresh:

Provider menu offers refresh models

hy3560 and others added 30 commits September 25, 2026 21:27
Assisted-by: CodeAF (gemini-3.8-flash-high)
Co-Authored-By: CodeAF <267109073+agentfield-bot@users.noreply.github.com>
Assisted-by: CodeAF (gemini-3.8-flash-high)
Co-Authored-By: CodeAF <267109073+agentfield-bot@users.noreply.github.com>
…lation (#1550)

Defend contracts that standing runs with branch-only grants execute in an
isolated git worktree and leave the host branch untouched, that false prose
claims of branch isolation are caught and marked failed, and that plain ambient
runs without grants remain unaffected.

Assisted-by: CodeAF (gemini-3.8-flash-high)
Co-Authored-By: CodeAF <267109073+agentfield-bot@users.noreply.github.com>
readText resolved one empty or all-blank argument to an empty goal with no
error, so `codeaf do ""` ran a paid job with a goal the planner invented.
The blank check now sits where the goal text is resolved, for every road.

Fixes #1566

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@santoshkumarradha

Copy link
Copy Markdown
Member Author

Recorded terminal verification: provider catalog refresh

Exact executable revision: eb92b814ed73ca79b90b6647c4c3c3bc0270c193. These are authentic stills from the original terminal recording of the ordinary linked-local application, cropped only to exclude private footer/settings rows. They are static screenshots, not a video or reconstructed UI. The unmodified original recording is retained privately.

Goal: load a connected custom provider's catalog, refresh it through the picker and provider menu, keep cached models usable when a refresh fails, and recover without restarting the application. This verifies the provider correction for #1508; it does not certify later aggregate revisions.

Failure with useful cached choices: a controlled catalog endpoint returns HTTP 503. With the matching /model catalog filter still applied, the failure and retry action remain visible beside the cached catalog-alpha choice.

Filtered model picker shows HTTP 503 and retry action while retaining cached catalog-alpha

Successful retry: the error clears and the recovered catalog now contains both catalog-alpha and catalog-beta.

Recovered model picker shows two available models and no refresh error

Provider-menu entry: keyboard navigation exposes the connected provider's refresh models action; the recorded run executes this action.

Connected provider menu offers refresh models

Recorded run checks: cold catalog loading, manual refresh, visible failure beside cached filtered choices, successful recovery, closing the picker during a delayed response without it reopening, and keyboard provider-menu refresh all passed. Retained request receipts show six catalog GET requests, zero POST requests, and zero inference calls. The selected deepseek/deepseek-v4.1-flash model remained unchanged.

Limits: catalog responses came from a deterministic loopback test service. This was real terminal interaction, but it was not a paid model workflow and does not establish alternate-provider inference behavior. Closing the picker was exercised; transport cancellation is not claimed. The three published stills illustrate failure/cache retention, recovery, and the menu; the additional timing and request-count checks rely on the retained original recording and request receipts.

@santoshkumarradha santoshkumarradha changed the title fix(providers): refresh connected catalogs on ordinary local launches fix: restore provider refresh and preserve recall re-ask outcomes Sep 28, 2026
@santoshkumarradha

Copy link
Copy Markdown
Member Author

Source evidence: saved preferences across new conversations (#1655)

Exact source executable: 73662adb5016673ab043282444942da6071dcb99 (source #1655). This is source-specific proof, not final combined acceptance for #1656 or any later revision.

Real workflow: a first conversation saved a Go preference for hard tabs and descriptive variable names. A new conversation created and ran hello.go; another new conversation created and ran farewell.go. Independent artifact checks confirmed both expected two-line outputs, actual tab characters, and descriptive variables.

First coding conversation: the genuine terminal checkpoint at 145 seconds shows Hello, Ada! and Hello, Grace!, followed by the applied preference explanation.

Recorded hello.go result and applied hard-tab and descriptive-variable preferences

Another new conversation: the genuine checkpoint at 160 seconds shows Goodbye, Ada! and Goodbye, Grace!, with the preference explanation and created file.

Recorded farewell.go result and applied saved Go preferences

Recall evidence and limits: both coding turns journaled recall:late. Retained request tracing confirms the preference was absent from the initial main requests, then arrived as an injected session-memory note in main request cf080b4b and subsequent requests. Both turns also searched earlier conversation history, so the useful artifacts cannot be attributed exclusively to automatic recall. No live recall:reasked occurred: this run does not prove the first-word reask race; that case is covered by the source regressions.

Model accounting: all recorded roles used OpenRouter deepseek/deepseek-v4.1-flash. There were 27 completed requests: 26 HTTP 200 responses and one auxiliary caption request cancelled by its context. These are not 27 successful responses. The retained usage log contains 26 rows for the exact model, with no fallback or remaining active requests; the tested binary hash stayed unchanged.

Media provenance: these two static images are cropped frames from the original live PTY recordings (149.59 seconds and 165.33 seconds respectively), preserving the recorded application rendering. Crops exclude private paths; no UI or output was recreated. They are screenshots, not a video. Original recordings and raw traces remain private and unchanged. The separately owned final combined recording will be labeled with its own revision and results.

@santoshkumarradha

santoshkumarradha commented Sep 28, 2026 •

Copy link
Copy Markdown
Member Author

Final combined real-user acceptance — 70babe7

Result: the useful workflow completed on this exact PR head. In ordinary local chat, a connected catalog failed and recovered correctly; then the existing invoice project gained a usable aging CSV and a second customer follow-up report in a fresh conversation. The actual generated programs were run and their outputs checked independently.

  • Providers: one word everywhere, a connected provider always lists its models, and adding one takes one step #1508 provider refresh: filtered /model + Ctrl+R reached a controlled catalog-only provider. HTTP 503 kept the cached model selectable and displayed the error; retry after recovery made another GET, returned 200, and cleared the error. The catalog fixture performed no inference. The selected DeepSeek model stayed unchanged.
  • Useful artifact: builder/checker collaboration added CSV aging buckets and exercised date boundaries and quoting. The exported four overdue invoices total 1849.95; the follow-up report correctly ranks Maple Studio first, 1500.00, oldest 27 days. Existing invoice data and text/JSON behavior were preserved.
  • Independent use: 27 generated artifact tests and 17 additional CLI/output assertions passed, including compatibility, exact CSV contents, customer ordering, totals, unchanged original data, and the saved report matching actual program output. These artifact checks are separate from the full repository gate already reported above.
  • Model audit: 70 requests, 70 HTTP 200 responses and 70 usage receipts, all OpenRouter deepseek/deepseek-v4.1-flash. Verified roles: turn, task, reflex, router, routerconfirm, markreader, handoff-draft, caption and title. No transport errors or empty-at-ceiling responses. Earlier revisions are excluded from these counts.
  • Late-recall regression reports recall instead of recall:late in combined gate #1655 scope: the fresh chat requested recall and reproduced the saved standard-library/two-decimal/runnable-command preference. Its delegated reader also searched retained history/transcripts. This is ordinary successful continuation, not isolated proof of automatic recall injection or the rare cancellation/re-ask race; the source-specific deterministic regression and separately labeled component evidence remain the proof for that race.

Recording provenance: genuine original terminal-output stream, retained in full with original timing. No pane-snapshot reconstruction, staged UI, or idle compression. The GIF is one contiguous 180-second source interval (01:50–04:50) rendered at the 1× setting; GIF frame timing yields approximately 181 seconds. It shows builder/checker results, starting a fresh chat, and producing/using the follow-up report. The full recording includes provider failure/retry and the complete workflow; rendered video timing is approximately 320 seconds for the 311-second original stream. Screenshots below are actual checkpoints from that same stream. Disposable fixture paths are test inputs; no credentials or personal host paths are included.

Three-minute continuous workflow excerpt:

Continuous invoice-building and fresh-chat follow-up workflow at exact 70babe7d9

Full original-session rendering — provider refresh through usable reports:

final-workflow-70babe7d9.mp4
Captioned checkpoints from the same final combined run

Provider temporarily unavailable: the filtered connected model remains cached/selectable, with its 503 error and retry instruction visible.

Final provider error with cached selectable model

Provider recovered: retry reached the catalog successfully and cleared the error; selection remains DeepSeek.

Final provider retry recovered

Invoice export completed: checker reports boundary/quoting checks and the actionable customer priority.

Final invoice CSV verified in real workflow

Fresh-chat report actually used: the generated summary contains correct customer ordering, balances and oldest overdue days, with a runnable command.

Actual generated customer follow-up report

Saved preference reproduced: standard-library, two-decimal money and runnable-command preferences appear in the fresh conversation; history-search limitation is stated above.

Fresh chat reporting preference reproduced

Native executable SHA256: 139da49975f488f0bfcea1f48a8bd4d72ab745ac15c0e5ce5a8ca54c57d55b6d. Original stream SHA256: 365c7e68a66183f6e33a14d45b5aa36b569c0740e385b5a5eb4050e72adad569. Original recordings, receipts and independent check results are retained. No installation or merge was performed by this acceptance run.

Base automatically changed from Santosh/dev2 to dev September 28, 2026 17:51
@santoshkumarradha
santoshkumarradha marked this pull request as ready for review September 28, 2026 18:03
@santoshkumarradha

Copy link
Copy Markdown
Member Author

@AbirAbbas This finite follow-up batch is ready for review at 70babe7d9936f92a3503dd8ce2dcd5eb14a5f160: connected-provider refresh (#1508) and recall ordering/test reliability (#1655). Combined Spark full affected-package gate, exact-head dispatched CI, and CLA passed. Final useful invoice workflow passed 27 artifact tests and 17 independent checks; all 70 requests across nine roles used DeepSeek v4.1 Flash. Recorded workflow: three-minute GIF, full video, screenshots and limitations. The PR summary distinguishes deterministic recall-race coverage from the live history-search confound and documents dispatched-CI coverage. This PR is stacked on #1632; review its delta now and retarget to dev after that parent lands. #1632 has not been changed.

@santoshkumarradha santoshkumarradha added bug Something the code does that it should not area:provider Routing, lanes, refusals, hedging, what a call costs area:session The engine — turns, tasks, the toolbelt, checkpoints sev:serious Wrong or missing behaviour a person meets in ordinary use labels Sep 28, 2026
@santoshkumarradha santoshkumarradha added this to the Reliable agent milestone Sep 28, 2026
@santoshkumarradha

Copy link
Copy Markdown
Member Author

PR-bar audit: the existing evidence comments (terminal verification, source evidence) and the body capture are good groundwork. Missing against the bar:

  1. CI — only license/cla has results on this head; check / light gate / touched packages need runs.
  2. Review passes — the three structured passes (architecture / security / efficiency & design) as separate comments.
  3. Spark verification — pending host availability.

Marked draft until 1–3 close; the evidence comments stay and count toward on-screen proof.
drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

Bar audit — gaps against the new PR bar

1. CI + changelog: ⚠️ the exact-head dispatched gate passed (run 36456944224, 2026-09-28) but no pull_request-triggered checks are attached to this PR, and dev has moved about a week since (stack parent #1632 has merged). A fresh green gate on the current head against current dev is required before ready. ✅ changelog entries present (multiple).

2. Review passes: ❌ verification and acceptance comments exist, but none of the three named passes (Pass 1 ARCHITECTURE, Pass 2 SECURITY, Pass 3 CODE EFFICIENCY & DESIGN) are posted, each as a separate comment naming what was checked and what was found.

3. Video proof: ⚠️ a three-minute GIF and full video exist in a comment (linked from the body); the bar requires the media embedded in the PR body itself.

4. Spark: the earlier gates ran on the Spark; the fresh gate run stays there.

Action taken: converted back to draft until fresh CI, the three passes, and the body-embedded video exist.

—
Drafted with CodeAF · reviewed and owned by the author

@santoshkumarradha
santoshkumarradha marked this pull request as draft October 3, 2026 18:53
@santoshkumarradha

Copy link
Copy Markdown
Member Author

PR Bar Audit — gaps against the new bar

1. CI + changelog: ⚠️ No CI workflow runs at all (only license/cla reports; head Santosh/dev3 is stale — needs a fresh push/rebase onto dev to trigger check/light gate/touched packages). ✅ Changelog present.
2. Review passes: ❌ None posted — Pass 1 (ARCHITECTURE), Pass 2 (SECURITY), Pass 3 (CODE EFFICIENCY & DESIGN) all absent.
3. Video proof: ✅ Video present in the body.
4. Spark runs: CI re-run must happen on the Spark (fleet).
drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

PR bar audit — directive #1790715358822565124

1. CI + changelog: ⚠️ current rollup shows only license/cla — the exact-head CI run referenced in the body predates the latest pushes; a fresh green run on the current head is required by the bar. ✅ thirteen unreleased changelog entries present.

2. Review passes: ❌ four verification/evidence comments exist, but none are the bar's named passes — Pass 1 — ARCHITECTURE, Pass 2 — SECURITY, and Pass 3 — CODE EFFICIENCY & DESIGN are all absent as separate comments.

3. Video proof: ⚠️ a three-minute GIF, full video and screenshots exist but live in a linked comment — the bar requires the video embedded in the PR body itself.

4. Merge state: CONFLICTING against dev — needs a rebase; CI must be re-run green on the rebased head.

Fixes for the gaps are in flight. This PR does not turn ready until all four conditions hold.

drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

Audit against the new PR bar

  • CI + changelog: changelog entries present ✅ — but no PR-gate run is attached to this head (the last green run was a manual workflow_dispatch on 2026-09-28) and the PR is CONFLICTING with dev. Make the branch mergeable (merge dev in; do not rebase-force-push) so the gate runs fresh.
  • Review passes: 0 of 3 — the existing comments are verification evidence, not the required passes. ARCHITECTURE, SECURITY, and CODE EFFICIENCY & DESIGN must each be posted as its own comment naming what was checked and what was found.
  • Video proof: missing — provider catalog refresh behaviour is user-facing; a capture must be embedded in the PR body.

Converted back to draft until all four conditions hold; nothing merges until then.

drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

PR-bar audit (team bar, 2026-10-03) - status per category:

  • CI: 1 of 1 checks green, but only one check is visible on this branch - confirm the required check set applies to it.
  • Changelog: present (13 entries).
  • Review passes: 0 of 3 - existing comments carry valuable verification evidence and partial security notes, but the three labeled pass comments are absent; they are being posted (100+ file diff).
  • Video proof: 3 embedded media refs are present but none is served as video (likely stills) - a capture of the change running is needed. yes

Per the no-video rule this returns to draft until a real capture is embedded.
drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

PR bar audit — gaps against the new bar

1. CI + changelog — ⚠️ only license/cla has run on the head commit. check, light gate and touched packages are absent, not green — they must run and pass. ✅ changelog entries present (13 unreleased files, incl. 1656-linked-local-provider-refresh.md).

2. Review passes — all three missing (Pass 1 ARCHITECTURE, Pass 2 SECURITY, Pass 3 CODE EFFICIENCY & DESIGN), as separate comments naming what was checked and what was found.

3. Video proof — missing. User-facing change (provider refresh restored, recall re-ask outcomes preserved). A Spark tmux capture of the refreshed provider flow, rendered to GIF/video, must be embedded in the PR body.

4. Runs on Spark — per the fleet skill.

Converted back to draft until all four conditions hold. Nothing merges before that.

drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

Audit against the PR bar — what is missing

1. CI + changelog — ❌ no Actions run has ever happened on this PR (only license/cla passes); ✅ changelog: thirteen unreleased entries present.
2. Three review passes — ❌ none posted yet. Pass 1 (ARCHITECTURE), Pass 2 (SECURITY) and Pass 3 (CODE EFFICIENCY & DESIGN) follow as three separate comments, each naming what was checked and what was found.
3. On-screen proof — ✅ three captures already embedded in the body.
4. Fleet runs — builds and tests run over ssh on the fleet, never a laptop.

Disposition — converted from ready back to draft until the three passes are posted and CI has actually run green. The branch will be brought current without rewriting its history; nothing merges.

drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

Audit against the new PR bar:

  • CI: only license/cla has run recently; the branch conflicts with current dev (mergeable=CONFLICTING) — rebase and re-run required.
  • Changelog: present (13 entries).
  • Review passes: none posted. Owed: Pass 1 — Architecture, Pass 2 — Security, Pass 3 — Code Efficiency & Design. This PR touches provider credentials and refresh, so Pass 2 — Security matters most here.
  • Video: none in the body; the provider tab refresh is user-facing, so a capture is owed.

Stays draft.

drafted with CodeAF

@santoshkumarradha

Copy link
Copy Markdown
Member Author

Bar audit — what is missing (per the new PR bar)

Audited at head 70babe7d9936f92a3503dd8ce2dcd5eb14a5f160 by @tags.

1. CI + changelog — no check runs recorded on the PR (only license/cla ⏳ pending); CI needs a fresh push or a workflow re-run from the owner. Changelog ✅ — docs/changes/unreleased/1656-linked-local-provider-refresh.md (plus the batch of earlier entries this branch carries).

2. Review passes (three, separate comments) — Pass 1 — ARCHITECTURE ❌ · Pass 2 — SECURITY ❌ · Pass 3 — CODE EFFICIENCY & DESIGN ❌ — 0 of 3 posted. (The existing comments are terminal-verification evidence — valued, but not the three named passes.)

3. Proof on screen — user-facing ✅ (provider refresh error surface, recall re-ask). Video/GIF embedded in the body ❌ missing — the comments hold stills, not a recording; required before ready.

4. Tags — ✅ bug · area:session · area:provider · sev:serious · milestone Reliable agent.

Actions: converted to draft until the bar holds; the three review passes are running, the capture is queued, and the owner is asked to trigger CI. Nothing merges before all four conditions hold.

drafted with CodeAF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider Routing, lanes, refusals, hedging, what a call costs area:session The engine — turns, tasks, the toolbelt, checkpoints bug Something the code does that it should not sev:serious Wrong or missing behaviour a person meets in ordinary use

Projects

None yet

3 participants