Skip to content

fix: resample empty provider responses like Grok Build to prevent premature session completion - #42

Open
Blankeos wants to merge 1 commit into
mainfrom
fix/grok-4.6-emptyresponseloop
Open

fix: resample empty provider responses like Grok Build to prevent premature session completion#42
Blankeos wants to merge 1 commit into
mainfrom
fix/grok-4.6-emptyresponseloop

Conversation

@Blankeos

Copy link
Copy Markdown
Owner

Sessions on xAI/Grok would silently finish after reasoning-only or no-visible-content responses because response.completed lacks end_turn and phase metadata. The stream treated these as finished turns and stopped the agent loop.

This change adds a resample loop in stream_with_tools that detects empty completions (no text, no tool calls) and retries the same request, matching Grok Build's AttemptOutcome::Empty behavior. Doom loop triggers are now parsed from ResponseCompleted payloads and used to inject recovery reminders when reasoning-only samples hit tail repetition.

  • Extract doom_loop_triggers_from and summarize_response_output_types in openai.rs
  • Log response.completed with status, end_turn, incomplete_reason, output_types, doom_loop
  • Add doom_loop_triggers field to ChunkType::ResponseCompleted
  • Add retryable patterns for "empty response", "reasoning_only", "no_visible_content"
  • Add x-grok-exact-repetition-check header to xAI build requests
  • Add tests: resamples_reasoning_only_completed_response_like_grok_build, reasoning_only_with_thinking_doom_loop_resamples_with_reminder

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Blankeos
Blankeos force-pushed the fix/grok-4.6-emptyresponseloop branch from 2023dcd to 3780c94 Compare August 31, 2026 10:33
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 31, 2026

Copy link
Copy Markdown

Deploying crabcode with  Cloudflare Pages  Cloudflare Pages

Latest commit: ed6d8e5
Status: ✅  Deploy successful!
Preview URL: https://7679fe77.crabcode.pages.dev
Branch Preview URL: https://fix-grok-4-6-emptyresponselo.crabcode.pages.dev

View logs

@Blankeos

Copy link
Copy Markdown
Owner Author

PR Review (updated)

fix/grok-4.6-emptyresponseloopmain

PR: #42

Merge confidence: 4.5/5 (was 3.5/5)


Suggested GitHub PR body

Grok 4.6 (xAI Responses / cli-chat-proxy) was silently ending the session after a reasoning-only response.completed — no text, no tool calls, end_turn=None, no phase metadata. The finish gate treated that as StopReason::Finish.

This matches Grok Build: classify those payloads as empty (reasoning_only / no_visible_content), resample the same request (rollback streamed reasoning), and fail after the retry budget instead of pretending the turn completed. Doom-loop tail_repetition@thinking on an empty sample injects the recovery reminder. Preamble text with no tools still finishes. Hosted provider-tool steps and content-filter empties are not resampled.

Also: parse doom_loop_check off response.completed, log output item types, send x-grok-exact-repetition-check: 64.


What changed

File Role
src/aisdk/response.rs Nested 'resample loop; empty terminal → rollback + retry; doom-loop reminder on empty+confident trigger; hosted tools / content-filter skipped
src/aisdk/chunk.rs ResponseCompleted.doom_loop_triggers
src/aisdk/providers/openai.rs Parse/log completed status, output_types, doom_loop_check
src/aisdk/retry.rs empty response / reasoning_only / no_visible_content are retryable
src/llm/xai_build.rs x-grok-exact-repetition-check: 64
_plans/PREMATURE_COMPLETE_BUG.md Incident notes

1 commit: 3780c94


Checks run (read-only)

Check Result
git diff --check clean
cargo fmt clean
just check-aisdk-boundary ok
cargo test --bin crabcode aisdk::response::tests 42/42
cargo test --bin crabcode aisdk::providers::openai::tests 45/45 (prior run)
cargo test --bin crabcode aisdk::retry::tests 8/8 (prior run)
cargo test --bin crabcode llm::xai_build::tests 11/11 (prior run)
Live grok-4.6 continue smoke no longer terminates; that session still flaky

New tests that passed: resamples_reasoning_only_completed_response_like_grok_build, resamples_no_visible_content_completed_response, empty_response_exhaustion_fails_instead_of_finish, content_filter_empty_does_not_resample, hosted_tool_only_completed_response_is_not_empty, reasoning_only_with_thinking_doom_loop_resamples_with_reminder. Empty-resample tests use start_paused so they do not wait real backoff.


Regressions?

Covered path is fixed. Reasoning-only / no-visible-content response.completed no longer Finishes. Preamble text still finishes. Hosted-tool-only and Chat Completions content-filter empties do not resample. Exhaustion is StopReason::Error, not a silent finish.

Intentional behavior change: persistent empty completions fail after retries instead of idle-complete. Better than silent stop.

Remaining risks (not blockers):

  1. Global in stream_with_tools. Empty resample applies to every provider, not just xAI Responses. Likely correct (Grok Build treats empty as transport-agnostic), but a legitimate empty stop from another provider now retries then errors.
  2. Responses content-filter. Skip is FinishReason::ContentFilter on ChunkType::End. response.completed never sets that, so a content-filtered empty Responses payload would still resample.
  3. Doom recovery exhausted ≠ accept. After 2 recoveries the tool-call doom path disarms. On the empty path, exhausted recoveries fall through to empty retry, then Failed.
  4. Backoff is real. Empty samples share PROVIDER_STEP_MAX_RETRIES (10) and 2s exponential sleep. Matches Grok Build EmptyResponse retryability. A sticky empty can stall ~3.5 minutes then fail. Live smoke already showed this is preferable to silent finish.

Migrations?

No.


Merge checklist

  • Open the GitHub PR (#42)
  • Tests: no_visible_content, exhaustion → Error, content-filter skip, hosted ProviderToolCall skip
  • Assert x-grok-exact-repetition-check=64
  • Keep network backoff (matches Grok Build)
  • Live grok-4.6 continue smoke — no longer terminates
  • Optional: skip empty resample when Responses incomplete_reason / status is content-filter
  • Optional: after DOOM_LOOP_MAX_RECOVERIES, accept the empty sample instead of failing

Confidence bumps

If you do this New score
Responses-path content-filter skip (incomplete_reason / completed status) 4.7/5
+ doom-exhausted empty sample is accepted (Grok Build disarm) instead of Failed 4.9/5

A 5/5 needs those two edges or a documented “won’t fix” on them. The original silent-finish bug is covered.

…mature session completion

Sessions on xAI/Grok would silently finish after reasoning-only or no-visible-content
responses because `response.completed` lacks `end_turn` and phase metadata. The stream
treated these as finished turns and stopped the agent loop.

This change adds a resample loop in `stream_with_tools` that detects empty completions
(no text, no tool calls) and retries the same request, matching Grok Build's
`AttemptOutcome::Empty` behavior. Doom loop triggers are now parsed from
`ResponseCompleted` payloads and used to inject recovery reminders when reasoning-only
samples hit tail repetition. Hosted provider-tool steps and content-filter empties
are not resampled.

- Extract `doom_loop_triggers_from` and `summarize_response_output_types` in openai.rs
- Log `response.completed` with status, end_turn, incomplete_reason, output_types, doom_loop
- Add `doom_loop_triggers` field to `ChunkType::ResponseCompleted`
- Add retryable patterns for "empty response", "reasoning_only", "no_visible_content"
- Add `x-grok-exact-repetition-check` header to xAI build requests
- Add tests: `resamples_reasoning_only_completed_response_like_grok_build`,
  `reasoning_only_with_thinking_doom_loop_resamples_with_reminder`,
  `resamples_no_visible_content_completed_response`,
  `empty_response_exhaustion_fails_instead_of_finish`,
  `content_filter_empty_does_not_resample`,
  `hosted_tool_only_completed_response_is_not_empty`
@Blankeos
Blankeos force-pushed the fix/grok-4.6-emptyresponseloop branch from 3780c94 to ed6d8e5 Compare September 1, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant