server: extend reasoning-delta token-boundary fix to Responses and Anthropic - #712
Open
Flor1an-B wants to merge 1 commit into
Open
server: extend reasoning-delta token-boundary fix to Responses and Anthropic#712Flor1an-B wants to merge 1 commit into
Flor1an-B wants to merge 1 commit into
Conversation
…thropic Issue antirez#685 reports that streaming reasoning_content/thinking deltas are split at an arbitrary fixed byte offset (7 bytes before the end, held back to detect a possibly-split "</think>" marker) instead of at actual token boundaries, across three independent but identically-shaped code paths: OpenAI chat/completions (~line 6477), Responses (~line 7188), and Anthropic (~line 8098). PR antirez#692 (@rareba) already fixes the OpenAI path: track the end of the previous update's text (reasoning_token_end) and use that as the emit boundary instead of a fixed suffix cut, since production code calls *_sse_stream_update() once per newly decoded token, so the boundary between two calls is always a real token edge. This commit applies the exact same technique to the other two paths antirez#685 explicitly flagged as having the identical shape, which antirez#692 doesn't touch (its diff is scoped to openai_sse_stream_update() only) -- so this is additive to antirez#692, not a competing fix for the same path. Changes, mirrored 1:1 from antirez#692's approach in both responses_sse_stream_update() and anthropic_sse_stream_update(): - add reasoning_token_end to responses_stream and anthropic_stream - set it to raw_len at each point the THINKING branch returns without a close tag or finish (the prefix-check early return, the tool-seen-but-incomplete return, and the plain "still streaming" return) - use it instead of the fixed "</think>" length - 1 holdback when computing the emit limit Tested: - Full existing ds4_test suite passes unchanged. - New test_anthropic_stream_preserves_reasoning_token_boundaries and test_responses_stream_preserves_reasoning_token_boundaries, mirroring antirez#692's own test: three *_sse_stream_update() calls simulating three decode steps assert reasoning deltas land on the actual per-call text growth ("We need" / " to generate a title"), not a fixed mid-word suffix cut, for both paths. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complements #692 (@rareba) — extends the same fix to the two other paths issue #685 explicitly flagged as having the identical bug, which #692 doesn't touch.
Background
Issue #685 reports streaming reasoning deltas split at an arbitrary fixed byte offset (7 bytes before the end, held back to detect a possibly-split
</think>marker) instead of at actual token boundaries — across three independently-shaped but identical code paths: OpenAI chat/completions, Responses, and Anthropic. #692 already fixes the OpenAI path (its diff is scoped toopenai_sse_stream_update()only): track the end of the previous update's text (reasoning_token_end) and use that as the emit boundary, since production code calls*_sse_stream_update()once per newly decoded token, so the boundary between two calls is always a real token edge.This PR applies the exact same technique, 1:1, to
responses_sse_stream_update()andanthropic_sse_stream_update()— the two other places #685 pointed at (~line 7188 and ~line 8098 respectively). Not a competing fix for the OpenAI path; doesn't touch it at all.Changes
In both
responses_streamandanthropic_stream:reasoning_token_endraw_lenat each point the THINKING branch returns without a close tag or finish (the prefix-check early return, the tool-seen-but-incomplete return, and the plain "still streaming" return)"</think>") - 1holdback when computing the emit limitTesting
ds4_testsuite passes unchanged.test_anthropic_stream_preserves_reasoning_token_boundariesandtest_responses_stream_preserves_reasoning_token_boundaries, mirroring server: preserve reasoning_content token boundaries #692's own test structure: three*_sse_stream_update()calls simulating three decode steps assert reasoning deltas land on the actual per-call text growth ("We need"/" to generate a title"), not a fixed mid-word suffix cut, for both paths.🤖 Generated with Claude Code