Skip to content

server: preserve reasoning_content token boundaries - #692

Draft
rareba wants to merge 1 commit into
antirez:mainfrom
rareba:agent/preserve-reasoning-token-boundaries
Draft

server: preserve reasoning_content token boundaries#692
rareba wants to merge 1 commit into
antirez:mainfrom
rareba:agent/preserve-reasoning-token-boundaries

Conversation

@rareba

@rareba rareba commented Aug 5, 2026

Copy link
Copy Markdown

Summary

  • keep OpenAI reasoning_content SSE deltas aligned to decoded-token boundaries
  • retain one-token look-behind while checking for the closing think marker
  • replace the regression that expected arbitrary byte splitting

Root cause

The thinking stream held a fixed seven-byte suffix to detect a possibly split
closing think marker. Every update therefore emitted the current decoded token
minus seven bytes, then emitted that tail with the next token.

Remembering the previous decoded-token end preserves whole-token deltas while
keeping the close-tag and UTF-8 safeguards intact. The only cost is one token
of reasoning-stream latency.

Impact

Clients that process individual reasoning deltas now receive stable token
units instead of character fragments. Reassembled output is unchanged.

Validation

Windows 11 host, WSL Linux CPU/reference build; no model required:

  • focused server test binary: ds4 tests: ok, server: OK
  • make -j2 cpu

Closes #685

@Flor1an-B

Copy link
Copy Markdown

Nice diagnosis and fix. Since your diff is scoped to the OpenAI path, opened #712 to apply the exact same reasoning_token_end technique to responses_sse_stream_update() and anthropic_sse_stream_update() — the other two places #685 pointed at with the identical shape. Not touching your diff at all, just extending the same idea to the rest of the file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ds4-server reasoning_content deltas fragmented mid-token at arbitrary byte offsets (OpenAI-compatible streaming)

2 participants