Skip to content

Fix reasoning_effort tier mapping and context-gate misreading - #704

Open
Flor1an-B wants to merge 2 commits into
antirez:mainfrom
Flor1an-B:fix-reasoning-effort-high-tier
Open

Fix reasoning_effort tier mapping and context-gate misreading#704
Flor1an-B wants to merge 2 commits into
antirez:mainfrom
Flor1an-B:fix-reasoning-effort-high-tier

Conversation

@Flor1an-B

@Flor1an-B Flor1an-B commented Aug 5, 2026

Copy link
Copy Markdown

Follows up on #635 (@kernelzeroday, @zom-2018) and #653 (@rell666) — this is the patch discussed there, posted as a comment first (#635 (comment)) and opened as a PR per @darkbasic's question.

Two independent bugs, same two functions

  1. DS4_THINK_HIGH never rendered a prefix at all. Only DS4_THINK_MAX did, using text that is actually the official "high" tier prefix per deepseek-ai/DeepSeek-V4-Flash-0731's encoding/README.md (verified verbatim against the primary source). Every reasoning_effort other than "max" silently rendered as official "low". The real "max" prefix ("Beyond maximum — exhaustive, relentless, and uncompromising...", note the em dash U+2014) was absent from the codebase entirely — unreachable via the API regardless of setting. Fixed in both independent prompt-rendering paths that had this bug: ds4.c's chat_push_think_prefix() (CLI/engine) and ds4_server.c's render_deepseek_chat_prompt_text() (server).

  2. ds4_think_mode_for_context() downgraded "max""high" below a 384K context threshold, citing DeepSeek's guidance as an input-context requirement. The model card's actual text: "For the high and max reasoning effort levels, we recommend a maximum output length of 384K tokens." That's an output-budget guideline, not a context-size precondition. Turned into a documented no-op rather than removed outright, since four ds4_server.c call sites use it and ds4_think_max_min_context() may still be useful for client-side output-length advice.

Testing

  • Adds test_render_think_high_prompt_prefix() — there was previously zero test coverage of the "high" tier ever rendering anything.
  • Fixes the now-incorrect context-gate assertion in test_reasoning_effort_mapping().
  • Adds a mutual-exclusivity assertion to the existing "max" prompt test.
  • make ds4_test passes clean against the full existing suite plus these changes.
  • Smoke-tested live against a real 0731 quant: reasoning_effort: high and : max now produce visibly distinct prompts; : max no longer collapses to the "high" text below 384K context.

Requirements

  • I have read and agree with the contributing guidelines.
  • AI usage disclosure: YES — diagnosed and patched with Claude Code, verified against the primary source (DeepSeek's own model card) and the project's existing test suite.

Update: a real DS4_THINK_LOW tier (issue #660)

Found while looking for other reasoning_effort-related community reports to help with: issue #660 ("New 0731 iq2 thinks forever!") describes reasoning_effort: "low" producing 9k tokens of runaway reasoning.

Root cause, again verified against DeepSeek-V4's own encoding spec (encoding/README.md, "Reasoning Effort" section): it defines exactly three levels — "low" (the default, prompt prefix "none"), "high", and "max". DS4's parse_reasoning_effort_name() collapsed "low" (and "medium"/"minimal") into the same DS4_THINK_HIGH enum value as an explicit "high" request — the code even had a comment acknowledging this: "DS4 only exposes HIGH and MAX above zero, so minimal collapses to the smallest non-zero level (HIGH)." So a client asking for "low" got the most elaborate non-max prompt DS4 has, instead of the model's own default behavior with no extra prompting — a very plausible direct cause of runaway reasoning on tasks that don't need it.

Added DS4_THINK_LOW as a genuine fourth ds4_think_mode value ("thinking enabled, no prompt prefix", distinct from DS4_THINK_NONE which disables thinking entirely) and wired it through every switch/branch over the enum. The default reasoning_effort when a client sends none at all is untouched (still high); this only fixes what an explicit "low"/"minimal" request does.

Tested the same way as the rest of this PR: full suite passes, new regression test (test_render_think_low_prompt_prefix) plus updated test_reasoning_effort_mapping, and a live A/B on the production Flash quant confirming reasoning_effort: "low" and "high" now produce genuinely different reasoning_content (they were byte-identical prompts before this fix).

Two independent bugs in the same two functions:

1. DS4_THINK_HIGH never rendered a prefix at all (only DS4_THINK_MAX did,
   using text that is actually the official "high" tier prefix per
   deepseek-ai/DeepSeek-V4-Flash-0731's encoding/README.md). Every
   reasoning_effort other than "max" silently rendered as official "low".
   The real "max" prefix ("Beyond maximum...") was absent from the codebase
   entirely. Fixed in both independent prompt-rendering paths: ds4.c's
   chat_push_think_prefix() (CLI/engine) and ds4_server.c's
   render_deepseek_chat_prompt_text() (server).

2. ds4_think_mode_for_context() downgraded "max"->"high" below a 384K
   context threshold, citing DeepSeek's guidance as an input-context
   requirement. The model card's actual text: "For the high and max
   reasoning effort levels, we recommend a maximum output length of
   384K tokens" -- an output-budget guideline, not a context-size
   precondition. Turned into a documented no-op rather than removed,
   since four ds4_server.c call sites use it and
   ds4_think_max_min_context() may still be useful for client-side
   output-length advice.

Adds test_render_think_high_prompt_prefix() (there was previously zero
coverage of the "high" tier ever rendering anything), fixes the now-wrong
context-gate assertion, and adds a mutual-exclusivity check to the
existing "max" prompt test.

make ds4_test passes clean against the full suite plus these changes.
Issue antirez#660: a user setting reasoning_effort to "low" saw the model reason
for 9k tokens on a simple task and never converge -- "it became qwen3.5 all
of a sudden." Community workaround suggestions (try "max" instead) only
underscored that "low" was doing something wrong, not that "max" was doing
something right.

Root cause, confirmed against DeepSeek-V4's own encoding spec
(encoding/README.md, "Reasoning Effort" section): the spec defines exactly
three levels -- "low" (default, prompt prefix "none"), "high", and "max".
DS4's parse_reasoning_effort_name() collapsed "low" (and "medium"/"minimal")
into DS4_THINK_HIGH, the same enum value as an explicit "high" request,
with an explicit comment acknowledging the simplification: "DS4 only
exposes HIGH and MAX above zero, so minimal collapses to the smallest
non-zero level (HIGH)." So a client asking for "low" got the *most*
elaborate non-max prompt DS4 has ("You MUST be very thorough... rigorously
stress-testing your logic against all potential paths, edge cases, and
adversarial scenarios...") instead of the model's own default behavior with
no extra prompting at all -- a plausible direct cause of runaway reasoning
on tasks that don't need it.

Fix: add DS4_THINK_LOW as a genuine fourth ds4_think_mode value --
"thinking enabled, no prompt prefix" -- distinct from DS4_THINK_NONE
(thinking disabled entirely). Wired through every switch/branch over the
enum (grep confirmed a small, complete list): ds4_think_mode_enabled(),
ds4_think_mode_name(), ds4_glm_reasoning_effort_text(),
chat_push_think_prefix() and its ds4_server.c equivalent in
render_deepseek_chat_prompt_text() (both already no-op for an unmatched
mode, so they needed no code change, just a comment explaining why),
think_mode_from_enabled() (now a straight pass-through instead of collapsing
non-NONE/non-MAX to HIGH), and parse_reasoning_effort_name() ("low" and
"minimal" now map to LOW; "medium"/"xhigh" -- not official DeepSeek values
-- still map to HIGH absent more specific guidance). The default
reasoning_effort when a client sends none at all is untouched (still HIGH);
this only changes what an *explicit* "low"/"minimal" request actually does.

Tested:
- Full existing ds4_test suite passes (compiler catches any missed switch
  case over the enum; there were none beyond the ones listed above).
- Updated test_reasoning_effort_mapping for the new low/minimal -> LOW
  mapping, plus direct checks of ds4_think_mode_enabled()/
  think_mode_from_enabled() on the new value.
- New test_render_think_low_prompt_prefix mirrors the existing high/max
  prompt-prefix regression tests: no prefix text, but the rendered prompt
  still ends in the thinking-enabled "<think>" (not "</think>").
- Live A/B on the production Flash quant, same prompt, temp=0:
  reasoning_effort "low" and "high" now produce genuinely different
  reasoning_content (were byte-identical prompts before this fix, so would
  have produced identical output).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant