fix(litellm): preserve reasoning_content for known reasoning model families

[daewoongoh]

Related GitHub Issue

Closes: #891

Description

LiteLLM's /v1/model/info response never exposes reasoning-related capability flags, so preserveReasoning was always resolving to false for reasoning/thinking models routed through a LiteLLM proxy (DeepSeek-Reasoner, GLM-4.7/5, Kimi-K2-Thinking, MiniMax-M2/M3, Qwen3-plus/max, etc.). As a result, LiteLLMHandler.createMessage always used convertToOpenAiMessages, so reasoning_content from a prior turn was never echoed back to the API after a tool-call turn. This differs from accessing the same underlying models directly through their native provider config (e.g. deepseek.ts already sets preserveReasoning: true), and some providers reject the request with a 400 error when required reasoning history is missing.

How:

• packages/types/src/providers/lite-llm.ts: Collected the alias/routed-model-name patterns for every model family that requires preserveReasoning: true in its native provider config (deepseek.ts, mimo.ts, moonshot.ts, bedrock.ts, fireworks.ts, zai.ts, minimax.ts, opencode-go.ts), and exported them as LITELLM_PRESERVE_REASONING_PATTERN.
• src/api/providers/fetchers/litellm.ts: getLiteLLMModels now concatenates model_name and litellm_params.model and tests them against that pattern while parsing each model, setting ModelInfo.preserveReasoning: true on a match.
• src/api/providers/lite-llm.ts: createMessage now branches on info.preserveReasoning — true routes through convertToR1Format (with mergeToolResultText: true), otherwise it keeps using convertToOpenAiMessages. convertToR1Format folds any text following a tool_result into the preceding tool message instead of inserting a separate user message, which prevents reasoning-mode providers from dropping reasoning_content when a user message appears mid-turn.

Trade-off: the matching is a string-pattern heuristic. Since LiteLLM never surfaces the routed model's actual capabilities, this can't be fully accurate — unrecognized aliases or renamed deployments simply won't match (documented in the lite-llm.ts comment).

Test Procedure

• pnpm vitest run (unit tests):
  • packages/types/src/__tests__/lite-llm.test.ts: verifies LITELLM_PRESERVE_REASONING_PATTERN matches/doesn't match each model family (DeepSeek, MiMo, Kimi-K2, MiniMax-M2/M3, GLM-4.7/5, Qwen3).
  • src/api/providers/fetchers/__tests__/litellm.spec.ts: verifies getLiteLLMModels sets preserveReasoning: true when either the alias (model_name) or the routed model name (litellm_params.model) matches, and omits the field otherwise.
  • src/api/providers/__tests__/lite-llm.spec.ts: verifies convertToR1Format is used and reasoning_content survives on the outgoing request when preserveReasoning: true; and that the non-preserveReasoning branch sends no reasoning_content (reflecting that Task.buildCleanConversationHistory already strips the reasoning block upstream in that case).
• Manual repro: configure the LiteLLM provider with litellm_params.model routed to a reasoning model (e.g. deepseek/deepseek-reasoner), run a multi-turn task involving a tool call — previously the follow-up request dropped reasoning_content; it's now preserved.

Pre-Submission Checklist

• Issue Linked: (fill in once the issue is created)
• Scope: Limited to LiteLLM's reasoning_content preservation.
• Self-Review: Done.
• Testing: Added unit tests across 3 files.
• Documentation Impact: None (internal behavior change, no user-facing settings changed).
• Contribution Guidelines: Reviewed.

Documentation Updates

• No documentation updates are required.

Additional Notes

If a reasoning model family or renamed alias isn't caught by this heuristic, additional patterns can be appended to LITELLM_PRESERVE_REASONING_FRAGMENTS.

Get in Touch

hehegwk_23849

Summary by CodeRabbit

• New Features

  • Added automatic reasoning preservation for supported LiteLLM-routed models.
  • Improved conversation formatting for reasoning-enabled models, including preservation of reasoning content and tool-result handling.
  • Added support for detecting reasoning capabilities through model names and aliases.

• Bug Fixes

  • Prevented unsupported model variants from being incorrectly treated as reasoning-enabled.
  • Preserved existing message formatting behavior for models without reasoning support.

• Tests

  • Added coverage for model detection, aliases, exclusions, and reasoning-aware message conversion.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 973ce162-1a90-40b7-828b-581f8a09624a

📥 Commits

Reviewing files that changed from the base of the PR and between 50801da and ce38822.

📒 Files selected for processing (6)

• packages/types/src/__tests__/lite-llm.test.ts
• packages/types/src/providers/lite-llm.ts
• src/api/providers/__tests__/lite-llm.spec.ts
• src/api/providers/fetchers/__tests__/litellm.spec.ts
• src/api/providers/fetchers/litellm.ts
• src/api/providers/lite-llm.ts

---

📝 Walkthrough

Walkthrough

LiteLLM model aliases are now matched against known reasoning families. Matching models receive preserveReasoning, causing message conversion to retain reasoning content and merge tool results using R1 formatting.

Changes

LiteLLM reasoning preservation

Layer / File(s)	Summary
Reasoning alias pattern packages/types/src/providers/lite-llm.ts, packages/types/src/__tests__/lite-llm.test.ts	Defines and tests the case-insensitive pattern for supported reasoning-model aliases and excluded variants.
Model capability inference src/api/providers/fetchers/litellm.ts, src/api/providers/fetchers/__tests__/litellm.spec.ts	Infers preserveReasoning: true from model names and routed aliases when returning LiteLLM models.
Reasoning-aware message conversion src/api/providers/lite-llm.ts, src/api/providers/__tests__/lite-llm.spec.ts	Uses R1-format conversion with tool-result merging for reasoning models and retains the existing OpenAI conversion otherwise. Tests verify reasoning content and message shapes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant getLiteLLMModels
  participant LiteLLMHandler
  participant convertToR1Format
  participant LiteLLM API
  getLiteLLMModels->>getLiteLLMModels: Match model name and routed alias
  getLiteLLMModels-->>LiteLLMHandler: Return preserveReasoning metadata
  LiteLLMHandler->>convertToR1Format: Convert messages and merge tool results
  convertToR1Format-->>LiteLLMHandler: Return reasoning-preserving messages
  LiteLLMHandler->>LiteLLM API: Send converted conversation

Loading

Possibly related PRs

• Zoo-Code-Org/Zoo-Code#892: Both changes use R1-format conversion and test LiteLLM reasoning-content preservation.

Suggested labels: awaiting-review

Suggested reviewers: taltas, edelauna, hannesrudolph, navedmerchant, jamesrobert20

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the LiteLLM reasoning_content preservation fix and matches the implemented changes.
Description check	✅ Passed	The description mostly follows the template, includes the linked issue, implementation details, testing, and checklist items.
Linked Issues check	✅ Passed	The changes address #891 by inferring preserveReasoning for LiteLLM reasoning models and preserving reasoning_content through tool-call turns.
Out of Scope Changes check	✅ Passed	The code changes and added tests stay focused on LiteLLM reasoning preservation and do not introduce unrelated scope.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!