fix: preserve Anthropic thinking blocks and signatures in LiteLLM round-trip#4999
Open
giulio-leone wants to merge 5 commits intogoogle:mainfrom
Open
fix: preserve Anthropic thinking blocks and signatures in LiteLLM round-trip#4999giulio-leone wants to merge 5 commits intogoogle:mainfrom
giulio-leone wants to merge 5 commits intogoogle:mainfrom
Conversation
giulio-leone
force-pushed
the
fix/litellm-anthropic-thinking-roundtrip
branch
from
4cc8546 to
10575cb
Compare
giulio-leone
force-pushed
the
fix/litellm-anthropic-thinking-roundtrip
branch
from
10575cb to
b6bbf5c
Compare
…nd-trip When using Claude models through LiteLLM, extended thinking blocks (with signatures) were lost after the first turn because: 1. _extract_reasoning_value() only read reasoning_content (flattened string without signatures), ignoring thinking_blocks 2. _content_to_message_param() set reasoning_content on the outgoing message, which LiteLLM's anthropic_messages_pt() template silently drops This fix: - Adds _is_anthropic_provider() helper to detect anthropic/bedrock/ vertex_ai providers - Updates _extract_reasoning_value() to prefer thinking_blocks (with per-block signatures) over reasoning_content - Updates _convert_reasoning_value_to_parts() to handle ChatCompletionThinkingBlock dicts, preserving thought_signature - Updates _content_to_message_param() to embed thinking blocks directly in the message content list for Anthropic providers, bypassing the broken reasoning_content path Fixes google#4801
Cover the model-driven LiteLLM Anthropic round-trip with a regression test.
giulio-leone
force-pushed
the
fix/litellm-anthropic-thinking-roundtrip
branch
from
b6bbf5c to
3f28b1d
Compare
rohityan
added
the
needs review
Collaborator
|
Hi @giulio-leone , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share. |
Collaborator
|
Hi @wukath , can you please review this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #4801 — Adaptive thinking is broken when using Claude models through LiteLLM.
Root Cause
When Claude produces extended thinking with
thinking_blocks(each containing atype,thinkingtext, andsignature), the round-trip through ADK's LiteLLM integration silently loses them:_extract_reasoning_value()only readreasoning_content(a flattened string without signatures), ignoring the richerthinking_blocksfield_content_to_message_param()setreasoning_contenton the outgoingChatCompletionAssistantMessage, but LiteLLM'santhropic_messages_pt()prompt template silently drops thereasoning_contentfield entirelyFix
Three coordinated changes in
lite_llm.py:_is_anthropic_provider()helperanthropic,bedrock,vertex_aiproviders_extract_reasoning_value()thinking_blocks(with per-block signatures) overreasoning_content_convert_reasoning_value_to_parts()ChatCompletionThinkingBlockdicts, preservingthought_signature_content_to_message_param()contentlist as{"type": "thinking", ...}dicts — this format passes through LiteLLM'santhropic_messages_pt()correctlyFor non-Anthropic providers (OpenAI, etc.), behavior is unchanged —
reasoning_contentis still used.Verification
anthropic_messages_pt()was tested to confirm:reasoning_contentfield → DROPPED (existing LiteLLM bug)contentas list with{"type": "thinking", ...}→ PRESERVED ✅Tests
Added 7 targeted tests covering:
_is_anthropic_provider()— provider detection_extract_reasoning_value()— prefersthinking_blocksoverreasoning_content_convert_reasoning_value_to_parts()— signature preservation from block dicts_convert_reasoning_value_to_parts()— plain string fallback (no signature)_content_to_message_param()— Anthropic: thinking blocks embedded in content list_content_to_message_param()— OpenAI: reasoning_content field used (unchanged)_content_to_message_param()— Anthropic thinking + tool calls combinedFull test suite: 4732 passed, 0 failures