feat(friendli): add reasoning params support#886
Conversation
…_template_kwargs, parse_reasoning - Add supportsReasoningEffort + reasoningEffort to GLM-5.x model catalog entries - Add supportsReasoningBinary to MiniMax-M2.5 model catalog entry - Add 'ultracode' to supportsReasoningEffort schema in model.ts - Override createStream + completePrompt in FriendliHandler - buildFriendliReasoningParams(): injects Friendli-specific reasoning params - Controllable (GLM-5.x): reasoning_effort + chat_template_kwargs + parse_reasoning - Always-reasoning (MiniMax-M2.5): parse_reasoning only - Non-reasoning: no extra params - Add 8 reasoning-specific test cases (29 total)
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughFriendli model metadata now describes reasoning capabilities. The provider builds model-specific reasoning parameters for streaming and non-streaming requests, handles Friendli response errors, and adds tests for request fields and streamed reasoning output. ChangesFriendli reasoning support
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant FriendliHandler
participant FriendliAPI
participant StreamConsumer
FriendliHandler->>FriendliAPI: send reasoning parameters and messages
FriendliAPI-->>FriendliHandler: return response or stream chunks
FriendliHandler-->>StreamConsumer: emit reasoning and text chunks
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/api/providers/__tests__/friendli.spec.ts`:
- Around line 397-583: The Friendli reasoning tests lack coverage for default
model settings and explicit reasoning disablement. Extend the “Friendli-specific
reasoning params” suite with a GLM-5.2 case omitting both enableReasoningEffort
and reasoningEffort, asserting the model default enables reasoning with the
expected reasoning_effort and chat_template_kwargs, and a disabled-reasoning
case asserting chat_template_kwargs.enable_thinking is explicitly false while
reasoning parameters remain disabled.
In `@src/api/providers/friendli.ts`:
- Around line 87-121: Update buildFriendliReasoningParams so
controllable-reasoning models receive chat_template_kwargs with enable_thinking
false when reasoningEffort is disabled or set to none, including the
early-return path. Preserve the existing empty-params behavior for models
without controllable reasoning support, and keep enable_thinking true for active
reasoning requests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b8feab0f-3a58-41ff-9a11-bee8c5dbdc20
📒 Files selected for processing (3)
packages/types/src/providers/friendli.tssrc/api/providers/__tests__/friendli.spec.tssrc/api/providers/friendli.ts
…en reasoning is off
GLM-5.x Jinja template defaults enable_thinking to true, so omitting
chat_template_kwargs when user sets none/disable leaves reasoning active.
Now sends { enable_thinking: false } to prevent burning reasoning tokens
against user intent. Also adds default model settings test per CodeRabbit.
taltas
left a comment
There was a problem hiding this comment.
Thanks for the PR. A couple of questons on some of the models I found.
| outputPrice: 4.4, | ||
| cacheWritesPrice: 0, | ||
| cacheReadsPrice: 0.26, | ||
| supportsReasoningEffort: ["minimal", "low", "medium", "high", "xhigh", "max"], |
There was a problem hiding this comment.
Does GLM offer a way to disable reasoning effort completely using 'disable' as a key?
| outputPrice: 1.2, | ||
| cacheWritesPrice: 0, | ||
| cacheReadsPrice: 0.06, | ||
| supportsReasoningBinary: true, |
There was a problem hiding this comment.
On MiniMax-M2.5 This will render the checkbox Use reasoning in the settings.... but the thing is, this model always has reasoning on though right? We should introduce or reuse metadata that distinguishes required/always reasoning from optional binary reasoning, hide or disable the on/off control for this model, and always request parsed reasoning for MiniMax-M2.5. In addition to this, we should add tests for MiniMax with the reasoning flag false and unset, not only true.
I think these changes should be in a separate PR in order not to bloat this one
Summary
Add Friendli-specific reasoning parameter support for GLM-5.x and MiniMax-M2.5 models served via Friendli Model APIs.
Friendli extends the OpenAI Chat Completions API with non-standard reasoning fields (
reasoning_effort,chat_template_kwargs,parse_reasoning,include_reasoning) that the baseBaseOpenAiCompatibleProviderdoes not handle. This PR overridescreateStreamandcompletePromptinFriendliHandlerto inject these parameters.Ref: https://friendli.ai/docs/guides/reasoning
Changes
Model catalog (
packages/types/src/providers/friendli.ts)supportsReasoningEffort: ["minimal","low","medium","high","xhigh","max"]+reasoningEffort: "high"(controllable reasoning models)supportsReasoningBinary: true(always-reasoning model)Handler (
src/api/providers/friendli.ts)createStreamandcompletePromptto inject Friendli reasoning params viabuildFriendliReasoningParams()reasoning_effort+chat_template_kwargs: { enable_thinking: true }+parse_reasoning/include_reasoningwhen user selects a valid effortparse_reasoning/include_reasoningonly"none"or"disable", orenableReasoningEffort=false, no Friendli reasoning params are sent — the model runs at its default behaviordelta.reasoning_contentstreaming already handled byextractReasoningFromDeltain base classTests (
src/api/providers/__tests__/friendli.spec.ts)Design decisions
enable_thinking: false: When reasoning is disabled (none/disable), we send nothing rather than explicitly disabling thinking. This lets the model use its default behavior.ultracode: Friendli API supportsultracodein the enum but we do not surface it — only the standard efforts up tomax.preserveReasoning: Friendli only supports stateless responses API — multi-turn reasoning re-submission has no effect.Test plan
pnpm vitest run api/providers/__tests__/friendli.spec.ts— 29/29 passpnpm run build— successpnpm run lint(src + packages/types) — cleanSummary by CodeRabbit