Skip to content

Commit a22dbc5

Browse files
A8Cl233395A8Cl233395
committed
feat: return upstream reasoning as reasoning_content instead of content
BREAKING CHANGE: with INCLUDE_REASONING enabled, upstream reasoning deltas are no longer appended to content. Streaming chunks carry them in delta.reasoning_content and non-streaming responses in message.reasoning_content. When the flag is off, the provider passthrough strips reasoning_content instead of leaving it in the payload. Reasoning content still counts toward the empty-visible-response guard, and assistant messages that carry reasoning_content are forwarded upstream as leading {type:"reasoning"} parts, matching the CommandCode CLI.
1 parent d325ce8 commit a22dbc5

10 files changed

Lines changed: 139 additions & 38 deletions

File tree

‎docs/DEPLOYMENT.ko.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -375,17 +375,17 @@ npm run smoke
375375

376376
### 서버와 client auth 옵션
377377

378-
| 변수 | 기본값 | 설명 |
379-
| -------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------- |
380-
| `HOST` | `127.0.0.1` | bind 주소입니다. local-only면 `127.0.0.1`, Tailscale/VPN/reverse proxy 뒤에서 접근하려면 `0.0.0.0`을 사용합니다. |
381-
| `PORT` | `9992` | HTTP listen port입니다. |
382-
| `BRIDGE_API_KEY` | 미설정 | client-facing bearer key입니다. 강력 권장합니다. Admin endpoint는 이 값이 설정되어 있어야 접근 가능합니다. |
383-
| `REQUEST_BODY_LIMIT_BYTES` | `1048576` | Fastify request body limit입니다. 매우 큰 prompt/tool schema를 받을 때만 늘리십시오. |
384-
| `RATE_LIMIT_MAX` | `60` | client별 rate-limit window 안에서 허용할 최대 request 수입니다. |
385-
| `RATE_LIMIT_WINDOW` | `1 minute` | `@fastify/rate-limit`이 이해하는 window 문자열입니다. |
386-
| `LOG_LEVEL` | `info` | Pino/Fastify log level입니다. 보통 `debug`, `info`, `warn`, `error`, `silent`를 사용합니다. |
387-
| `CORS_ORIGIN` | 미설정 | browser client 특정 origin에 CORS를 열 때 사용합니다. CLI/server client만 쓰면 비워두십시오. |
388-
| `INCLUDE_REASONING` | `false` | `true`면 reasoning delta를 visible content에 붙입니다. 일반 OpenAI-compatible client에서는 `false` 유지가 안전합니다. |
378+
| 변수 | 기본값 | 설명 |
379+
| -------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
380+
| `HOST` | `127.0.0.1` | bind 주소입니다. local-only면 `127.0.0.1`, Tailscale/VPN/reverse proxy 뒤에서 접근하려면 `0.0.0.0`을 사용합니다. |
381+
| `PORT` | `9992` | HTTP listen port입니다. |
382+
| `BRIDGE_API_KEY` | 미설정 | client-facing bearer key입니다. 강력 권장합니다. Admin endpoint는 이 값이 설정되어 있어야 접근 가능합니다. |
383+
| `REQUEST_BODY_LIMIT_BYTES` | `1048576` | Fastify request body limit입니다. 매우 큰 prompt/tool schema를 받을 때만 늘리십시오. |
384+
| `RATE_LIMIT_MAX` | `60` | client별 rate-limit window 안에서 허용할 최대 request 수입니다. |
385+
| `RATE_LIMIT_WINDOW` | `1 minute` | `@fastify/rate-limit`이 이해하는 window 문자열입니다. |
386+
| `LOG_LEVEL` | `info` | Pino/Fastify log level입니다. 보통 `debug`, `info`, `warn`, `error`, `silent`를 사용합니다. |
387+
| `CORS_ORIGIN` | 미설정 | browser client 특정 origin에 CORS를 열 때 사용합니다. CLI/server client만 쓰면 비워두십시오. |
388+
| `INCLUDE_REASONING` | `false` | `true`면 reasoning delta가 `content` 대신 `reasoning_content` 필드로 반환됩니다. 일반 OpenAI-compatible client에서는 `false` 유지가 안전합니다. |
389389

390390
### CommandCode upstream 옵션
391391

‎docs/DEPLOYMENT.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -376,17 +376,17 @@ npm run smoke
376376

377377
### Server and client-auth options
378378

379-
| Variable | Default | Description |
380-
| -------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------- |
381-
| `HOST` | `127.0.0.1` | Bind address. Use `127.0.0.1` for local-only, `0.0.0.0` for Tailscale/VPN/reverse-proxy exposure. |
382-
| `PORT` | `9992` | HTTP listen port. |
383-
| `BRIDGE_API_KEY` | unset | Client-facing bearer key. Strongly recommended; required for admin endpoints. |
384-
| `REQUEST_BODY_LIMIT_BYTES` | `1048576` | Fastify request body limit. Increase only for unusually large prompts/tool schemas. |
385-
| `RATE_LIMIT_MAX` | `60` | Max requests per rate-limit window per client. |
386-
| `RATE_LIMIT_WINDOW` | `1 minute` | Rate-limit window string accepted by `@fastify/rate-limit`. |
387-
| `LOG_LEVEL` | `info` | Pino/Fastify log level. Common values: `debug`, `info`, `warn`, `error`, `silent`. |
388-
| `CORS_ORIGIN` | unset | Enables CORS for a specific browser origin. Leave unset for non-browser clients. |
389-
| `INCLUDE_REASONING` | `false` | If `true`, reasoning deltas are appended to visible content. Keep `false` for normal OpenAI-compatible use. |
379+
| Variable | Default | Description |
380+
| -------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
381+
| `HOST` | `127.0.0.1` | Bind address. Use `127.0.0.1` for local-only, `0.0.0.0` for Tailscale/VPN/reverse-proxy exposure. |
382+
| `PORT` | `9992` | HTTP listen port. |
383+
| `BRIDGE_API_KEY` | unset | Client-facing bearer key. Strongly recommended; required for admin endpoints. |
384+
| `REQUEST_BODY_LIMIT_BYTES` | `1048576` | Fastify request body limit. Increase only for unusually large prompts/tool schemas. |
385+
| `RATE_LIMIT_MAX` | `60` | Max requests per rate-limit window per client. |
386+
| `RATE_LIMIT_WINDOW` | `1 minute` | Rate-limit window string accepted by `@fastify/rate-limit`. |
387+
| `LOG_LEVEL` | `info` | Pino/Fastify log level. Common values: `debug`, `info`, `warn`, `error`, `silent`. |
388+
| `CORS_ORIGIN` | unset | Enables CORS for a specific browser origin. Leave unset for non-browser clients. |
389+
| `INCLUDE_REASONING` | `false` | If `true`, reasoning deltas return in the `reasoning_content` field instead of `content`. Keep `false` for normal OpenAI-compatible use. |
390390

391391
### CommandCode upstream options
392392

‎src/converter.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,9 @@ function convertMessages(messages: OpenAIChatMessage[]): CommandCodeMessage[] {
160160

161161
if (message.role === "assistant") {
162162
const content: CommandCodeContentPart[] = [];
163+
const reasoning =
164+
typeof message.reasoning_content === "string" ? message.reasoning_content : "";
165+
if (reasoning.length > 0) content.push({ type: "reasoning", text: reasoning });
163166
const text = flattenOpenAIContent(message.content).trim();
164167
if (text.length > 0) content.push(...asTextContent(text));
165168

‎src/openai.ts‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,7 @@ export async function collectOpenAICompletion(
359359
options: CollectOpenAICompletionOptions,
360360
): Promise<OpenAIChatCompletion> {
361361
let content = "";
362+
let reasoningContent = "";
362363
const toolCalls: OpenAIToolCall[] = [];
363364
let usage: CommandCodeUsage | undefined;
364365
let finishReason: string | undefined;
@@ -373,7 +374,7 @@ export async function collectOpenAICompletion(
373374
content += event.text;
374375
} else if (options.includeReasoning && isReasoningDelta(event)) {
375376
sawCompletionSignal = true;
376-
content += event.text;
377+
reasoningContent += event.text;
377378
} else if (isToolCallEvent(event)) {
378379
sawCompletionSignal = true;
379380
for (const toolCall of openAIToolCallsFromCommandCodeEvent(
@@ -398,15 +399,15 @@ export async function collectOpenAICompletion(
398399
if (
399400
shouldFailEmptyVisibleResponse({
400401
policy: options.emptyVisibleResponsePolicy,
401-
visibleContentLength: content.length,
402+
visibleContentLength: content.length + reasoningContent.length,
402403
toolCallCount: toolCalls.length,
403404
finishReason: finalReason,
404405
})
405406
) {
406407
throw new CommandCodeEmptyVisibleResponseError({
407408
model: options.model,
408409
finishReason: "length",
409-
visibleContentLength: content.length,
410+
visibleContentLength: content.length + reasoningContent.length,
410411
toolCallCount: toolCalls.length,
411412
});
412413
}
@@ -415,6 +416,7 @@ export async function collectOpenAICompletion(
415416
role: "assistant",
416417
content: toolCalls.length > 0 && content.length === 0 ? null : content,
417418
};
419+
if (reasoningContent.length > 0) message.reasoning_content = reasoningContent;
418420
if (toolCalls.length > 0) message.tool_calls = toolCalls;
419421

420422
return {
@@ -508,7 +510,7 @@ export async function* streamOpenAIChunks(
508510
options.id,
509511
options.created,
510512
options.model,
511-
{ content: event.text },
513+
{ reasoning_content: event.text },
512514
null,
513515
options.includeUsage,
514516
),

‎src/provider-chat.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,10 +133,16 @@ export async function handleProviderChat(options: {
133133
const message = firstChoice && isRecord(firstChoice.message) ? firstChoice.message : undefined;
134134
const content = typeof message?.content === "string" ? message.content : "";
135135
const toolCalls = Array.isArray(message?.tool_calls) ? message.tool_calls : [];
136+
const reasoningContent =
137+
typeof message?.reasoning_content === "string" ? message.reasoning_content : "";
138+
if (!config.includeReasoning && message && typeof message.reasoning_content === "string") {
139+
delete message.reasoning_content;
140+
}
136141
if (
137142
config.emptyVisibleResponsePolicy === "error_on_length" &&
138143
firstChoice?.finish_reason === "length" &&
139144
content.length === 0 &&
145+
reasoningContent.length === 0 &&
140146
toolCalls.length === 0
141147
) {
142148
reply.code(502).send({

‎src/provider.ts‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -260,13 +260,7 @@ export class CommandCodeProviderSseTransform extends Transform {
260260
const firstChoice = Array.isArray(json.choices) ? json.choices[0] : undefined;
261261
const delta =
262262
isRecord(firstChoice) && isRecord(firstChoice.delta) ? firstChoice.delta : undefined;
263-
if (
264-
this.includeReasoning &&
265-
delta &&
266-
typeof delta.reasoning_content === "string" &&
267-
delta.reasoning_content.length > 0
268-
) {
269-
delta.content = `${typeof delta.content === "string" ? delta.content : ""}${delta.reasoning_content}`;
263+
if (delta && typeof delta.reasoning_content === "string" && !this.includeReasoning) {
270264
delete delta.reasoning_content;
271265
}
272266
return `data: ${JSON.stringify(json)}\n`;

‎src/server.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ const messageSchema = z.object({
8080
name: z.string().optional(),
8181
tool_call_id: z.string().optional(),
8282
tool_calls: z.array(toolCallSchema).optional(),
83+
reasoning_content: z.string().nullish(),
8384
});
8485

8586
const chatCompletionRequestSchema = z.object({

‎src/types.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ export interface OpenAIChatMessage {
3131
name?: string;
3232
tool_call_id?: string;
3333
tool_calls?: OpenAIToolCall[];
34+
reasoning_content?: string | null;
3435
}
3536

3637
export interface OpenAIFunctionTool {
@@ -101,9 +102,15 @@ export interface CommandCodeImagePart {
101102
mimeType: string;
102103
}
103104

105+
export interface CommandCodeReasoningPart {
106+
type: "reasoning";
107+
text: string;
108+
}
109+
104110
export type CommandCodeContentPart =
105111
| OpenAITextContentPart
106112
| CommandCodeImagePart
113+
| CommandCodeReasoningPart
107114
| CommandCodeToolCallPart
108115
| CommandCodeToolResultPart;
109116

@@ -254,6 +261,7 @@ export interface OpenAIChatCompletion {
254261
message: {
255262
role: "assistant";
256263
content: string | null;
264+
reasoning_content?: string;
257265
tool_calls?: OpenAIToolCall[];
258266
};
259267
finish_reason: "stop" | "length" | "tool_calls" | "content_filter" | null;

‎tests/converter.test.ts‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -502,4 +502,73 @@ describe("OpenAI to CommandCode conversion", () => {
502502
});
503503
expect(body.params.system).toMatch(/valid JSON object/i);
504504
});
505+
506+
it("forwards assistant reasoning_content as a leading reasoning part", () => {
507+
const body = buildCommandCodeGenerateBody({
508+
request: {
509+
model: "deepseek/deepseek-v4.1-flash",
510+
messages: [
511+
{ role: "user", content: "Remember the number 7391." },
512+
{
513+
role: "assistant",
514+
content: "Got it.",
515+
reasoning_content: "The secret number is 7391. I must remember it.",
516+
},
517+
{ role: "user", content: "What is the secret number?" },
518+
],
519+
},
520+
upstreamModel: "deepseek/deepseek-v4.1-flash",
521+
});
522+
523+
expect(body.params.messages[1]).toEqual({
524+
role: "assistant",
525+
content: [
526+
{ type: "reasoning", text: "The secret number is 7391. I must remember it." },
527+
{ type: "text", text: "Got it." },
528+
],
529+
});
530+
});
531+
532+
it("keeps reasoning alongside tool calls and skips empty reasoning_content", () => {
533+
const body = buildCommandCodeGenerateBody({
534+
request: {
535+
model: "deepseek/deepseek-v4.1-flash",
536+
messages: [
537+
{ role: "user", content: "Check the time." },
538+
{
539+
role: "assistant",
540+
content: null,
541+
reasoning_content: "I should call get_time first.",
542+
tool_calls: [
543+
{
544+
id: "call_time",
545+
type: "function",
546+
function: { name: "get_time", arguments: "{}" },
547+
},
548+
],
549+
},
550+
{ role: "tool", tool_call_id: "call_time", content: "12:00" },
551+
{ role: "assistant", content: "It is noon.", reasoning_content: null },
552+
{ role: "assistant", content: "Now.", reasoning_content: "" },
553+
],
554+
},
555+
upstreamModel: "deepseek/deepseek-v4.1-flash",
556+
});
557+
558+
expect(body.params.messages[1]).toEqual({
559+
role: "assistant",
560+
content: [
561+
{ type: "reasoning", text: "I should call get_time first." },
562+
{ type: "tool-call", toolCallId: "call_time", toolName: "get_time", input: {} },
563+
],
564+
});
565+
expect(body.params.messages[3]).toEqual({
566+
role: "assistant",
567+
content: [{ type: "text", text: "It is noon." }],
568+
});
569+
expect(body.params.messages[4]).toEqual({
570+
role: "assistant",
571+
content: [{ type: "text", text: "Now." }],
572+
});
573+
});
505574
});

‎tests/provider.test.ts‎

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -216,10 +216,28 @@ describe("provider chat request shaping", () => {
216216
expect(body.stream).toBe(true);
217217
expect(body.max_tokens).toBe(10);
218218
});
219+
220+
it("passes through assistant reasoning_content messages unchanged", () => {
221+
const messages = [
222+
{ role: "user" as const, content: "hi" },
223+
{
224+
role: "assistant" as const,
225+
content: "Got it.",
226+
reasoning_content: "The secret number is 7391.",
227+
},
228+
];
229+
const body = buildProviderChatRequestBody(
230+
{ model: "default", messages },
231+
"deepseek/deepseek-v4-pro",
232+
);
233+
234+
expect(body.messages).toEqual(messages);
235+
expect(body.messages[1]?.reasoning_content).toBe("The secret number is 7391.");
236+
});
219237
});
220238

221239
describe("provider SSE transform", () => {
222-
it("rewrites the public model id and folds reasoning deltas into content", async () => {
240+
it("rewrites the public model id and keeps reasoning deltas as reasoning_content", async () => {
223241
const out = await transformSse(
224242
[
225243
'data: {"id":"x","model":"deepseek/deepseek-v4-pro","choices":[{"delta":{"content":"h"}}]}',
@@ -240,19 +258,19 @@ describe("provider SSE transform", () => {
240258

241259
expect(out).toContain('"model":"deepseek-v4-pro"');
242260
expect(out).toContain('"content":"h"');
243-
expect(out).toContain('"content":"think "');
261+
expect(out).toContain('"reasoning_content":"think "');
262+
expect(out).not.toContain('"content":"think "');
244263
expect(out).toContain('"content":"i"');
245264
expect(out).toContain("data: [DONE]");
246-
expect(out).not.toContain("reasoning_content");
247265
expect(out).not.toContain('"model":"deepseek/deepseek-v4-pro"');
248266
});
249267

250-
it("leaves reasoning deltas untouched when includeReasoning is disabled", async () => {
268+
it("strips reasoning deltas when includeReasoning is disabled", async () => {
251269
const out = await transformSse(
252270
'data: {"model":"m","choices":[{"delta":{"reasoning_content":"think"}}]}\n\ndata: [DONE]\n\n',
253271
{ publicModel: "p", includeReasoning: false },
254272
);
255-
expect(out).toContain('"reasoning_content":"think"');
273+
expect(out).not.toContain('"reasoning_content":"think"');
256274
expect(out).toContain('"model":"p"');
257275
});
258276

0 commit comments

Comments
 (0)