improvement(providers): validation pass, and stream tool loop improvements

Author: icecrasher321

.agents/skills/add-model/SKILL.md

@@ -151,8 +151,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

.claude/commands/add-model.md

@@ -150,8 +150,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

.cursor/commands/add-model.md

@@ -145,8 +145,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

apps/docs/content/docs/en/workflows/blocks/agent.mdx

@@ -102,20 +102,19 @@ While an agent runs, Sim can stream its thinking and tool lifecycle live — the
 
 {/* agent-stream-capabilities:begin — generated by `bun run agent-stream-docs:generate`; do not edit between markers */}
 
-Live tool-call chips stream for **Anthropic, Azure Anthropic, Google, Vertex AI, DeepSeek, Groq, AWS Bedrock** models. Other providers run tools without live chips — tool results still appear in the block output when the run completes.
+Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Vertex AI, DeepSeek, Groq, AWS Bedrock** models. Other providers run tools without live chips and project the settled final answer when the run completes; they do not ask the model to regenerate that answer just to create a stream.
 
 | Provider | Streamed thinking | Models |
 |----------|-------------------|--------|
 | OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5-pro`, `gpt-5.5`, `gpt-5.4-pro`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.2-pro`, `gpt-5.2`, `gpt-5.1`, `gpt-5-pro`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o4-mini`, `o3`, `o3-mini`, `o1` |
-| Anthropic | Full thinking deltas | `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, `claude-sonnet-4-5`, `claude-haiku-4-5` |
-| Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `claude-fable-5`, `claude-sonnet-5`, `claude-opus-4-8`, `claude-opus-4-7` |
+| Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `claude-fable-5`, `claude-sonnet-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, `claude-sonnet-4-5`, `claude-haiku-4-5` |
 | Azure OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `azure/gpt-5.4`, `azure/gpt-5.4-mini`, `azure/gpt-5.4-nano`, `azure/gpt-5.2`, `azure/gpt-5.1`, `azure/gpt-5.1-codex`, `azure/gpt-5`, `azure/gpt-5-mini`, `azure/gpt-5-nano`, `azure/o3`, `azure/o4-mini` |
-| Azure Anthropic | Full thinking deltas | `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-opus-4-1`, `azure-anthropic/claude-haiku-4-5` |
+| Azure Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-opus-4-1`, `azure-anthropic/claude-haiku-4-5` |
 | Google | Summaries only | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
 | Vertex AI | Summaries only | `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` |
-| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-chat`, `deepseek-reasoner` |
+| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-reasoner` |
 | Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.6-27b` |
-| Meta | Full thinking deltas | `muse-spark-1.1` |
+| Meta | Not streamed | `muse-spark-1.1` |
 | Kimi | Full thinking deltas | `kimi-k2.6` |
 | Z.ai | Full thinking deltas | `glm-5.2`, `glm-5.1`, `glm-5`, `glm-5-turbo`, `glm-4.7`, `glm-4.6`, `glm-4.5`, `glm-4.5-air` |
 

apps/docs/content/docs/en/workflows/deployment/agent-events.mdx

@@ -12,18 +12,19 @@ Agent blocks can emit more than answer text while they run: **provider-exposed t
 Sim does **not** invent thinking for providers that do not stream it. Bedrock Converse, many OpenAI-compat models, and non-reasoning chat models stay text-only (plus tools when a live tool loop is wired).
 </Callout>
 
-## Dual gate (public chat / simple SSE)
+## Independent event gates (public chat / simple SSE)
 
-Thinking and tool frames leave the public chat or workflow simple-SSE path only when **both** are true:
+Thinking and tool frames each require the stream protocol header plus their own deployment policy:
 
-1. Deployment policy: chat `includeThinking` is enabled (default **off**).
-2. Request opts in with header:
+1. Thinking frames require chat `includeThinking` (default **off**).
+2. Tool lifecycle frames require chat `includeToolCalls` (default **off**).
+3. Both require the request to opt in with:
 
 ```http
 X-Sim-Stream-Protocol: agent-events-v1
 ```
 
-Legacy clients that omit the header keep today’s text-only SSE even if the deployment has thinking enabled. The hosted chat UI always sends the header for its own deployments.
+Legacy clients that omit the header keep today’s text-only SSE even if either deployment policy is enabled. The hosted chat UI always sends the header for its own deployments.
 
 ## Simple SSE frame shapes
 
@@ -44,7 +45,7 @@ Answer text stays on `chunk`. Thinking and tools never reuse `chunk` (so older c
 
 During a live tool loop, the model can’t be classified mid-turn: text it emits may turn out to be the final answer or preamble before a tool call (the stop reason arrives only at turn end).
 
-- **Opted-in clients** (protocol header + `includeThinking`) receive answer text as `chunk` frames **live**, token by token. If the turn then resolves to tool calls, a `chunk_reset` frame tells the client to discard that block’s streamed text — the final turn re-streams live after tools settle. Append `chunk`, honor `chunk_reset`, and the displayed answer always converges to the block’s final content.
+- **Opted-in clients** (protocol header + either event policy) receive answer text as `chunk` frames **live**, token by token. If the turn then resolves to tool calls, a `chunk_reset` frame tells the client to discard that block’s streamed text — the final turn re-streams live after tools settle. Append `chunk`, honor `chunk_reset`, and the displayed answer always converges to the block’s final content.
 - **Legacy clients** (no header) never see provisional text: only settled final-turn text is emitted as `chunk`, delivered when the turn completes.
 
 Logs, memory, and the block’s `content` output always contain final-turn text only — intermediate preamble is never persisted.
@@ -68,9 +69,9 @@ When you click **Run** in the builder, the execution-events SSE path forwards th
 
 The terminal output panel shows Thinking / Tools chrome above the block output when those events arrive. PII redaction on block output still disables live forwarding (executor rule).
 
-## Chat deployment toggle
+## Chat deployment toggles
 
-In **Deploy → Chat**, enable **Include thinking** so public chat can expose thinking/tool frames (still requires the protocol header). Redeploy / update the chat after changing Agent models or tools.
+In **Deploy → Chat**, enable **Include thinking** for provider-exposed thinking and **Include tool calls** for tool names and lifecycle status. The switches are independent, and both event types still require the protocol header. Tool arguments and results are never included in lifecycle frames. Redeploy or update the chat after changing Agent models or tools.
 
 ## Capability honesty (high level)
 
@@ -97,4 +98,4 @@ curl -N -X POST 'http://localhost:3000/api/chat/your-slug' \
   </Tab>
 </Tabs>
 
-See also [Chat deployment](/docs/workflows/deployment/chat) for access control and the Include thinking setting.
+See also [Chat deployment](/docs/workflows/deployment/chat) for access control and the event settings.

apps/docs/content/docs/en/workflows/deployment/chat.mdx

@@ -32,7 +32,8 @@ Configure the following fields, then click **Launch Chat**:
 | **Output** | Output fields from your workflow blocks returned as the chat response. At least one must be selected. |
 | **Welcome Message** | Greeting shown before the user sends their first message. Defaults to `"Hi there! How can I help you today?"`. |
 | **Access Control** | Controls who can access the chat. See [Access Control](#access-control) below. |
-| **Include thinking** | When enabled, the hosted chat can show provider-exposed thinking and tool lifecycle (name + status). Requires the chat client to send `X-Sim-Stream-Protocol: agent-events-v1` (the hosted UI always does). Default is off. See [Agent stream events](/docs/workflows/deployment/agent-events). |
+| **Include thinking** | When enabled, the hosted chat can show provider-exposed thinking. Requires the chat client to send `X-Sim-Stream-Protocol: agent-events-v1` (the hosted UI always does). Default is off. See [Agent stream events](/docs/workflows/deployment/agent-events). |
+| **Include tool calls** | When enabled, the hosted chat can show tool names and lifecycle status independently of thinking. Tool arguments and results are never exposed. Requires the same stream protocol header and defaults to off. |
 
 ### Output Selection
 

apps/sim/app/api/chat/[identifier]/otp/route.ts

@@ -161,6 +161,7 @@ export const PUT = withRouteHandler(
           password: chat.password,
           outputConfigs: chat.outputConfigs,
           includeThinking: chat.includeThinking,
+          includeToolCalls: chat.includeToolCalls,
         })
         .from(chat)
         .where(
@@ -211,6 +212,7 @@ export const PUT = withRouteHandler(
         authType: deployment.authType,
         outputConfigs: deployment.outputConfigs,
         includeThinking: deployment.includeThinking ?? false,
+        includeToolCalls: deployment.includeToolCalls ?? deployment.includeThinking ?? false,
       })
       setChatAuthCookie(response, deployment.id, deployment.authType, deployment.password)