feat(providers): prompt caching capability + usage-based cache pricing (#5922)

* improvement(providers): validation pass, and stream tool loop improvements

* remove deploy options correctly

* fix

* feat(providers): prompt caching capability and usage-based cache pricing

Replace the arbitrary cached-rate heuristic with a single cache-aware pricing
function, and add prompt caching as an opt-in capability for Anthropic.

Pricing: priceModelUsage in cost-policy.ts is now the only place cache
arithmetic happens. Provider adapters normalize their wire shape into
ModelUsage (input always excludes cache buckets); the pricing function never
branches on provider. This removes five divergent behaviors, including the
!!request.context heuristic that gave Router and Evaluator an unearned 10x
input discount, and the overwrite that silently billed Anthropic cache reads
and writes at zero. Also parses OpenAI cache_write_tokens, previously ignored.

Caching: Anthropic gets a capability-gated advanced switch that places
cache_control on the last tool and last system block; system is now always a
TextBlockParam array. OpenAI gets a stable per-block prompt_cache_key with no
UI, since its caching is automatic.

* fix(providers): route OpenAI and Gemini block cost through cache-aware pricing

Cache-aware pricing only reached trace segments. The billable block cost still
called calculateCost on the cache-inclusive prompt total, so OpenAI cache hits
and Gemini implicit-cache hits were charged at the full input rate and GPT-5.6+
cache writes went unbilled.

Both providers now accumulate cache buckets and price through priceModelUsage,
matching the Anthropic token convention where input excludes cache reads and
writes. Cached counts are clamped to the prompt total so an over-reporting
payload cannot bill more input than the request contained.

* fix(streaming): redact tool payloads on selected outputs in public chat

Redaction only ran on the empty-selection branch, but a deployment almost
always selects outputs, so it was dead in the case it exists for. Selecting
toolCalls streamed the raw arguments and results to a public chat client in a
chunk frame, and providerTiming carried thinking content the same way.

Both paths now extract from the sanitized block output rather than the raw log:
the streamed selected output, which is the reachable vector, and the final
envelope. Sanitizing the source rather than per selected path means a newly
selectable field cannot reopen the hole.

* refactor(providers): drop unreachable billing fallbacks

Every provider pricing helper took a policy parameter no caller passed. Worse
than dead: passing one would have double-applied the margin the central layer
already applies. Removed, so providers can only price at list.

Also removed guards that cannot fire. The central fallback normalized cache
buckets no provider can reach it with (all three that report cache usage price
themselves) and did so at a 1x write multiplier no vendor charges.
priceModelUsage re-validated token counts the adapter had already clamped, and
applyModelCostPolicy defaulted a required total field.

Validation now happens once, in the adapter that parses the vendor payload and
is the only layer that knows cache buckets are a subset of the prompt total.

Author: icecrasher321

.agents/skills/add-model/SKILL.md

@@ -151,8 +151,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

.claude/commands/add-model.md

@@ -150,8 +150,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

.cursor/commands/add-model.md

@@ -145,8 +145,8 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
 
 If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
 
-- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
-- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
+- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is `omitted` into `display: 'summarized'` on agent-events runs, so current Claude thinking models use `'summary'`. Use `'full'` only if future official API docs explicitly guarantee raw thinking deltas. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
+- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.
 - After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
 - Include the `streamed` value (with its source URL) in the verification report when set.
 

apps/docs/content/docs/en/workflows/blocks/agent.mdx

@@ -80,8 +80,11 @@ Some settings live under advanced, or appear only for models that support them:
 - **Temperature.** How varied the output is. Stay low (0–0.3) when you need accuracy and repeatability, middle (around 0.5) for everyday work, higher (0.7+) when you want creative variety.
 - **Max output tokens.** Caps the response length. Defaults to the model's full limit.
 - **Reasoning effort / Thinking level.** For models with extended reasoning, how much the model thinks before answering. Higher is more thorough but slower and costs more tokens.
+- **Prompt caching.** For Anthropic Claude models, reuses the system prompt and tool definitions between runs instead of re-reading them every time. Cached input costs a tenth of the normal rate, but writing the cache costs 1.25x, so leave it off for one-off runs and turn it on when the same agent runs repeatedly. The cache covers a prefix only if it reaches 1,024 tokens (2,048 on Haiku) — below that Anthropic ignores it and nothing changes. Entries expire after five minutes of no use.
 - **API key.** Your key for the chosen provider. Hidden on hosted Sim, which supplies one.
 
+OpenAI and Gemini cache automatically at no extra cost and need no setting; their discount is already reflected in what you are charged.
+
 ## Outputs
 
 After the agent runs, later blocks read its result by name:
@@ -102,20 +105,19 @@ While an agent runs, Sim can stream its thinking and tool lifecycle live — the
 
 {/* agent-stream-capabilities:begin — generated by `bun run agent-stream-docs:generate`; do not edit between markers */}
 
-Live tool-call chips stream for **Anthropic, Azure Anthropic, Google, Vertex AI, DeepSeek, Groq, AWS Bedrock** models. Other providers run tools without live chips — tool results still appear in the block output when the run completes.
+Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Vertex AI, DeepSeek, Groq, AWS Bedrock** models. Other providers run tools without live chips and project the settled final answer when the run completes; they do not ask the model to regenerate that answer just to create a stream.
 
 | Provider | Streamed thinking | Models |
 |----------|-------------------|--------|
 | OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5-pro`, `gpt-5.5`, `gpt-5.4-pro`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.2-pro`, `gpt-5.2`, `gpt-5.1`, `gpt-5-pro`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o4-mini`, `o3`, `o3-mini`, `o1` |
-| Anthropic | Full thinking deltas | `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, `claude-sonnet-4-5`, `claude-haiku-4-5` |
-| Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `claude-fable-5`, `claude-sonnet-5`, `claude-opus-5`, `claude-opus-4-8`, `claude-opus-4-7` |
+| Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `claude-fable-5`, `claude-sonnet-5`, `claude-opus-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, `claude-sonnet-4-5`, `claude-haiku-4-5` |
 | Azure OpenAI | Summaries only — Requires OpenAI organization verification; falls back to no summaries. | `azure/gpt-5.4`, `azure/gpt-5.4-mini`, `azure/gpt-5.4-nano`, `azure/gpt-5.2`, `azure/gpt-5.1`, `azure/gpt-5.1-codex`, `azure/gpt-5`, `azure/gpt-5-mini`, `azure/gpt-5-nano`, `azure/o3`, `azure/o4-mini` |
-| Azure Anthropic | Full thinking deltas | `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-opus-4-1`, `azure-anthropic/claude-haiku-4-5` |
+| Azure Anthropic | Summaries only — These generations omit full thinking; Sim requests summarized thinking on streaming runs. | `azure-anthropic/claude-opus-4-6`, `azure-anthropic/claude-opus-4-5`, `azure-anthropic/claude-sonnet-4-5`, `azure-anthropic/claude-opus-4-1`, `azure-anthropic/claude-haiku-4-5` |
 | Google | Summaries only | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
 | Vertex AI | Summaries only | `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` |
-| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-chat`, `deepseek-reasoner` |
+| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-reasoner` |
 | Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.6-27b` |
-| Meta | Full thinking deltas | `muse-spark-1.1` |
+| Meta | Not streamed | `muse-spark-1.1` |
 | Kimi | Full thinking deltas | `kimi-k2.6` |
 | Z.ai | Full thinking deltas | `glm-5.2`, `glm-5.1`, `glm-5`, `glm-5-turbo`, `glm-4.7`, `glm-4.6`, `glm-4.5`, `glm-4.5-air` |
 
@@ -142,6 +144,7 @@ The Agent reads the message from Start with `<start.input>` and returns a result
   { question: "What is the difference between the tool usage controls (Auto, Force, None)?", answer: "In Auto, the model decides when to call a tool based on context. In Force, the model must call the tool on every run. In None, the tool is hidden from the model and never sent, which disables it without removing it from the block." },
   { question: "How does the Response Format work?", answer: "It enforces structured output by providing a JSON Schema. When set, the model's response is constrained to match the schema exactly, and each field is read directly by downstream blocks using <agent.fieldName>. Without a response format, the agent returns its standard outputs: content, model, tokens, and toolCalls." },
   { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI o-series and GPT-5 models) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." },
+  { question: "When should I turn on Prompt Caching?", answer: "Turn it on when the same agent runs repeatedly with a large, stable system prompt or tool set — cached input bills at a tenth of the normal input rate. Leave it off for one-off runs, because writing the cache costs 1.25x and nothing reads it back. The setting appears only for Anthropic Claude models; OpenAI and Gemini cache automatically with no setting and no write fee. Anthropic only caches a prefix of at least 1,024 tokens (2,048 on Haiku), and entries expire after five minutes of no use." },
   { question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." },
   { question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." },
 ]} />