Skip to content

Commit e00b02d

Browse files
Merge remote-tracking branch 'origin/staging' into feat/cli-auth-key
# Conflicts: # package.json # scripts/check-api-validation-contracts.ts
2 parents 9db7998 + 5a8d219 commit e00b02d

402 files changed

Lines changed: 77914 additions & 4547 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/add-model/SKILL.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Use a precise WebFetch prompt: *"Extract for {model_id}: exact model id string,
5252
| `reasoningEffort` | `openai/core.ts`, `azure-openai`, `anthropic/core.ts` (mapped to thinking), `gemini/core.ts` | **Dead on xai, deepseek, mistral, groq, cerebras, openrouter, fireworks, bedrock, vertex** unless their core consumes it — re-grep before assuming |
5353
| `verbosity` | `openai/core.ts`, `azure-openai/index.ts` only | Dead elsewhere |
5454
| `thinking` | `anthropic/core.ts`, `gemini/core.ts` | Dead elsewhere |
55+
| `thinking.streamed` | Docs generator + `getThinkingStreamVisibility` (`models.ts`); `anthropic/core.ts` uses `'summary'` to request `display: 'summarized'` on agent-events runs | **Mandatory on Anthropic-family thinking models** (`agent-stream-docs:check` fails without it); other families fall back to provider defaults |
5556
| `nativeStructuredOutputs` | `anthropic/core.ts`, `fireworks/index.ts`, `openrouter/index.ts` | Dead on openai, xai, google, vertex, bedrock, azure-openai, deepseek, mistral, groq, cerebras |
5657
| `maxOutputTokens` | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
5758
| `computerUse` | `anthropic/core.ts` | Dead elsewhere |
@@ -146,6 +147,15 @@ If anything matches, run the affected provider tests and update assertions as ne
146147

147148
The Consumption Matrix (Step 2) tells you which capability *flags* are honored by existing provider code. But if the new model needs **net-new** request handling that the provider doesn't implement yet — a new beta header (e.g. Anthropic's `anthropic-beta` structured-outputs header in `anthropic/index.ts`), a new thinking/reasoning encoding, a Responses-API quirk — you must edit `apps/sim/providers/<provider>/core.ts` / `index.ts`. Setting a flag whose behavior isn't implemented is a silent no-op. When you do edit provider code, reuse the shared helpers rather than hand-rolling: streaming responses are assembled via `createStreamingExecution` (`@/providers/streaming-execution`) and tool schemas via `adaptOpenAIChatToolSchema` / `adaptAnthropicToolSchema` (`@/providers/tool-schema-adapter`).
148149

150+
### Thinking/reasoning models: `streamed` visibility + generated docs
151+
152+
If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
153+
154+
- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
155+
- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
156+
- After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
157+
- Include the `streamed` value (with its source URL) in the verification report when set.
158+
149159
### Wrong family entirely?
150160

151161
- **Embedding or rerank model** → it does NOT go in the `models[]` array. Use `EMBEDDING_MODEL_PRICING` / `RERANK_MODEL_PRICING` in `models.ts` instead.
@@ -155,6 +165,7 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
155165

156166
```bash
157167
bun run lint
168+
bun run agent-stream-docs:generate # only when the entry has thinking/reasoningEffort
158169
```
159170

160171
Lint must pass before reporting done. **If lint fails:** read the error, fix the syntax/typing issue in the entry you just wrote (do not delete the entry — it's the work product), re-run lint, and note the fix in a "Lint adjustments" line in the verification report. Never report done with lint failing.
@@ -201,6 +212,7 @@ Omitting a field is **not the same as verifying it**. Any field you cannot confi
201212
- ❌ Trusting a marketing email (xAI's grok-4.3 email claimed "3 reasoning efforts" but the API rejects `reasoning_effort` — verified by official docs only)
202213
- ❌ Setting `nativeStructuredOutputs: true` on xai/openai/google (dead — only anthropic/fireworks/openrouter consume it)
203214
- ❌ Setting `thinking` on non-Anthropic/non-Gemini providers
215+
- ❌ Adding an Anthropic-family thinking model without `thinking.streamed` (CI `agent-stream-docs:check` fails), or skipping `bun run agent-stream-docs:generate` after adding any thinking/reasoning model
204216
- ❌ Setting `verbosity` on anything other than OpenAI gpt-5.x
205217
- ❌ Copying `pricing.updatedAt` from a sibling instead of using today's date
206218
- ❌ Inventing a `cachedInput` price by dividing input by 4 (varies by provider — find an explicit number)

.agents/skills/ship/SKILL.md

Lines changed: 41 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,47 @@ When the user runs `/ship`:
3535
5. **Run migration safety** — only if the diff touches `packages/db/migrations/**` or `packages/db/schema.ts`:
3636
- Run `/db-migrate` to review the migration for zero-downtime safety (expand/contract phasing, backward-compatibility with the deployed app version).
3737
- `bun run check:migrations origin/staging` must pass (staging is the PR base). Do not silence a flagged statement with a `-- migration-safe:` annotation unless `/db-migrate` confirmed the old code no longer depends on it; otherwise split the destructive change into a later deploy.
38-
6. **Run pre-ship checks** from the repo root before staging:
39-
- `bun run lint` to fix formatting issues
40-
- `bun run check:api-validation:strict` to catch boundary contract failures before CI
41-
7. **Stage and commit** the changes with the generated message
38+
6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `Lint and Test` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
39+
40+
**Phase A — regenerate the always-in-repo committed artifacts (parallel), then let step 7 stage whatever changed.** Regenerate only the generators whose inputs live entirely in this repo and that any ordinary code change can drift — `agent-stream-docs:generate` (derives from the provider model registry) and `skills:sync` (derives from `.agents/skills/**`). They write disjoint trees (`apps/docs/…/agent.mdx` vs the `.claude`/`.cursor` command projections), so they parallelize safely, and each is idempotent (a no-op when already in sync):
41+
```bash
42+
rm -f /tmp/ship-gen-results
43+
for g in agent-stream-docs:generate skills:sync; do
44+
( bun run "$g" >"/tmp/ship-gen-${g//:/-}.log" 2>&1; echo "$? $g" >>/tmp/ship-gen-results ) &
45+
done
46+
wait
47+
# any non-zero line is a FAILED generator — read /tmp/ship-gen-<name>.log and fix before shipping;
48+
# a silently-failed generate leaves a stale artifact that Phase B / CI then rejects.
49+
# The `exit 1` makes this block itself exit non-zero on failure, so anything gating on the
50+
# command's status (an agent, or a wrapping script) actually stops — do NOT collapse it to
51+
# `grep … && echo ❌ || echo ✅`, which always exits 0 and silently lets ship continue.
52+
if grep -vE '^0 ' /tmp/ship-gen-results; then echo "❌ generator(s) failed — do not ship"; exit 1; fi
53+
echo "✅ artifacts regenerated"
54+
```
55+
Then `git status --short` to see what regenerated — those files must be staged in step 7 alongside your own changes.
56+
57+
**Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present.
58+
59+
**Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then fan the rest out and collect exit codes. This is exactly the read-only audit set from CI's `Lint and Test` job (all in-repo, runnable in any worktree):
60+
```bash
61+
# autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too —
62+
# a non-zero lint (unfixable errors) must abort before the audits run, not be ignored.
63+
bun run lint || { echo "❌ lint failed — do not ship"; exit 1; }
64+
rm -f /tmp/ship-audit-results
65+
for s in check:boundaries check:api-validation:strict check:utils check:zustand-v5 \
66+
check:react-query check:client-boundary check:bare-icons check:icon-paths \
67+
check:realtime-prune skills:check agent-stream-docs:check; do
68+
( bun run "$s" >"/tmp/ship-audit-${s//:/-}.log" 2>&1; echo "$? $s" >>/tmp/ship-audit-results ) &
69+
done
70+
wait
71+
# any non-zero line is a failing audit — read its /tmp/ship-audit-<name>.log and fix before shipping.
72+
# `exit 1` on failure preserves the original sequential checks' semantics (their non-zero exit is
73+
# what an agent gates on); never use `grep … && echo ❌ || echo ✅` here — it always exits 0.
74+
if grep -vE '^0 ' /tmp/ship-audit-results; then echo "❌ audit(s) failed — do not ship"; exit 1; fi
75+
echo "✅ all audits passed"
76+
```
77+
If Phase A regenerated a file, its matching `:check` in Phase B now passes trivially — that parity is the point. Do not ship with any generator or audit failing; fix the cause (never silence it) and re-run. `check:migrations` and `type-check` are covered by steps 5 and CI respectively and are not repeated here.
78+
7. **Stage and commit** the changes with the generated message — including any files Phase A regenerated in step 6
4279
8. **Push to origin** using the current branch name — `--force-with-lease` if step 2's sync
4380
check did any history rewrite (a clean rebase or a cherry-pick rebuild) on a branch that had
4481
already been pushed once; a plain push would be rejected in exactly the polluted-remote case

.agents/skills/validate-model/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ For each model, evaluate every row. Statuses: ✓ matches docs, ✗ disagrees,
8989
- [ ] `reasoningEffort.values` — list matches docs; **omitted** for always-reasoning models that reject the parameter (e.g., grok-4.3, where xAI docs explicitly state `reasoning_effort` is not supported). Verify per model — some always-reasoning models (e.g., OpenAI's o-series) DO accept `reasoning_effort` and should keep the flag.
9090
- [ ] `verbosity.values` — only on OpenAI gpt-5.x family; values match docs
9191
- [ ] `thinking.levels` + `thinking.default` — only on Anthropic/Gemini; values match docs
92+
- [ ] `thinking.streamed` — REQUIRED on Anthropic-family thinking models (`'full'` for generations returning full thinking deltas, `'summary'` for omitted-display generations like Opus 4.7+/Sonnet 5/Fable 5 where Sim requests `display: 'summarized'`); verify against the provider's thinking-display docs. After any change, run `bun run agent-stream-docs:generate` so the Agent block docs table stays in sync (CI diffs it)
9293
- [ ] `nativeStructuredOutputs` — only on anthropic/fireworks/openrouter; provider must document Structured Outputs / JSON-mode for this model
9394
- [ ] `toolUsageControl` — provider supports `tool_choice` semantics
9495
- [ ] `computerUse` — provider implements computer-use loop AND model is a computer-use SKU

.claude/commands/add-model.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ Use a precise WebFetch prompt: *"Extract for {model_id}: exact model id string,
5151
| `reasoningEffort` | `openai/core.ts`, `azure-openai`, `anthropic/core.ts` (mapped to thinking), `gemini/core.ts` | **Dead on xai, deepseek, mistral, groq, cerebras, openrouter, fireworks, bedrock, vertex** unless their core consumes it — re-grep before assuming |
5252
| `verbosity` | `openai/core.ts`, `azure-openai/index.ts` only | Dead elsewhere |
5353
| `thinking` | `anthropic/core.ts`, `gemini/core.ts` | Dead elsewhere |
54+
| `thinking.streamed` | Docs generator + `getThinkingStreamVisibility` (`models.ts`); `anthropic/core.ts` uses `'summary'` to request `display: 'summarized'` on agent-events runs | **Mandatory on Anthropic-family thinking models** (`agent-stream-docs:check` fails without it); other families fall back to provider defaults |
5455
| `nativeStructuredOutputs` | `anthropic/core.ts`, `fireworks/index.ts`, `openrouter/index.ts` | Dead on openai, xai, google, vertex, bedrock, azure-openai, deepseek, mistral, groq, cerebras |
5556
| `maxOutputTokens` | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
5657
| `computerUse` | `anthropic/core.ts` | Dead elsewhere |
@@ -145,6 +146,15 @@ If anything matches, run the affected provider tests and update assertions as ne
145146

146147
The Consumption Matrix (Step 2) tells you which capability *flags* are honored by existing provider code. But if the new model needs **net-new** request handling that the provider doesn't implement yet — a new beta header (e.g. Anthropic's `anthropic-beta` structured-outputs header in `anthropic/index.ts`), a new thinking/reasoning encoding, a Responses-API quirk — you must edit `apps/sim/providers/<provider>/core.ts` / `index.ts`. Setting a flag whose behavior isn't implemented is a silent no-op. When you do edit provider code, reuse the shared helpers rather than hand-rolling: streaming responses are assembled via `createStreamingExecution` (`@/providers/streaming-execution`) and tool schemas via `adaptOpenAIChatToolSchema` / `adaptAnthropicToolSchema` (`@/providers/tool-schema-adapter`).
147148

149+
### Thinking/reasoning models: `streamed` visibility + generated docs
150+
151+
If the entry has `capabilities.thinking` or `capabilities.reasoningEffort`, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
152+
153+
- **Anthropic-family (`anthropic`, `azure-anthropic`) thinking models MUST declare `capabilities.thinking.streamed`** (`'full' | 'summary' | 'none'`) — visibility varies per Claude generation. Verify against the provider's thinking/display docs (e.g. Anthropic's "controlling thinking display" page and per-model "what's new" notes): generations that default `thinking.display` to `omitted` (Opus 4.7+, Sonnet 5, Fable 5) are `'summary'` — Sim opts back in with `display: 'summarized'` on agent-events runs; older generations that return full thinking deltas are `'full'`. `bun run agent-stream-docs:check` (CI) fails if the field is missing.
154+
- Other families usually omit the field and inherit the provider default in `getThinkingStreamVisibility` (Gemini/OpenAI → summaries, Bedrock → none, OpenAI-compat vendors → full raw CoT). Set it explicitly only when the model deviates from its family.
155+
- After inserting the entry, run `bun run agent-stream-docs:generate` and commit the regenerated `apps/docs/content/docs/en/workflows/blocks/agent.mdx` — CI diffs it.
156+
- Include the `streamed` value (with its source URL) in the verification report when set.
157+
148158
### Wrong family entirely?
149159

150160
- **Embedding or rerank model** → it does NOT go in the `models[]` array. Use `EMBEDDING_MODEL_PRICING` / `RERANK_MODEL_PRICING` in `models.ts` instead.
@@ -154,6 +164,7 @@ The Consumption Matrix (Step 2) tells you which capability *flags* are honored b
154164

155165
```bash
156166
bun run lint
167+
bun run agent-stream-docs:generate # only when the entry has thinking/reasoningEffort
157168
```
158169

159170
Lint must pass before reporting done. **If lint fails:** read the error, fix the syntax/typing issue in the entry you just wrote (do not delete the entry — it's the work product), re-run lint, and note the fix in a "Lint adjustments" line in the verification report. Never report done with lint failing.
@@ -200,6 +211,7 @@ Omitting a field is **not the same as verifying it**. Any field you cannot confi
200211
- ❌ Trusting a marketing email (xAI's grok-4.3 email claimed "3 reasoning efforts" but the API rejects `reasoning_effort` — verified by official docs only)
201212
- ❌ Setting `nativeStructuredOutputs: true` on xai/openai/google (dead — only anthropic/fireworks/openrouter consume it)
202213
- ❌ Setting `thinking` on non-Anthropic/non-Gemini providers
214+
- ❌ Adding an Anthropic-family thinking model without `thinking.streamed` (CI `agent-stream-docs:check` fails), or skipping `bun run agent-stream-docs:generate` after adding any thinking/reasoning model
203215
- ❌ Setting `verbosity` on anything other than OpenAI gpt-5.x
204216
- ❌ Copying `pricing.updatedAt` from a sibling instead of using today's date
205217
- ❌ Inventing a `cachedInput` price by dividing input by 4 (varies by provider — find an explicit number)

0 commit comments

Comments
 (0)