You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The gating risk was whether the pinned Claude CLI would pass an unrecognized model string through. It does. Ran agent/scripts/diagnostics/test_sdk_smoke.py — the real claude-agent-sdk → Claude Code CLI → Bedrock path — against the pinned toolchain:
No SDK or CLI bump is required for this issue. The #215 lockstep pin can stay as-is. (Cost: the per-token rate is unchanged. Same one-turn prompt on both models implies exactly $5.00/MTok input — Opus 4.8 at 32,145 tok/$0.160850 and Opus 5 at 37,584 tok/$0.188020, a 1.169 token ratio and an identical 1.169 cost ratio. So the delta is token volume on an identical prompt, not a price increase. Re-baseline token volume, not rates.)
Documented default (per #742's new canonical section + drift test)
cli/src/repo-display.ts:48 is load-bearing beyond display: platform doctor derives the model it probes for access from this value, so a stale value makes doctor probe the wrong model.
The drift guard at cdk/test/constructs/bedrock-models.test.ts:83 enforces the config.py↔CDK coupling and passes unchanged here — the new default is still us.-prefixed, satisfying its /^us\./ assertion, and #744 already added the bare ID to the grant list. This is precisely why the us. step precedes the global step.
Behavioral re-baselining
A model bump is not purely mechanical. Before/after a representative task set (minimum: coding/new-task-v1, coding/pr-review-v1):
Cost + turn budgets. Per-token rate is unchanged ($5.00/MTok both models); token volume on an identical prompt is ~1.17x. Re-measure max_turns (default 100) empirically rather than applying that ratio as a multiplier — agentic loops compound differently than a single turn. Note max_budget_usd has no platform default (USER_GUIDE.md:231): unset means unlimited, so a token-volume increase is not capped by default. The documented mitigation is a lighter-token model per repo/task (blueprint agent.modelId / payload model_id) — see docs(model): canonical model-configuration reference + fix stale defaults #742's cost section.
Budget guardrail trustworthiness.cost_usd is the SDK's client-side estimate from a bundled price table (COST_ATTRIBUTION.md:6), and it is what max_budget_usd enforces against. Verified the pinned SDK prices Opus 5 correctly (implied $5.00/MTok, matching Opus 4.8), so the guardrail holds for this bump.
Prompt over-prescription. Scaffolding tuned for an older model can reduce quality on a newer one. A/B the agent/src/ pipeline prompts with step-by-step scaffolding removed.
Code-review recall. If coding/pr-review-v1 instructs the model to be conservative or report only high-severity findings, newer models follow that more literally and measured recall can drop even as bug-finding improves. Prefer report-everything-with-confidence + filter downstream.
Longer turns. Check agent-runtime and orchestrator poll/await timeouts before assuming a slow run is a hang.
Acceptance criteria
All four sites updated in one commit
cdk/test/constructs/bedrock-models.test.ts:83 drift guard passes without modification
Deployed smoke test on agentcore: one task completes end-to-end and opens a PR — this is what proves the IAM grant covers the invoked profile ARN, which unit tests cannot
ECS / lambda-microvm verification is out of scope — CI deploys agentcore only (ALLOWED_COMPUTE_TYPES: "agentcore" in both .github/workflows/build.yml and deploy.yml; the build matrix at build.yml:52 is [agentcore]). ECS unit-test coverage in cdk/test/constructs/ecs-agent-cluster.test.ts still applies and must pass
Cost/turn baseline captured before and after on the representative task set
platform doctor reports the new model and passes its access probe
mise run build green
Notes
Blocked by #744. Recommended after #743. Independent of #745/#746.
Child 4 of 6 — parent #741. Behavior change. Depends on #744 being deployed first.
Single-value flip across four call sites. Revert is a one-line change per site.
Blocked by
AccessDenied.run.shkeeps overriding the new default and local runs stay on Sonnet 4.6.SDK/CLI compatibility — VERIFIED, not assumed
The gating risk was whether the pinned Claude CLI would pass an unrecognized model string through. It does. Ran
agent/scripts/diagnostics/test_sdk_smoke.py— the realclaude-agent-sdk→ Claude Code CLI → Bedrock path — against the pinned toolchain:claude-agent-sdk==0.2.110(matchesagent/pyproject.toml:19)2.1.191(matches the npm pin atagent/Dockerfile:90)ANTHROPIC_MODELus.anthropic.claude-opus-5AssistantMessage+ResultMessage status=success turns=1, 3.9sglobal.anthropic.claude-opus-5global.anthropic.claude-opus-5+ANTHROPIC_DEFAULT_HAIKU_MODEL=global.anthropic.claude-haiku-4-5-20251001-v1:0No SDK or CLI bump is required for this issue. The #215 lockstep pin can stay as-is. (Cost: the per-token rate is unchanged. Same one-turn prompt on both models implies exactly $5.00/MTok input — Opus 4.8 at 32,145 tok/$0.160850 and Opus 5 at 37,584 tok/$0.188020, a 1.169 token ratio and an identical 1.169 cost ratio. So the delta is token volume on an identical prompt, not a price increase. Re-baseline token volume, not rates.)
Scope — four call sites, one commit
agent/src/config.pyANTHROPIC_MODELfallback →us.anthropic.claude-opus-5agent/src/models.pyanthropic_modelfield default → samecli/src/repo-display.tsPLATFORM_REPO_DEFAULTS.model_id→ samecli/src/repo-display.ts:48is load-bearing beyond display:platform doctorderives the model it probes for access from this value, so a stale value makes doctor probe the wrong model.The drift guard at
cdk/test/constructs/bedrock-models.test.ts:83enforces the config.py↔CDK coupling and passes unchanged here — the new default is stillus.-prefixed, satisfying its/^us\./assertion, and #744 already added the bare ID to the grant list. This is precisely why theus.step precedes the global step.Behavioral re-baselining
A model bump is not purely mechanical. Before/after a representative task set (minimum:
coding/new-task-v1,coding/pr-review-v1):max_turns(default 100) empirically rather than applying that ratio as a multiplier — agentic loops compound differently than a single turn. Notemax_budget_usdhas no platform default (USER_GUIDE.md:231): unset means unlimited, so a token-volume increase is not capped by default. The documented mitigation is a lighter-token model per repo/task (blueprintagent.modelId/ payloadmodel_id) — see docs(model): canonical model-configuration reference + fix stale defaults #742's cost section.cost_usdis the SDK's client-side estimate from a bundled price table (COST_ATTRIBUTION.md:6), and it is whatmax_budget_usdenforces against. Verified the pinned SDK prices Opus 5 correctly (implied $5.00/MTok, matching Opus 4.8), so the guardrail holds for this bump.agent/src/pipeline prompts with step-by-step scaffolding removed.coding/pr-review-v1instructs the model to be conservative or report only high-severity findings, newer models follow that more literally and measured recall can drop even as bug-finding improves. Prefer report-everything-with-confidence + filter downstream.Acceptance criteria
cdk/test/constructs/bedrock-models.test.ts:83drift guard passes without modificationagentcore: one task completes end-to-end and opens a PR — this is what proves the IAM grant covers the invoked profile ARN, which unit tests cannotlambda-microvmverification is out of scope — CI deploysagentcoreonly (ALLOWED_COMPUTE_TYPES: "agentcore"in both.github/workflows/build.ymlanddeploy.yml; the build matrix atbuild.yml:52is[agentcore]). ECS unit-test coverage incdk/test/constructs/ecs-agent-cluster.test.tsstill applies and must passplatform doctorreports the new model and passes its access probemise run buildgreenNotes
Blocked by #744. Recommended after #743. Independent of #745/#746.