Skip to content

feat(agent): flip platform default model to us.anthropic.claude-opus-5 #745

Description

@scottschreckengaust

Child 4 of 6 — parent #741. Behavior change. Depends on #744 being deployed first.

Single-value flip across four call sites. Revert is a one-line change per site.

Blocked by

SDK/CLI compatibility — VERIFIED, not assumed

The gating risk was whether the pinned Claude CLI would pass an unrecognized model string through. It does. Ran agent/scripts/diagnostics/test_sdk_smoke.py — the real claude-agent-sdk → Claude Code CLI → Bedrock path — against the pinned toolchain:

  • claude-agent-sdk==0.2.110 (matches agent/pyproject.toml:19)
  • bundled CLI 2.1.191 (matches the npm pin at agent/Dockerfile:90)
ANTHROPIC_MODEL Result
us.anthropic.claude-opus-5 PASSAssistantMessage + ResultMessage status=success turns=1, 3.9s
global.anthropic.claude-opus-5 PASS — same, 3.9s
global.anthropic.claude-opus-5 + ANTHROPIC_DEFAULT_HAIKU_MODEL=global.anthropic.claude-haiku-4-5-20251001-v1:0 PASS — 3.6s

No SDK or CLI bump is required for this issue. The #215 lockstep pin can stay as-is. (Cost: the per-token rate is unchanged. Same one-turn prompt on both models implies exactly $5.00/MTok input — Opus 4.8 at 32,145 tok/$0.160850 and Opus 5 at 37,584 tok/$0.188020, a 1.169 token ratio and an identical 1.169 cost ratio. So the delta is token volume on an identical prompt, not a price increase. Re-baseline token volume, not rates.)

Scope — four call sites, one commit

File Line Change
agent/src/config.py 563 ANTHROPIC_MODEL fallback → us.anthropic.claude-opus-5
agent/src/models.py 157 anthropic_model field default → same
cli/src/repo-display.ts 48 PLATFORM_REPO_DEFAULTS.model_id → same
docs Documented default (per #742's new canonical section + drift test)

cli/src/repo-display.ts:48 is load-bearing beyond display: platform doctor derives the model it probes for access from this value, so a stale value makes doctor probe the wrong model.

The drift guard at cdk/test/constructs/bedrock-models.test.ts:83 enforces the config.py↔CDK coupling and passes unchanged here — the new default is still us.-prefixed, satisfying its /^us\./ assertion, and #744 already added the bare ID to the grant list. This is precisely why the us. step precedes the global step.

Behavioral re-baselining

A model bump is not purely mechanical. Before/after a representative task set (minimum: coding/new-task-v1, coding/pr-review-v1):

  • Cost + turn budgets. Per-token rate is unchanged ($5.00/MTok both models); token volume on an identical prompt is ~1.17x. Re-measure max_turns (default 100) empirically rather than applying that ratio as a multiplier — agentic loops compound differently than a single turn. Note max_budget_usd has no platform default (USER_GUIDE.md:231): unset means unlimited, so a token-volume increase is not capped by default. The documented mitigation is a lighter-token model per repo/task (blueprint agent.modelId / payload model_id) — see docs(model): canonical model-configuration reference + fix stale defaults #742's cost section.
  • Budget guardrail trustworthiness. cost_usd is the SDK's client-side estimate from a bundled price table (COST_ATTRIBUTION.md:6), and it is what max_budget_usd enforces against. Verified the pinned SDK prices Opus 5 correctly (implied $5.00/MTok, matching Opus 4.8), so the guardrail holds for this bump.
  • Prompt over-prescription. Scaffolding tuned for an older model can reduce quality on a newer one. A/B the agent/src/ pipeline prompts with step-by-step scaffolding removed.
  • Code-review recall. If coding/pr-review-v1 instructs the model to be conservative or report only high-severity findings, newer models follow that more literally and measured recall can drop even as bug-finding improves. Prefer report-everything-with-confidence + filter downstream.
  • Longer turns. Check agent-runtime and orchestrator poll/await timeouts before assuming a slow run is a hang.

Acceptance criteria

  • All four sites updated in one commit
  • cdk/test/constructs/bedrock-models.test.ts:83 drift guard passes without modification
  • feat(cdk): grant Claude Opus 5 (additive IAM + workflow allowlist, no default change) #744 confirmed deployed to the target account before merge
  • Deployed smoke test on agentcore: one task completes end-to-end and opens a PR — this is what proves the IAM grant covers the invoked profile ARN, which unit tests cannot
  • ECS / lambda-microvm verification is out of scope — CI deploys agentcore only (ALLOWED_COMPUTE_TYPES: "agentcore" in both .github/workflows/build.yml and deploy.yml; the build matrix at build.yml:52 is [agentcore]). ECS unit-test coverage in cdk/test/constructs/ecs-agent-cluster.test.ts still applies and must pass
  • Cost/turn baseline captured before and after on the representative task set
  • platform doctor reports the new model and passes its access probe
  • mise run build green

Notes

Blocked by #744. Recommended after #743. Independent of #745/#746.

Metadata

Metadata

Labels

P1medium priorityagent-runtimePython agent container: pipeline, runner, hooks, prompts, tools, DockerfileapprovedWhen an issue has been approved and readyenhancementNew feature or requestinfra-cdkCDK stacks/constructs, bootstrap, deploy topology, tags, IAM wiring, teardown

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions