Skip to content

Default build-session model to Opus 5 - #95

Merged
dev-dull merged 2 commits into
mainfrom
feat/default-model-opus-5
Jul 28, 2026
Merged

Default build-session model to Opus 5#95
dev-dull merged 2 commits into
mainfrom
feat/default-model-opus-5

Conversation

@dev-dull

Copy link
Copy Markdown
Owner

Changes the default Claude model for build sessions from the sonnet alias to opus (Claude Opus 5, which just shipped).

Why

Claude Opus 5 is Anthropic's recommended model for agentic coding — which is precisely what a SUS build session is. For non-technical users, higher build quality and first-shot success means less frustrating back-and-forth (which can also reduce total token spend).

Tradeoff (documented)

Opus 5 costs ~2.5× more per token than Sonnet 5 ($5/$25 vs $2/$10 introductory) and responds a bit slower. Operators who prefer lower cost / faster responses can set --set buildPod.claudeModel=sonnet. This is called out in both values.yaml and the README.

What changed

Updated the default in every place it's resolved so "the default is Opus" holds regardless of entry point:

  • charts/sus/values.yamlbuildPod.claudeModel: opus (+ comment explaining the tradeoff)
  • landing/app/pods.pySUS_CLAUDE_MODEL env fallback
  • build-pod/entrypoint.shCLAUDE_MODEL fallback + doc comment
  • README.md → config table row + a build-session-model note

Uses the opus alias (not the pinned claude-opus-5) so it auto-tracks future Opus releases, matching the existing sonnet/haiku convention. A full ID like claude-opus-5 can be set to pin a version.

Verification

helm template renders SUS_CLAUDE_MODEL="opus"; helm lint clean; pods.py compiles. Model IDs/pricing verified against Anthropic's live model + pricing docs.

🤖 Generated with Claude Code

Claude Opus 5 shipped and is Anthropic's recommended model for agentic coding,
which is exactly what a build session is — so it gives the best build quality
for SUS's non-technical users (and better first-shot success can reduce total
iterations). Switch the default from the sonnet alias to opus across every place
the default is resolved: buildPod.claudeModel (chart), the SUS_CLAUDE_MODEL env
fallback in pods.py, and the CLAUDE_MODEL fallback in build-pod/entrypoint.sh.

Uses the `opus` alias (not the pinned claude-opus-5) so it auto-tracks future
Opus releases, matching the existing sonnet/haiku alias convention.

Tradeoff documented in values.yaml and the README: Opus 5 costs ~2.5x more per
token than Sonnet 5 ($5/$25 vs $2/$10 intro) and is a bit slower; operators who
prefer lower cost / faster responses set buildPod.claudeModel=sonnet.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Claude finished @dev-dull's task in 3m 38s —— View job


Review: default build-session model → Opus 5

  • Gather context (diff, repo conventions, build-pod/CLAUDE.md)
  • Verify model ID / alias validity
  • Check every default-resolution point is consistent
  • Render Helm chart (auth on and off) and check values wiring
  • Assess test coverage gaps
  • Post review

The change is mechanically correct and complete. I verified the claim that "the default is Opus regardless of entry point" actually holds — a case-insensitive sweep for sonnet/claudeModel/CLAUDE_MODEL across the repo turns up no stragglers, and run-entrypoint.sh never invokes claude, so the four touched files really are the full set. Two non-blocking comments posted inline.

Verification performed

Check Result
helm lint charts/sus clean (only the pre-existing icon is recommended info)
helm template (auth off) SUS_CLAUDE_MODEL: "opus" ✅ — renders no Authelia/auth resources, so the feature stays opt-in
helm template --set auth.enabled=true SUS_CLAUDE_MODEL: "opus" ✅ + Authelia secret/configmap/service/deployment + networkpolicy
--set buildPod.claudeModel=sonnet renders "sonnet" ✅ — documented escape hatch works
Model IDs opus alias and full ID claude-opus-5 both valid; alias choice matches the existing sonnet/haiku convention
Empty-value edge case claudeModel: ""pods.py passes "", but ${CLAUDE_MODEL:-opus} in entrypoint.sh recovers it. Fine — the two layers happen to defend each other

Findings

1. Silent ~2.5× cost increase on helm upgrade (charts/sus/values.yaml:45) — the substantive one. Existing operators get the new per-session cost with no signal, and there's no token cap or spend limit anywhere in the repo to bound an interactive session. The tradeoff is well documented in the files, but nothing surfaces it at install/upgrade time. templates/NOTES.txt already uses exactly this pattern for auth; two lines there would fix it. Suggested wording is inline.

2. Hardcoded ~2.5× multiplier in two files (README.md:135, charts/sus/values.yaml:42) — derived from introductory pricing per the PR body, so it will rot silently. Suggest keeping the qualitative tradeoff and linking to Anthropic's pricing page for the numbers.

Notes

  • Shell injection — I checked entrypoint.sh:261, since CLAUDE_MODEL is interpolated into a single-quoted context inside a bash -c string. A single quote in the value would break out. But the value originates from buildPod.claudeModel in Helm values → operator-controlled at install time, never user- or request-controlled, and the operator already has cluster control. Not a trust-boundary issue, and the quoting is unchanged by this PR — flagging only to record that it was looked at, not as something to fix here.
  • Test coverage — nothing in landing/tests/ exercises BuildPodManager env construction, so the model default has no test asserting it. Pre-existing gap rather than something this PR introduces, but it does mean the change is unverified in CI; a one-line assertion on the CLAUDE_MODEL env var would lock the default in cheaply.
  • First-run dialogsbuild-pod/Dockerfile:74-75 pre-seeds opus46FeedSeenCount / opus1mMergeNoticeSeenCount, so the existing onboarding suppression already anticipates Opus-related notices. No new interactive prompt risk spotted from the model switch.

I could not verify the specific $5/$25 vs $2/$10 pricing figures — WebFetch permission wasn't granted in this run, so I had no way to reach Anthropic's pricing docs. Finding 2 is about the durability of hardcoding a derived ratio, not a claim that the numbers are wrong. If you want the figures confirmed on a re-run, adding WebFetch to --allowedTools would let me check them directly.

Nothing here blocks merge — both points are about operator-visibility rather than correctness.
• branch feat/default-model-opus-5

Comment thread charts/sus/values.yaml
# coding, i.e. the best build quality. It costs ~2.5x more per token than
# "sonnet" and is a bit slower; set claudeModel: sonnet to trade some build
# quality for lower cost and faster responses.
claudeModel: opus

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing a default means every operator who runs helm upgrade without reading the diff gets a ~2.5× per-session cost increase with no signal. That's the one thing I'd want mitigated before merge.

SUS spins up a pod per session running claude --dangerously-skip-permissions for an interactive, long-lived session (ttyd stays connected, auto-commits every 5 min), and there's no token cap or per-session spend limit anywhere in the repo — so the downside per session is unbounded, while the "better first-shot success reduces total spend" argument is plausible but unmeasured. That asymmetry is a fine call to make deliberately; it just shouldn't be invisible.

templates/NOTES.txt already exists and already uses this exact pattern for auth (surfacing active config plus WARNING: lines). Surfacing the resolved model there costs two lines and makes the change visible at install/upgrade time:

Build sessions use Claude model: {{ .Values.buildPod.claudeModel }}
  (set buildPod.claudeModel=sonnet for lower cost / faster responses)

Worth a note in the release notes too, since the cost delta lands on existing installs rather than new ones.

Comment thread README.md

See [`charts/sus/values.yaml`](charts/sus/values.yaml) for all Helm values.

**Build-session model:** the default is `opus` (Claude Opus 5) — Anthropic's recommended model for agentic coding, so the best build quality. It costs roughly **2.5× more per token** than `sonnet` and responds a bit slower. If you'd rather trade some build quality for lower cost and faster responses, set `--set buildPod.claudeModel=sonnet` (Claude Sonnet 5). Aliases track the latest model in each family; pass a full ID like `claude-opus-5` to pin a version.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ~2.5× multiplier is now hardcoded in two places (here and charts/sus/values.yaml:42), and the PR description sources it from introductory pricing — which by definition changes. Derived numbers like this rot silently, and a stale cost ratio in the README is the kind of thing an operator makes a budget decision on.

Consider keeping the qualitative tradeoff (Opus costs more and is a bit slower) and linking to Anthropic's pricing page for the actual numbers, rather than restating a specific ratio in two files.

Note: I wasn't able to independently verify the $5/$25 vs $2/$10 figures in this environment (no network access granted for the pricing docs), so I'm flagging the durability of the claim rather than disputing its accuracy. The model IDs and aliases do check out — opus and claude-opus-5 are both valid, and using the alias to auto-track future releases is the right call for the stated goal.

Existing operators upgrading pick up the Opus 5 default (~2.5x per-token
cost vs sonnet) with no signal. NOTES.txt already uses this pattern for
auth; add a line showing the active model and the cost tradeoff when it's
opus, with the sonnet escape hatch. Addresses PR #95 review finding 1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Claude finished @dev-dull's task in 3m 45s —— View job


Re-review: default build-session model → Opus 5

  • Gather context (diff vs merge base c5161f1, build-pod/CLAUDE.md)
  • Review new NOTES.txt template for correctness
  • Render chart (auth on / off) + values wiring
  • Verify pricing claims against Anthropic's live pricing docs
  • Check status of prior findings
  • Post review

The new commit (51e4696) addresses the visibility finding from the last round — NOTES.txt prints on helm upgrade as well as install, so upgrading operators see the cost change. That was the one thing I'd wanted mitigated before merge, and it is. Two inline comments on the new code, neither blocking.

What changed since last review

charts/sus/templates/NOTES.txt (+6) surfaces the resolved model, plus a cost note gated on eq .Values.buildPod.claudeModel "opus". values.yaml comment expanded. The four originally-touched files are unchanged in substance.

Verification performed

Check Result
helm lint charts/sus clean (only pre-existing icon is recommended). Lint renders NOTES.txt, so the new template parses
helm template (auth off) SUS_CLAUDE_MODEL: "opus" ✅ — no Authelia resources, no networkpolicy. Feature stays opt-in
helm template --set auth.enabled=true SUS_CLAUDE_MODEL wired ✅ + Authelia secret/configmap/service/deployment + networkpolicy + landing-config
--set buildPod.claudeModel=claude-opus-5 renders "claude-opus-5" ✅ — pinning works (but see finding 1)
--set buildPod.claudeModel=null renders "", no template erroreq tolerates nil in current Go templates, and entrypoint.sh's ${CLAUDE_MODEL:-opus} recovers it
Pricing figures verified — see below

I could not run helm install --dry-run, so I did not see the NOTES text itself render; the guard logic is assessed from the template plus helm lint (which does execute it).

Pricing — the PR's numbers check out, with an expiry date

Last round I couldn't reach the pricing docs. This run I could, and the figures in the PR body are correct: Opus 5 is $5/$25 per MTok, Sonnet 5 is $2/$10 — exactly 2.5×.

The catch: Sonnet 5's $2/$10 is introductory pricing through Aug 31, 2026, rising to $3/$15 on Sept 1. Opus 5 is unchanged. So ~2.5× is accurate for about another month and then becomes ~1.67×. That number now appears in three files, including the NOTES text printed to operators on every upgrade. Details and a suggested rewording are in the README inline comment.

Findings (both non-blocking)

  1. NOTES.txt:16 — cost note skipped for pinned Opus IDs. eq ... "opus" matches only the bare alias, but both the README and values.yaml tell operators to pass claude-opus-5 to pin a version. The operator deliberate enough to pin Opus is the one who doesn't get told what it costs. A contains "opus" check (with default "" for nil-safety) covers both spellings. The suggestion also drops the hardcoded Claude Opus 5 — asserting the version defeats the point of using an auto-tracking alias.
  2. README.md:135 — the 2.5× multiplier is now in three places and has a known Sept 1, 2026 expiry. Suggest linking to Anthropic's pricing rather than restating a ratio that's already scheduled to change.

Notes

  • Test coverage — still nothing in landing/tests/ (test_build_resolution.py, test_build_routes.py, test_identity.py, test_proxy_headers.py) that touches V1EnvVar construction or CLAUDE_MODEL; a grep for either turns up zero hits. Pre-existing gap, not introduced here, but it does mean the default this PR changes is unasserted in CI. One assertion on the env var would lock it in.
  • Shell injection — unchanged from last round: CLAUDE_MODEL is interpolated into a single-quoted bash -c string at entrypoint.sh:261, but the value is operator-supplied Helm values, never request- or user-controlled. Not a trust-boundary issue and not touched by this PR.
    • branch feat/default-model-opus-5

Comment thread README.md

See [`charts/sus/values.yaml`](charts/sus/values.yaml) for all Helm values.

**Build-session model:** the default is `opus` (Claude Opus 5) — Anthropic's recommended model for agentic coding, so the best build quality. It costs roughly **2.5× more per token** than `sonnet` and responds a bit slower. If you'd rather trade some build quality for lower cost and faster responses, set `--set buildPod.claudeModel=sonnet` (Claude Sonnet 5). Aliases track the latest model in each family; pass a full ID like `claude-opus-5` to pin a version.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to reach Anthropic's pricing docs on this run, so I can put a number on the durability concern from the last round rather than just flagging it.

The ~2.5× figure is correct today and expires on a known date:

input output
Claude Opus 5 $5 / MTok $25 / MTok
Claude Sonnet 5 — through Aug 31, 2026 $2 / MTok $10 / MTok
Claude Sonnet 5 — from Sep 1, 2026 $3 / MTok $15 / MTok

So 2.5× holds for about another month, then becomes ~1.67× — the Sonnet 5 introductory window ends Sept 1, and Opus 5 pricing is unchanged. That number is now stated in three places (here, charts/sus/values.yaml:41, and charts/sus/templates/NOTES.txt:17), and the NOTES one is printed to operators on every install and upgrade, so a stale ratio there will be actively misleading rather than just wrong-in-a-doc.

Suggest dropping the specific multiplier and linking instead, so there's nothing to update on Sept 1:

Suggested change
**Build-session model:** the default is `opus` (Claude Opus 5) — Anthropic's recommended model for agentic coding, so the best build quality. It costs roughly **2.5× more per token** than `sonnet` and responds a bit slower. If you'd rather trade some build quality for lower cost and faster responses, set `--set buildPod.claudeModel=sonnet` (Claude Sonnet 5). Aliases track the latest model in each family; pass a full ID like `claude-opus-5` to pin a version.
**Build-session model:** the default is `opus` (Claude Opus 5) — Anthropic's recommended model for agentic coding, so the best build quality. It costs meaningfully more per token than `sonnet` and responds a bit slower; see [Anthropic's pricing](https://claude.com/pricing#api) for current rates. If you'd rather trade some build quality for lower cost and faster responses, set `--set buildPod.claudeModel=sonnet` (Claude Sonnet 5). Aliases track the latest model in each family; pass a full ID like `claude-opus-5` to pin a version.

If you'd rather keep the concrete number (it is more persuasive than "meaningfully more"), that's a fine call — but then it's worth a note that it needs revisiting after Aug 31, 2026.

Comment on lines +15 to +19
Build sessions use the "{{ .Values.buildPod.claudeModel }}" Claude model.
{{- if eq .Values.buildPod.claudeModel "opus" }}
This is Claude Opus 5 — best build quality, but ~2.5x the per-token cost of
"sonnet" and a bit slower. Set --set buildPod.claudeModel=sonnet to lower cost.
{{- end }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addresses the visibility gap from the last round — NOTES is printed on helm upgrade as well as install, so upgrading operators now see the change. Good.

One gap in the guard: eq ... "opus" matches only the bare alias. Both this file and README.md:135 tell operators to pass a full ID like claude-opus-5 to pin a version — and I confirmed that path renders fine (--set buildPod.claudeModel=claude-opus-5SUS_CLAUDE_MODEL: "claude-opus-5"), but it silently skips the cost note. So the operator who was deliberate enough to pin Opus is the one who doesn't get told what Opus costs. Same for claude-opus-5[1m].

A substring check covers both spellings. The default "" keeps it safe if claudeModel is nil (a values override with a bare claudeModel: key yields nil, and contains on nil errors — eq tolerates it, so this is only needed once you switch to contains):

Suggested change
Build sessions use the "{{ .Values.buildPod.claudeModel }}" Claude model.
{{- if eq .Values.buildPod.claudeModel "opus" }}
This is Claude Opus 5 — best build quality, but ~2.5x the per-token cost of
"sonnet" and a bit slower. Set --set buildPod.claudeModel=sonnet to lower cost.
{{- end }}
Build sessions use the "{{ .Values.buildPod.claudeModel }}" Claude model.
{{- if contains "opus" (default "" .Values.buildPod.claudeModel) }}
This is Claude Opus — best build quality, but several times the per-token cost
of "sonnet" and a bit slower. Set buildPod.claudeModel=sonnet to lower cost.
{{- end }}

Two smaller things folded into that suggestion:

  • "Set --set buildPod.claudeModel=sonnet" reads as a stutter, and --set is wrong for the operators who use a values file. Bare buildPod.claudeModel=sonnet works for both audiences.
  • "This is Claude Opus 5" is asserted unconditionally, but the whole point of the alias is that it tracks future releases — the day Opus 6 ships under the opus alias this line is wrong with no code change. Dropping the version number keeps it true. (Same reason I'd drop the 2.5x here — see the README comment; the Sonnet 5 introductory rate ends Aug 31, 2026, after which the real ratio is ~1.67×.)

@dev-dull
dev-dull merged commit 74cb713 into main Jul 28, 2026
1 check passed
@dev-dull
dev-dull deleted the feat/default-model-opus-5 branch July 28, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant