cloud: make the Piccolissimo + Altissimo tier actually mean Altissimo, in the cloud, automatically - #259
Open
Rchari1 wants to merge 10 commits into
Open
cloud: make the Piccolissimo + Altissimo tier actually mean Altissimo, in the cloud, automatically#259Rchari1 wants to merge 10 commits into
Rchari1 wants to merge 10 commits into
Conversation
Selecting Piccolissimo + Altissimo is meant to mean "every solve runs Altissimo, in Harmoniqs Cloud". Four gaps between that promise and the code: 1. The solve template defaulted to `SOLVER = :ipopt`, so nothing actually selected the Altissimo backend — a paid HP run could quietly solve on IPOPT and look identical in the artifacts. The template is now STAGED at session prep with SOLVER substituted from the solver mode, so the backend follows the selected tier instead of being an authoring decision the agent can get wrong. The placeholder lives inside a string, so an unstaged template still parses and degrades to a local IPOPT solve rather than a syntax error. 2. Altissimo produced NO telemetry on the shipped stack. Piccolissimo 0.2.0 forwards a hardcoded kwarg whitelist to Altissimo.optimize! and drops `callback`, and `AltissimoOptions.verbose` defaults false — so neither the callback nor the iteration table existed, and a cloud solve reported iterations = 0 with an empty Run Inspector. Now: verbose = true, plus a stdout bridge that translates Altissimo's own table rows into AMICODE_ITER. Verified locally at 154 streamed iterations (objective 48 -> 7.8e-3) with the callback never firing. Where a newer Piccolissimo DOES forward the callback it supersedes the bridge, so there is one numbering scheme per run. 3. A defaulted `amico-run script.jl` under the HP tier exited 64 and relied on the agent reading the refusal and retrying with --executor remote — users saw that round-trip as a failed run. It is now PROMOTED to remote. An explicit --executor local is still refused, and a promotion with no cloud connection refuses with the Connections panel instead of failing deep in RemoteExecutor with a cloud.json path. 4. Nothing in the UI said where a run executed: a cloud run and a local run produced an identical Inspector pane. The topbar now reads "<runId> · Harmoniqs Cloud" for remote runs, keyed on remote.json (written only by RemoteExecutor) rather than a new run.toml field — run.schema.json is additionalProperties:false. Also corrects four agent-guidance claims that no longer matched the code (the local-launch refusal, the "AMICODE_ITER not available on cloud" note, and the quoted SOLVER line), and states the two real remaining limits: cooperative Stop needs the callback the cloud bundle does not forward, and re-rollout verification is skipped for cloud runs. 941 amico-run tests and the extension suite pass; the two live-model e2e tests are unrelated (they need a working provider). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit put the cloud fact in the Run Inspector's run label — which is styled `mono small dim`, i.e. the least prominent element on the bar. The information was there and nobody would see it. Two changes: 1. A real badge, not appended text. The pill atom gains a `cloud` state: a FILLED brand-lemon swatch with a ☁ glyph, which is how this product says "special". It obeys the rule in brand.css — yellow is a FILL, never an ink — so the label is --color-on-accent (black, 18.7:1) and the edge is the theme-solved hairline, because the lemon cannot bound itself against white (1.1:1). Rendered and eyeballed in both themes. The badge is hidden unless the run is remote. No "LOCAL" counterpart on purpose: a badge present on every run is chrome users learn to ignore, and this one needs to still mean something on the run that costs money. 2. The status bar says it too — the one surface visible with the Inspector closed, and "am I burning cloud credits right now?" belongs to the question it already answers. A queued cloud run now says "queued" rather than "warming": warming describes Julia precompiling locally, which is exactly what does not happen on this tier, so the local word misdescribes where the wait is. The location shows only while the run is in flight — once it is done, its outcome is the story. Both surfaces derive location from remote.json via one seam. The six status-bar call sites now route through setRunState(), which stamps it in one place: six hand-maintained flags is how five get set and the sixth quietly does not. Cached permanently rather than on a TTL (unlike liveStatus, which it sits beside) because remote.json is written once at submit and never removed. Tests: the badge is asserted through the real DOM the user sees — appears on the run it was sent for, not on a sibling local run, never lights on a malformed message, and does not disturb the status pill beside it. Plus the brand-rule contract (no gold ink), glyph-survives-dot:false source ordering, and per-pane buffering so a panel reopen keeps the badge. `runLocationLabel` is deleted rather than left as a dead export. One test-helper fix falls out: `pillText` matched the first `.pill`, which is now the badge. It targets `.pill[role=status]` — the status pill's own semantic hook, the same one a screen reader uses. 886 extension tests pass (86 files). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… tier Field report: a solve ran locally with Piccolissimo + Altissimo selected. Reproduced from the real machine state, and the promotion logic was not at fault — the signal it read was: issimo entitlement GRANTED cloud.json present connection connected solver-mode.json piccolo (Jul 28) <- only this was read solver-mode.json only changes when something posts a `status:"switching"` request, so one dropped write leaves it stale indefinitely. Every cloud decision keyed off that single file, so the whole paid tier silently reverted: no HP guidance, a template staged for IPOPT, no promotion, and a local solve that looked completely normal. The fix is to stop trusting one mutable file: - amico-run's hpTierSelected() takes hp from EITHER solver-mode.json or the entitlement-resolved allowlist in authoring.json (Piccolissimo present ⇒ issimo granted). It already read that file for the gate, so this is a new reading of data in hand, not a new dependency. Self-cleaning: switching to Piccolo revokes the entitlement, so both signals agree again. - The extension's effectiveSolverMode() does the same for the three decisions that were wrong (routing guidance, template SOLVER, HP authoring guidance). - reconcileSolverMode() heals the file itself at activation, because it is a SHARED contract: the app's solver toggle renders from it, so a stale file showed "Piccolo" selected while the user was on the paid tier. Healing once at the source beats teaching every reader the same OR. It never touches a switch in flight (the watcher owns that write). OR, not AND, deliberately, and asymmetric: `hp` is sticky. Of the two ways to be wrong, silently giving someone a local IPOPT solve on the tier they are paying for is far worse than telling a revoked user to reconnect — the latter is visible and actionable. Nothing is stranded, because switching to Piccolo writes both halves. Also closes a live-cloud hazard this created. The allowlist signal meant any test launching without --executor read the DEVELOPER's real authoring.json, promoted, and submitted a billed staging job: estimate.test.ts sat 13.5 minutes polling the real cloud. The guard belongs in test/setup.ts next to the existing no-billed-model-calls and no-real-ledger guards — same reasoning, since the risk is in the test someone writes next — and it fails closed the same way, via paths that cannot exist. Side effect: the suite dropped from 655s to 36s, because tests are no longer waiting on a real endpoint. 944 amico-run tests and 896 extension tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nvented API
Third cloud solve, third hand-authored script, zero uses of the vetted
template. The latest (task 975a7c07, exit 1) invented API wholesale:
CubicSplinePulse(; T=, n_knots=, n_drives=, bounds=) every real method takes
POSITIONAL args, so an
all-keyword call matches
no method
CallbackLogger(qcp) not defined in Piccolo
or Piccolissimo
get_fidelity(qcp) not defined either
It died with a MethodError at LOAD time, before any optimization — after the
user had already paid the full queue and instance-boot wait. Verified by
running the script locally and by checking each symbol with isdefined against
both packages.
Naming the specific invented symbols is what made the earlier `using Piccolo`
guidance stick (#225), so the same treatment here rather than another abstract
"follow the template" line.
Worth being explicit about what this does NOT fix: guidance is not enforcement.
The durable version is a preflight that resolves the script's symbols against
the entitled packages before submitting, which needs the Julia stack — not
available on a cloud-only machine, which is the whole point of the tier. In the
meantime aws-infra#230 makes the failure VISIBLE: with the runner's output
redirected to run.log, a MethodError reaches the user's run dir instead of
dying invisibly in the SSM stream, which is why this one presented as a bare
"failed, exit 1".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Report: "if I say run a piccolissimo altissimo solve it might use piccolo
because the vetted template uses piccolo." Correct, and there were three
reasons, compounding:
templates/solve_template.jl Piccolo + IPOPT the ONLY vetted registry
entry, so tier resolution
can return nothing else
templates/solve_template_hp.jl Piccolissimo + IPOPT never registered at all —
and despite the name it
called solve!(qcp;
max_iter, print_level=1),
with zero AltissimoOptions
scores/…/templates/solve.jl either backend the only one that really
runs Altissimo (dispatch
bridge, verbose table
telemetry, run.log emit,
honest-result guard)
So even picking the "HP template" got IPOPT. Two files claiming to be the HP
path, neither doing it, is what made the behaviour look nondeterministic.
Consolidated to the score template, staged with SOLVER substituted from the
effective mode. The mode branch now lives in ONE line inside one file instead of
selecting between two files that drift — which is exactly how solve_template_hp.jl
came to be an IPOPT script with an HP name. solve_template_hp.jl is deleted.
Not a new pair of buttons, deliberately. The axes are not independent: Altissimo
is the cloud GPU solver for the issimo stack, so Piccolo + Altissimo is not a
supported combination and Piccolissimo + IPOPT is precisely the accidental state
above. Four buttons would offer two combinations we cannot honour. The existing
single toggle is the right control; it just was not reaching the decision.
Safe for the free tier: the template imports Piccolissimo only on the altissimo
branch via `@eval using Piccolissimo`, and the import scanner is line-anchored
(`^\s*(using|import)`), so it does not see a guarded import — a piccolo-mode
script needs no entitled package. Verified against the scanner's regex rather
than assumed.
Still open, and worth its own change: `amico-run resolve` returns the registry
path, which is the bundled file with `{{SOLVER}}` unsubstituted (it degrades to
ipopt by design). Registering the consolidated template for tier-1 needs the
resolve output to point at the STAGED copy, which means threading it through
authoring.json. The session path — what AGENTS.md hands the agent — is correct as
of this commit.
902 extension tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by running an Altissimo solve against Piccolissimo 0.3.1 (which, unlike 0.2.0, DOES forward `callback` into Altissimo.optimize!). Both telemetry channels emitted and the iteration numbers collided: records 1..51 the stdout table bridge, counting INNER steps then 1..4 the callback, counting OUTER iterations sequence: 51 -> 1 -> 1 -> 2 -> 2 ... The Inspector plots that as a sawtooth, which reads as a diverging solve. The `alt_cb_fired` guard was reactive and therefore always too late: alt_cb fires at the END of an outer iteration, so the bridge has already translated every inner row of outer #1 before the guard can trip. The two scales cannot be reconciled (dense inner steps vs sparse outer iterations), so exactly one channel may emit and the choice has to be made BEFORE the solve. CB_FORWARDED decides it from the installed Piccolissimo version — 0.3.x forwards, 0.2.x drops it via a hardcoded kwarg whitelist — and prefers the callback wherever it exists, because that channel carries pulse frames as well as numbers. A version gate rather than introspection: kwarg forwarding is not detectable from the method object. alt_cb_fired stays as a belt for a backport that forwards on an older version. Verified on 0.3.1: iterations [1, 2, 3, 4], strictly increasing, one scheme, plus 4 AMICODE_PULSE records and the meta line — the frame channel that did not exist on 0.2.0, where alt_cb never ran at all. The closing NOTE now states which channel carried the run, because the two count different things and a reader comparing runs needs to know which scale they are looking at. Unchanged and still load-bearing: the DirectTrajOpt 0.9.7 dispatch bridge. Piccolissimo main still defines only the OLD extension point (0 `_solve`, 2 `Solvers.solve!`), so without the bridge an Altissimo solve on 0.3.1 is still a silent no-op reporting success. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ith real output
Diagnosis of three cloud failures in two days. Every one was a script that did not
come from the template, and that single fact explains BOTH symptoms — the failures
and the empty Run Inspector:
x-gate-transmon-56 authored from memory: CubicSplinePulse kwargs, CallbackLogger,
get_fidelity — none exist
x-gate-transmon-hpc REUSED a Jul-28 script verbatim; ZeroOrderPulse into
SplinePulseProblem, which Piccolo 1.19 rejects outright
x-gate-transmon-hpc-3 authored from memory: CubicSplinePulseProblem (undefined),
AltissimoOptions(intermediate_callback=…) (no such field)
Each reproduced locally and each symbol checked with isdefined/fieldnames, not
assumed. Note the middle one: ~/.amico/problems/<slug>/solve.jl persists across
sessions and the agent reuses it, so template improvements never reach an existing
slug.
A correction to my own earlier claim: cloud streaming does NOT need aws-infra#230.
The template's emit() appends to run.log whenever TASK_ID is set, and the runner
sets TASK_ID — so a template-derived script streams today. All three of these wrote
run.log zero times, which is the entire reason nothing reached the Inspector. The
evidence is in the run itself: /frames reports iter: 60 for a solve whose /stats was
empty. It ran 60 iterations and wrote 60 frames; only the numbers were lost. #230
remains worth deploying — it captures stderr, which is why these failures presented
as a bare "failed, exit 1" — but it is not the streaming blocker.
So: a cloud telemetry preflight. A remote launch whose script never writes run.log
is refused in about a second, naming the cause and the fix, instead of costing ten
minutes and naming nothing. A refusal rather than a warning because it is a
certainty, not a risk, and stderr warnings have not changed the behaviour. Gated on
hasCloudConfig() so the missing-connection error keeps precedence — it is the more
fundamental problem. Local runs are untouched: LocalExecutor pipes the child's
stdout into run.log itself (local_executor.ts:293), which is why local has always
worked and why this must be remote-only.
Verification, and the reason it is worth trusting this time: FakeCloud now derives
BOTH /stats and /pulse from ONE run.log through the deployed lambda's own transform
(statsFromRunLog is a line-for-line port), so tests see the records the live service
actually returns — {raw: "iter=7 f=…"}, key=value, never pre-parsed JSON. Seeding
what the CLIENT expects is how three drift bugs shipped green. The end-to-end test
drives a run.log CAPTURED from a real Altissimo solve on Piccolissimo 0.3.1
(test/fixtures/altissimo_cloud_run.log) and asserts all four iterations arrive with
the real objectives, plus the pulse meta.
Chain status: template+TASK_ID → run.log (verified, real solve); S3 sync (proven in
production — frames arrive this way); /stats grep (verified against the live lambda);
client {raw} parse → local run.log (verified); tailer → Inspector (verified with
captured output). Every link but the S3 sync exercised locally.
948 amico-run + 903 extension tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The preflight worked — the agent copied the template byte-for-byte (401 lines,
diff-clean against the staged copy) and the telemetry chain came alive: task 582a
returned AMICODE_PULSE_META through /pulse into the local run.log, the first live
evidence of that path working.
The solve still failed, and I could not say why. julia's stderr goes to the SSM
command stream, which no API exposes, so a cloud failure arrives as `failed, exit 1`
and nothing else. Since it reached AMICODE_PULSE_META, it died AFTER problem
construction — and that is all the run could tell me.
Two additions, both inside the template and neither needing an infra change:
1. report_and_rethrow — any exception from the solve onward is emitted as
AMICODE_ERROR lines (message + 12 stack frames) via emit(), which writes
run.log, which the sidecar syncs and the poller greps. Rethrown, never
swallowed: the run must still fail. A solve reporting success after an exception
is the silent-no-op class this template already guards against.
2. AMICODE_ENV — a stack fingerprint emitted before the solve:
piccolo=1.19.0 piccolissimo=0.3.1 dto=0.9.7 callback_forwarded=true
julia=1.12.3 solver=altissimo
A cloud run executes against the runner's BAKED bundle, not the caller's
environment, so "which Piccolissimo was that?" is unanswerable after the fact —
and it is the first question every cloud failure raises. It also records which
telemetry channel is live, since the two count different things (outer
iterations vs inner steps).
Both verified by running, not by reading: the reporter writes the stacktrace and
still exits 1, and the fingerprint line above is from a real solve. The first
placement of the fingerprint referenced CB_FORWARDED before it was defined and
threw UndefVarError at load — caught only because I ran it, which is the argument
for running it.
What this does NOT do is fix the underlying cloud failure. It makes the next one
self-diagnosing instead of silent, which is the prerequisite for fixing it: the
cloud image's stack is still unknown to me, and the fingerprint is how we learn it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s the toggle
The last way an HP run could still land on Piccolo/IPOPT. AGENTS.md documents the
vetted path as "copy template_path", and template_path was always the BUNDLED
registry file — whose {{SOLVER}} placeholder is unsubstituted and, by design,
degrades to :ipopt. Meanwhile the AGENTS.md preamble pointed at the STAGED copy,
with SOLVER already resolved. Two instructions, two different files, and the
documented one was wrong.
So session prep now records the staged path in authoring.json (the seam amico-run
already reads for the allowlist and asset paths) and `resolve` returns it as
template_path. The two instructions now name the same file.
Falls back to the registry path when nothing was staged (a bare dev invocation) or
when staged_template points at a file that no longer exists — a stale authoring.json
must not send the agent to a missing path, which would be a worse failure than the
one being fixed.
Verified against the real machine state, not just in tests: resolve --platform
transmon --kind gate_synthesis --size 1 now returns the staged solve.jl whose SOLVER
line reads "altissimo". Before this it returned templates/solve_template.jl, which
is Piccolo + IPOPT.
951 amico-run + 903 extension tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
First live cloud run to stream iterations (task c584: 60 records, strictly monotonic 1→60, inf_pr 78.6 → 1.24e-05) exposed a defect in my own regex. ALT_ROW's `[-+0-9.eE]+` also matches the bare "-" Altissimo prints as a placeholder in columns that do not apply to a row, so the first record reached the Inspector as `AMICODE_ITER iter=1 f=-` — a non-numeric objective at the head of the curve. Captures are now tryparse'd as Float64 and the row is skipped unless all three are real numbers. This does NOT fully explain that run's second record (`f=1`), which is a plausible number in the wrong column — evidence that the cloud's baked Altissimo prints a different table layout than the local one, and that scraping a table is inherently version-coupled in a way the callback channel is not. Worth knowing before trusting the first points of a cloud curve; the fix is the runner getting a Piccolissimo that forwards `callback` (0.3.x), which retires the bridge entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Selecting Piccolissimo + Altissimo is supposed to mean one thing: every solve runs Altissimo, in Harmoniqs Cloud, without anyone deciding anything. We wired that quickly for the hackathon and four gaps were left between the promise and the code. This closes them and adds the tests that would have caught each one.
1. Nothing actually selected Altissimo
The solve template shipped
SOLVER = :ipopt. Choosing the paid cloud tier changed the entitlement, the routing, and the UI — but not the backend. A "High-Performance" run could solve on IPOPT and be indistinguishable in the artifacts: same run dir, same frames, same converged badge.The template is now staged at session prep with
SOLVERsubstituted from the solver mode, so the backend follows the selected tier instead of being an authoring decision the agent can get wrong. The placeholder lives inside a string literal (SOLVER = let s = "{{SOLVER}}" …), so an unstaged template still parses and degrades to a local IPOPT solve rather than dying on a syntax error.2. Altissimo produced no telemetry at all on the shipped stack
This is why cloud runs came back with an empty Run Inspector. Two independent causes, both silent:
solve!(prob, ::AltissimoOptions; kwargs...)forwards a hardcoded whitelist toAltissimo.optimize!.callbackis not on it, so the callback we hang telemetry on never fires.AltissimoOptions.verbosedefaults tofalse, so the iteration table never printed either.Neither path existed, so a run reported
iterations = 0and looked like it had converged. Now:verbose = true(load-bearing, not chatter — it is plumbed toverbose_outer/verbose_inner), plus a stdout bridge that translates Altissimo's own table rows intoAMICODE_ITER. Numbering is sequential over rows because the table'sitercolumn is·for every inner step and only becomes an integer on the final outer iteration — trusting that column yields a single point at the end.Verified locally: 154 streamed iterations, objective 48 → 7.8e-3, with the callback never firing once. Where a newer Piccolissimo does forward the callback it supersedes the bridge (it also carries frames), so there is exactly one numbering scheme per run.
Also still in: the DirectTrajOpt 0.9.7
_solvedispatch bridge, without which every Altissimo solve is a silent no-op that reports success.3. The default launch failed instead of going to the cloud
A plain
amico-run script.jlunder the HP tier exited 64 with a message telling the agent to retry with--executor remote. That round-trip surfaced to users as a failed run.It is now promoted to remote — selecting the cloud tier is the routing decision. An explicit
--executor localis still refused, because silently inverting a flag someone typed is worse than an error. And a promotion with no cloud connection refuses with "Connect Harmoniqs Cloud in the Connections panel" rather than failing deep inRemoteExecutorwith a~/.amico/cloud.jsonpath the user has never heard of.4. The UI never said where a run ran
A cloud run and a local run produced an identical Inspector pane, so "did that actually use the cloud I'm paying for?" was unanswerable from the UI. The topbar now reads
<runId> · Harmoniqs Cloudfor remote runs.Keyed on
remote.json(written byRemoteExecutorand nothing else) rather than a newrun.tomlfield —run.schema.jsonisadditionalProperties: false, which is why the remote executor puttask_idin that sidecar to begin with.Guidance corrections
Four claims in the agent guidance no longer matched the code and are now accurate: the local-launch refusal (it promotes), the "
AMICODE_ITERnot yet available on the cloud bundle" note (it is), and the quotedSOLVER = :altissimoline (the guidance now says the line arrives already set, so the agent doesn't "fix" it).Two real limits are now stated plainly instead of being papered over: the cooperative Stop needs the solver callback the cloud bundle's Piccolissimo does not forward, and re-rollout verification is skipped for cloud runs.
Tests
--juliathat writes a sentinel file if it ever runs — it doesn't, andAMICODE_FINISHEDcomes from FakeCloud.--executor localstill exits 64; the no---specpath is covered (the gate never runs there).altissimo, piccolo →ipopt, agent and grants point at the same staged file, and the shipped template is asserted to keep its placeholder inside a string.remote_statemachine.test.ts: a realRemoteExecutor.submitwrites theremote.jsonthe label reads.941 amico-run tests and the extension suite pass. The two
slow/*_e2etests need a live model provider and fail identically onmain.🤖 Generated with Claude Code