Add an Atomic SDK harness for the hosted Planner - #210
lavaman131 wants to merge 6 commits into
Conversation
|
The preview deployment for chopin is ready. 🟢 Open app | Open Build Logs | Open Application Logs Last updated at: 2026-10-01 16:21:12 CET |
4c685ef to
115df25
Compare
| policy.hostNames = hostNames; | ||
| policy.structured = false; | ||
| if (!live) { | ||
| directory ??= await mkdtemp(join(tmpdir(), "chopin-atomic-planner-")); |
There was a problem hiding this comment.
Closing a session while its first turn is still setting up can leave a temporary folder behind. I reproduced this by starting doPromptTurn() and immediately calling doDestroy(): cleanup finishes before the folder exists, then setup creates it and stops because the session is closed. The folder never gets deleted.
Could we delete the folder in the if (leak || closed) branch too? Putting that deletion in a finally block would ensure it happens even if session.dispose() fails. A test that closes the session during setup and checks the folder is gone would cover this.
There was a problem hiding this comment.
Thanks, reproduced and fixed in 12815c8. When the session closes during setup, the working directory is now deleted on that path, in a finally so a failed dispose() still removes it. The new lifecycle test closes the session while its first turn is still setting up and checks that no chopin-atomic-planner-* directory is left. It fails without the fix and passes with it.
Assistant-workflow: inline
Assistant-verification: fail-before/pass-after passed: the lifecycle test left a directory behind without the fix
Co-authored-by: Alex Lavaee lavaman131@github.com
HARNESS=atomic embeds Atomic 0.9.23 in the server process through its headless SDK (createAgentSession, ModelRuntime, DefaultResourceLoader, and in-memory session and settings managers). It does not spawn the atomic CLI, use RPC mode, or alias Pi's runtime. Each harness session owns one Atomic AgentSession in a private temporary directory. Every shipped Atomic package is off, the tool allowlist holds only the host tools and a result tool, and the loader discovers no extensions, skills, prompt templates, or context files. A per-turn hook replaces the whole system prompt, so the model never sees Atomic's coding-agent preamble. The adapter fails the turn before any model request if the live session reports an extra extension, tool, context file, skill, prompt template, or system prompt. It checks the tools offered to the model again before every model request. Host tools round-trip through submitToolResult and settle on abort. Structured output uses a terminating result tool enabled only from the turn's JSON response format, which captures the calling model's arguments without a second inference. An unrecognized provider/model fails the turn instead of letting Atomic pick another model, and MODEL is now required for HARNESS=atomic as it is for Pi. HARNESS_AUTH is required and is either auto or ai-gateway. auto copies the host's Atomic login into memory and is refused off loopback; ai-gateway accepts only vercel-ai-gateway models. Credentials are never written to disk. The Pi auth checks now share the same table with unchanged messages. Docs cover selection, trust, credentials, and Atomic's caveats. Assistant-workflow: ralph (run 0cfa5220-f488-43ef-8bb5-2f54cf5141f2) Assistant-model: Claude Opus 5.5 Assistant-duration: 30m converged, estimated 65m Assistant-verification: bun test passed: 1683 pass, 2 PostgreSQL skips, 0 fail, including the atomic contract suite and the unchanged copilot-sdk and pi suites Assistant-verification: bun run types passed: all workspaces and E2E Assistant-verification: bun run ci passed: dprint, oxlint with no errors, tokens Assistant-verification: bun run fix passed: only formatting changes, inspected Assistant-verification: mutation checks passed: turning builtins back on, enabling skill discovery, or offering the result tool on plain turns each fail the atomic suite Assistant-verification: agent-browser E2E passed: a real Chopin server with HARNESS=atomic and a local stub model ran an @chopin turn that called read_plan and streamed the reply into Chat; the model was offered exactly PLANNER_TOOL_NAMES and Chopin's Planner prompt Assistant-verification: qlty smells passed: no blocking findings; complexity warnings match the Copilot adapter's closure shape User-preference: Build the Atomic harness on Atomic's headless SDK, which separates runtime from hosts Co-authored-by: Alex Lavaee <lavaman131@github.com>
The ralph run's research stage mapped Atomic's SDK onto HarnessV1: event translation, host tool round trips, the tool boundary, auth modes, and caveats such as no cross-process resume and oversized tool result truncation. Keep it with the branch so review can continue on another machine. The run was stopped during its first review round. Both reviewers had no blocking findings; reviewer B noted one doc nit: self-hosting.md names only ATOMIC_CODING_AGENT_DIR, but Atomic also honors legacy PI_CODING_AGENT_DIR for the auto-mode auth.json path. The Docker build could not be verified on this machine because its Docker Desktop network fails TLS to registry.npmjs.org (npm.pkg.github.com and api.github.com still work), so bun install inside the image cannot fetch the new lockfile entries. Assistant-workflow: ralph (run 0cfa5220-f488-43ef-8bb5-2f54cf5141f2) Assistant-model: Claude Opus 5.5 Assistant-duration: 72m abandoned, estimated 90m Assistant-verification: docker build unavailable: container TLS to registry.npmjs.org fails with UNKNOWN_CERTIFICATE_VERIFICATION_ERROR on this Docker Desktop host User-preference: Stop the workflow and hand off when the local environment blocks verification; continue on another machine Co-authored-by: Alex Lavaee <lavaman131@github.com>
Atomic reads PI_CODING_AGENT_DIR when ATOMIC_CODING_AGENT_DIR is unset, so the auto-mode auth.json lookup lists both variables in precedence order. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: source check passed: @bastani/atomic 0.9.23 dist/config.js getEnvValue prefers ATOMIC_CODING_AGENT_DIR, then PI_CODING_AGENT_DIR, and a set variable disables the ~/.pi fallback Assistant-verification: dprint ci passed: bun run ci Co-authored-by: Alex Lavaee <lavaman131@github.com>
The release keeps the same five builtin packages, so BUILTINS_OFF still covers every one and the Planner session stays isolated to Chopin's host tools. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: bun test passed: 1688 pass, 2 PostgreSQL skips, 0 fail, including the atomic contract and isolation suites Assistant-verification: bun run types passed: all workspaces and E2E Assistant-verification: bun run ci passed: dprint, oxlint, tokens, type scale, Impeccable baseline Assistant-verification: source check passed: @bastani/atomic 0.9.24-alpha.1 dist/builtin lists intercom, mcp, subagents, web-access, workflows, matching BUILTINS_OFF Co-authored-by: Alex Lavaee <lavaman131@github.com>
08da2e5 to
7544ed0
Compare
Closing an Atomic session while its first turn was still setting up ran cleanup before the working directory existed; setup then created it, saw the session closed, and stopped without deleting it. Delete the directory on that closed path, in a finally so a failed dispose still removes it. Reported by Maggie Appleton in review. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: fail-before/pass-after passed: the new lifecycle test left a chopin-atomic-planner directory behind without the fix and passes with it Assistant-verification: bun test passed: 1824 pass, 2 PostgreSQL skips, 0 fail Assistant-verification: bun run types passed: all workspaces Assistant-verification: bun run ci passed: dprint, oxlint, tokens, design contract, design record, Impeccable (no new findings) Co-authored-by: Alex Lavaee <lavaman131@github.com>
0.9.25 is the stable release that rolls up the 0.9.25 prereleases, including SDK workflow run control and the native MCP client. It ships the same five builtin packages, so BUILTINS_OFF still turns off each one for the isolated Planner. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: bun test passed: 1824 pass, 2 PostgreSQL skips, 0 fail Assistant-verification: bun run types passed: all workspaces Assistant-verification: bun run ci passed: dprint, oxlint, tokens, design contract, design record, Impeccable (no new findings) Assistant-verification: source check passed: @bastani/atomic 0.9.25 dist/builtin lists intercom, mcp, subagents, web-access, workflows Co-authored-by: Alex Lavaee <lavaman131@github.com>
Summary
Adds
HARNESS=atomic, aHarnessV1adapter (apps/server/src/harness/atomic/adapter.ts) that embeds@bastani/atomic0.9.23 in the Chopin server process through its headless SDK:createAgentSession,ModelRuntime,DefaultResourceLoader, and in-memory session and settings managers. It does not spawn theatomicCLI, use RPC mode, or alias Pi. Thecopilot-sdkandpiharnesses are unchanged.07c5852adds the adapter, its unit tests (atomic/adapter.test.ts), the shared contract suite (harness/atomic.contract.test.ts),harnesses.ts/config.tswiring and tests, and docs (docs/hosted-agent.md,docs/self-hosting.md,docs/architecture.md,README.md).878147akeeps the research notes that map the SDK toHarnessV1and list the caveats.4c685efdocuments thatautoalso honors the legacyPI_CODING_AGENT_DIR. Atomic checksATOMIC_CODING_AGENT_DIRfirst, thenPI_CODING_AGENT_DIR, and a set variable disables the~/.pifallback.This branch was rebased onto
main(99204aa). The only conflict wasbun.lock. I regenerated it from main's lockfile, and it matches the branch's original lockfile except for main'simpeccableentries.Security boundary
MODELfails the turn before any model request. Atomic would otherwise silently pick another model.createStructuredOutputTool, which makes a second model call.Auth modes
HARNESS=atomicrequires an explicitHARNESS_AUTH:autocopies the host's Atomic login into memory ($ATOMIC_CODING_AGENT_DIR, then$PI_CODING_AGENT_DIR, otherwise~/.atomic/agentover~/.pi/agent), plus env keys and ambient cloud credentials. Startup refuses it unlessSERVER_HOSTis loopback-only.ai-gatewayreadsAI_GATEWAY_API_KEYand accepts onlyvercel-ai-gatewaymodels. It is the only mode allowed on a public bind.No credentials,
models.json, ormodels-store.jsonare written to disk.Documented caveats
@bastani/atomic.auto, an in-memory OAuth refresh can rotate the host CLI's refresh token.!commandAPI-key entries inauth.jsonare executed.maxAiCreditsdoes not apply.@bastani/atomicadds more than 250 MB of dependencies. The rebuiltchopin-atomicimage is 2.1 GB.typebox1.3.27 while Chopin pins 1.3.7. Host tool schemas cross as plain JSON Schema, so the two versions never meet.Verification on this machine
These checks ran on a new checkout after the rebase. Earlier results from the original machine are in the
07c5852commit trailers.The Playwright suite does not exercise
HARNESS=atomic; its agent servers useAGENT=offor the fake harness. I tested the Atomic harness separately in a real browser against a real model.Browser test of the Atomic harness with a live model
I started a server on loopback with
AGENT=on HARNESS=atomic HARNESS_AUTH=auto MODEL=github-copilot/gpt-6-luna, using the E2E preloads (e2e/github.tsande2e/harness/preload.ts). Those preloads fake only GitHub's REST and OAuth endpoints and the GitHub MCP URL. Model traffic went to GitHub Copilot through the host's Atomic login. With agent-browser I signed in through the server's real OAuth callback, addedocto-org/scorein the UI, created a document, typed a sentence, and sent two@chopinturns.read_planandlist_pull_requests, then answered with the document's sentence and the fixture PR title "Restring the harp section". The fake MCP log showslist_pull_requestswas called forocto-org/scorewith a bearer token,readonly=true, andtoolsets=pull_requests.read_planand thenedit_plantwice to append a three-item list and keep the existing sentence. The document and the transcript were unchanged after a page reload.The server's
[agent]diagnostic listed exactly Chopin's 16 host tools pluschopin_submit_atomic_result. It showed nobash, filesystem, fetch, or Intercom tool. Nothing in~/.atomic/agentwas modified during the run.Assistant-verification: docker build passed:
docker build --no-cache -t chopin-atomic .completed; bothbun install --frozen-lockfilestages fetched from registry.npmjs.org (build 995 packages, production 908 packages)Assistant-verification: docker runtime import passed:
docker run --rm chopin-atomic bun -e "await import('./apps/server/src/harness/harnesses.ts')"and a direct import ofatomic/adapter.tsboth loaded@bastani/atomicin the production imageAssistant-verification: bun run fix passed: no changes beyond the doc edit; one existing oxlint warning in the untouched
github-tools.test.tsAssistant-verification: bun test passed: 1688 pass, 2 PostgreSQL skips, 0 fail across 190 files;
apps/server/src/harnessalone 127 pass, covering atomic, copilot-sdk and piAssistant-verification: bun run types passed: all workspaces and E2E
Assistant-verification: bun run ci passed: dprint, oxlint (0 errors), tokens, type scale, Impeccable design check
Assistant-verification: bun run e2e passed: 236 Chromium tests after confirming ports 8788, 8789, 8791, 8792 and 8797 were free
Assistant-verification: agent-browser E2E passed: live github-copilot/gpt-6-luna turns through HARNESS=atomic HARNESS_AUTH=auto ran read_plan, list_pull_requests over the fake GitHub MCP (bearer, readonly, pull_requests toolset) and edit_plan; the edit and transcript persisted across reload; ~/.atomic/agent was not modified
Assistant-verification: atomic source check passed:
@bastani/atomic0.9.23dist/config.jsreadsATOMIC_CODING_AGENT_DIR, thenPI_CODING_AGENT_DIRAssistant-model: Claude Opus 5.5
Assistant-workflow: inline
Assistant-duration: 12m converged, estimated none given
User-preference: Build the Atomic harness on Atomic's headless SDK, which separates runtime from hosts
Co-authored-by: Alex Lavaee lavaman131@github.com