Skip to content

feat(cli): forge try — instant-gratification onboarding (#350)#353

Open
initializ-mk wants to merge 9 commits into
mainfrom
feat/forge-try-350
Open

feat(cli): forge try — instant-gratification onboarding (#350)#353
initializ-mk wants to merge 9 commits into
mainfrom
feat/forge-try-350

Conversation

@initializ-mk

Copy link
Copy Markdown
Contributor

Implements #350: a single forge try command that takes a first-time user from install to talking to a working agent whose loop they can watch, in under 60 seconds, with zero required decisions.

Stacked on #351 (keyless weather skill). The first commit (fix(skills): weather … wttr.in) belongs to #351 and dedupes on rebase once #351 lands on main — the demo hero needs a keyless weather skill. Review the seven forge try commits.

What it does

$ forge try
  Using OpenAI (signed in).
  Agent: quickstart · skills: weather · tools: http_request, datetime_now, math_calculate

you › what's the weather in Tokyo, should I pack an umbrella?
  ▸ tool   weather_current(location=Tokyo)
  ▸ egress wttr.in   ✓ allowed
  ◂ 18C, light rain this evening
agent › 18C in Tokyo with light rain tonight. Yes, take the umbrella.
  audit  {"tools":["weather_current"],"egress":["wttr.in:allow"]}

The differentiator: the agent's own audit stream (tool calls, egress checks, guardrail blocks) renders inline — the same signal the enterprise story rests on, surfaced as first-run delight.

Design (per the plan, locked)

  • In-process, not HTTP. No A2A server, daemon, or port binding. A new runtime.LocalSession assembles the same coreruntime.LLMExecutor forge run uses (builtin tools + vendored skills, egress-enforced client + subprocess proxy, audit + progress hooks, provider client) minus the server/scheduler/MCP/memory. No second executor — a trimmed bootstrap around the shared sub-builders; Run() is untouched (zero risk to the server path).
  • Ephemeral by default. Scaffolds into a temp dir cleaned on exit; --keep writes ./forge-quickstart. No secrets on disk for the ephemeral run.
  • Reuses scaffold() via two initOptions fields (OutputDir, Preset) — no forked init logic.
  • Credential auto-resolution layered over existing resolution: flags → env key (ANTHROPIC>OPENAI>GEMINI) → saved OpenAI OAuth → local Ollama → picker (TTY) → actionable error (no TTY).
  • Renderer consumes existing audit events only (no new constants); lives in forge-cli, degrades to plain text (NO_COLOR / non-TTY). --quiet hides it, --audit shows full NDJSON.

Phases (one commit each)

  1. Command skeleton + demo preset + ephemeral scaffold
  2. Model credential auto-resolution
  3. In-process REPL driving the executor
  4. Visible-loop renderer
  5. First-run polish + graduation hook
  6. Docs: quick-start leads with forge try; pipeline moved to new ship-to-production.md
  7. Tests

Verification

  • Real turn via a mock OpenAI server (LocalSession makes the LLM call, history accumulates); end-to-end test renders a real datetime_now tool call.
  • 12 renderer unit tests (event→line mapping, truncation, quiet, --audit echo, no-color); resolveTryProvider ordering + workspace-lifecycle tests.
  • go build ./..., go vet ./..., go test ./forge-cli/... all green; gofmt + golangci-lint clean; docs broken-link check clean; no channels in the quick start.

Out of scope (named, not built)

Hosted metered demo key, forge try --ui, a second demo agent — all deferred per the plan.

…APIs (#349)

The bundled weather skill allowed egress only to api.openweathermap.org
and api.weatherapi.com — both require an API key — yet declared no key
env at all, so it was dead on arrival. Its tool bodies were also prose
only, with no runnable command.

Repoint it at wttr.in (free, keyless, accepts a city name directly so it
maps 1:1 to the location input) and ship two runnable scripts:

- scripts/weather-current.sh  '{"location":"Tokyo"}'
- scripts/weather-forecast.sh '{"location":"Tokyo","days":3}'

Both use curl + jq, need no API key, validate input, check the HTTP
status, and emit shaped JSON. egress_domains is now [wttr.in].

Kept in sync: the egress-wizard domain hints (egress_step.go), the
scanner test fixture, and the weather example in .claude/skills/forge.md.

Verified live against wttr.in (current + forecast); scripts are
shellcheck-clean; forge-skills and forge-cli/internal/tui suites green.
…ffold (#350)

New `forge try` command that scaffolds a keyless demo agent (the native
forge LLM executor + weather skill + http_request/datetime_now/math_calculate
builtins) and prints its summary. Ephemeral temp workspace by default,
cleaned on exit; --keep writes ./forge-quickstart.

Reuses scaffold() via two additions to initOptions (no duplication):
- OutputDir overrides the target dir (temp dir / ./forge-quickstart)
- Preset stops scaffold() before its banner + auto-run of `forge run`,
  so try drives the agent in-process (Phases 2-4).

Egress auto-derives from the weather skill (allowlist, wttr.in — keyless;
depends on #351). No channels, no secrets on disk. No LLM call yet.

Verified: ephemeral scaffold + summary + cleanup (no cwd leak); --keep
writes a valid forge.yaml; init suite green; gofmt/vet/lint clean.
resolveTryProvider layers an ordering policy over the runner's existing
resolution: explicit flags -> env key (ANTHROPIC > OPENAI > GEMINI) ->
saved OpenAI OAuth -> local Ollama (short-dial probe) -> interactive
picker (TTY) -> actionable error (no TTY). Prints a human 'Using ...'
label. Picker offers OpenAI sign-in (runOAuthFlow), paste-key (masked,
in-memory), or Ollama. No new credential store; SilenceUsage so the
no-credential error stays clean.

Verified: env-key path silent + correct label with the documented
precedence; no-creds/no-TTY yields the actionable error and exit 1;
explicit flags still win. gofmt/vet/lint clean.
…350)

New forge-cli/runtime/LocalSession assembles the SAME coreruntime.LLMExecutor
that forge run uses (builtin tools + vendored skills, egress-enforced client +
subprocess proxy, audit + progress hooks, provider client) WITHOUT an HTTP
server, scheduler, MCP, or long-term memory. No second executor — a trimmed
bootstrap around the shared sub-builders. History rides in task.History; the
executor Store is nil so nothing persists.

try.go drives it: RunTurn per line, streaming the reply under 'agent ›',
/exit + Ctrl-D + Ctrl-C (signal.NotifyContext) exit, --once single turn, an
optional positional prompt seeds the first turn. The runner's JSON logger is
silenced so stdout stays clean for the chat.

Verified: real turn via a mock OpenAI server (llm_call fires, reply returned,
history accumulates to 2 then 4 across turns); session construction (egress
proxy + skill registration + client build) succeeds live; runtime + cmd
suites green; gofmt/vet/lint clean.
New forge-cli/internal/tryview.Renderer implements the audit Sink contract and
maps the agent's own audit events to inline loop lines: tool calls
(▸ tool name(args)), egress checks (▸ egress domain ✓/✗), guardrail blocks,
and tool results (◂ preview), plus a dim compact 'audit {…}' summary after
each reply. lipgloss orange accent, degrades to plain text with NO_COLOR /
non-TTY. --quiet hides the loop; --audit echoes full NDJSON.

LocalSession now builds its audit logger on a discard base (no stderr noise)
and captures tool args/results (redacted) so the renderer can preview them;
try.go attaches the renderer as an extra sink. Reads existing events only —
no new audit constants, no forge-core TTY code.

Verified: 12 renderer unit tests (event→line mapping, truncation, quiet,
--audit echo, no-color degradation, summary reset) + an end-to-end
LocalSession test rendering a real datetime_now tool call. Suites green;
gofmt/lint clean.
Add the intro banner, three starter prompts (all within the keyless egress
allowlist so none dead-ends on a blocked domain), and an exit graduation
message that points the user up the ladder: keep the demo (--keep writes
./forge-quickstart), run it with forge serve, or deploy with forge package.
Orange accent (lipgloss) on command tokens only; degrades to plain text on
non-TTY / NO_COLOR. No em-dashes in printed strings (house style).

Verified visually: banner + summary + suggestions render for the interactive
REPL; graduation prints on exit (keep-aware); --once stays terse. Build +
lint clean.
cmd/try_test.go: quickstartPreset scaffolds a valid, keyless forge.yaml
(egress wttr.in, never the key-gated hosts); resolveTryProvider ordering
(flags win, ANTHROPIC>OPENAI env precedence, Ollama probe via a live
listener, no-creds/no-TTY error with HOME isolated so a dev's real OAuth
token can't leak in); tryWorkspace lifecycle (--keep targets cwd with no
cleanup; ephemeral temp base is removed).

Complements local_session_test.go (real turn + tool-loop render) and the
tryview renderer suite. All green; vet + lint clean.
…production (#350)

Rewrite quick-start.md around a single command (brew install + forge try),
the hero transcript, and a 4-rung ladder (try -> keep+skill -> serve+guardrail
-> package). Channels/build/deploy are gone from the quick start.

New ship-to-production.md carries the full init -> skills -> secrets ->
validate -> run -> build -> package pipeline. README Quick Start leads with
forge try + transcript and links both pages; forge.md §15 gets a one-line
forge try pointer at the top.

Verified: broken-link check clean; no channel steps remain in quick-start.

@initializ-mk initializ-mk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: forge try — instant-gratification onboarding

Reviewed the seven forge try commits (skipped the leading #351 weather commit — already reviewed on #351; it dedupes on rebase as noted). Strong, well-architected PR; the security-critical claims hold up under tracing. Merge-ready — findings are one cross-PR merge-coordination item and four low/nit polish items.

What I verified holds (the parts that matter)

  • Genuinely reuses the shared executor, not a fork. LocalSession assembles the same coreruntime.LLMExecutor via the runner's sub-builders; Store is nil so history stays in memory and nothing persists. The "no second executor, Run() untouched" claim is accurate.
  • Egress is actually wired, not just constructed. buildTryEgress builds both the in-process enforced client (installed on ctx via WithEgressClient in RunTurn) and the subprocess proxy, both emitting egress_allowed/egress_blocked. This is exactly the "works in isolation vs. where it's mounted" trap this line of work keeps hitting — avoided here.
  • The paste-key secret doesn't leak into skill subprocesses. I chased this: try.go does os.Setenv on the pasted key, but SkillCommandExecutor.Run builds a minimal allowlisted env (clean slice, PATH+HOME, then only the skill's declared env vars + proxy/trace/OTel). The weather skill declares env.required: [], so it gets neither the key nor the ambient env. No leak in the shipped demo.
  • Fail-closed where it counts: no credential + no TTY → actionable error + exit 1; ephemeral temp dir removed via defer RemoveAll(base); no secrets on disk for the ephemeral run. CI fully green.

Findings (see inline)

  1. Medium (cross-PR merge coordination): buildTryEgress is a fourth copy of the egress-construction pattern, on the pre-#348 signature — compiles today only because #348 isn't merged; when #348 lands this won't compile until updated, and until then forge try silently ignores allowed_private_cidrs.
  2. Low: egress-resolve failure falls back to a fully-unenforced http.DefaultClient (fail-open).
  3. Low/nit: the paste-key comment claims the key is persisted under --keep, but the code never writes it to disk (the safe behavior) — reconcile comment + graduation text.
  4. Nit: --yes is a dead flag (parsed, never read).
  5. Nit (UX): Ctrl-C at the you › input prompt doesn't exit — signal.NotifyContext catches SIGINT but scanner.Scan() isn't ctx-aware, so it hangs until Enter/Ctrl-D. Honestly documented (Ctrl-D / /exit), but Ctrl-C is a reflex in a hero demo. Consider reading stdin in a goroutine and selecting on ctx.Done().

Nice work — the audit-stream-as-first-run-delight framing is a genuinely good idea, and the reuse discipline (one executor, trimmed bootstrap) is exactly right.

allowPrivateIPs = true
}

enforcer := security.NewEgressEnforcer(nil, egressCfg.Mode, egressCfg.AllDomains, allowPrivateIPs)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium (cross-PR merge coordination). This block (security.Resolve at L205, NewEgressEnforcer here at L224, NewEgressProxy at L240) is a fourth copy of the egress-construction pattern, and it's on the pre-#348 signatureResolve 5-arg, NewEgressEnforcer 4-arg, NewEgressProxy 2-arg. It compiles today only because #348 isn't merged yet. The moment #348 lands (which adds allowedPrivateCIDRs to all three), this file won't compile until updated, and until then forge try will silently ignore allowed_private_cidrs where the server path honors it. Since you own both PRs: land #348 first and rebase this onto it (threading allowedPrivateCIDRs through buildTryEgress), or track it as a required follow-up. Also directly reinforces #348's "consolidate the copies" note — this is now the copy most likely to diverge, since its comment says it "mirrors Run()'s egress setup" but it's a hand-maintained duplicate.

Comment thread forge-cli/runtime/local_session.go Outdated
)
if err != nil {
r.logger.Warn("egress resolve failed; using unenforced client", map[string]any{"error": err.Error()})
return http.DefaultClient, "", noop

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low. On a security.Resolve error this returns http.DefaultClient — a fully unenforced client — with only a Warn. That's a fail-open on a security path. Effectively unreachable for the fixed quickstart preset (its scaffolded egress config resolves fine), but the wrong default direction: a future config change that makes Resolve fail would silently drop egress enforcement entirely. Prefer a deny-all / empty-allowlist enforced client here so the failure mode stays closed.

Comment thread forge-cli/cmd/try.go Outdated
}

// pasteKeyResolution prompts for a provider and a masked API key, held only in
// memory (written to disk solely under --keep, via the scaffold env path).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low/nit — comment vs. code. This says the pasted key is "written to disk solely under --keep, via the scaffold env path," but it isn't: the key goes into res.EnvOverrides (session-only, consumed by NewLocalSession), while opts.EnvVars is always map[string]string{} — so the key is never persisted, even under --keep. The never-persist behavior is the correct, safe one and matches the PR's "no secrets on disk" promise, so just fix the comment. Worth surfacing the consequence in the graduation text too: forge try --keep + a pasted key writes ./forge-quickstart with no credential, so a later forge serve there needs the env var set by hand.

Comment thread forge-cli/cmd/try.go Outdated
f.onceSet = cmd.Flags().Changed("once")
f.quiet, _ = cmd.Flags().GetBool("quiet")
f.audit, _ = cmd.Flags().GetBool("audit")
f.yes, _ = cmd.Flags().GetBool("yes")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — dead flag. f.yes is parsed here but never read anywhere in runTry/downstream. It's advertised in --help ("assume yes to prompts") but does nothing. Either wire it (e.g. auto-select the picker's recommended default so --yes is truly non-interactive) or drop the flag.

…, dead flag)

- [Low] egress-resolve failure now fails CLOSED: a deny-all allowlist enforcer
  instead of an unenforced http.DefaultClient, so a future config that makes
  Resolve error can't silently drop egress enforcement.
- [Nit] drop the dead --yes flag (parsed, never read).
- [Low/nit] correct the paste-key comment: the key is session-only and never
  written to disk, even under --keep; note the consequence (a kept agent has no
  credential on disk).
- [Nit/UX] Ctrl-C at the 'you ›' prompt now exits: stdin reads on a goroutine
  and the REPL selects on ctx.Done() (scanner.Scan isn't ctx-aware).
- [Medium/#348 coupling] add an explicit NOTE that buildTryEgress is on the
  pre-#348 security signatures and must thread allowedPrivateCIDRs when #348
  lands (tracked; can't rebase onto an unmerged branch).

Build/vet/lint clean; cmd + runtime + tryview suites green.
@initializ-mk

Copy link
Copy Markdown
Contributor Author

Addressed the review in a16dc14:

Build/vet/lint clean; cmd + runtime + tryview suites green. Thanks for the careful trace on the egress-mount and paste-key-leak paths.

@initializ-mk initializ-mk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: fix commit a16dc140 — all actionable findings resolved

Verified each finding against the fix diff. CI fully green (all 6 builds, Lint, Test, Integration, Doc-link).

Finding 2 (Low — fail-open egress) — ✅ resolved

On security.Resolve error, buildTryEgress now returns a deny-all enforced client (NewEgressEnforcer(nil, ModeAllowlist, nil, allowPrivateIPs) — allowlist mode with no domains blocks everything) instead of the unenforced http.DefaultClient. Fail-closed, exactly as recommended. The allowPrivateIPs computation was correctly hoisted above the error branch so both paths share it.

Finding 4 (dead --yes) — ✅ resolved

Removed from the struct, init(), and parseTryFlags — no longer advertised in --help.

Finding 3 (paste-key comment) — ✅ resolved

Comment rewritten accurately: the key is session-only (res.EnvOverrides, consumed by NewLocalSession), never written to disk even under --keep, and it now spells out the consequence (a kept agent has no on-disk credential; a later forge serve needs the env var set by hand).

Finding 5 (Ctrl-C at the you › prompt) — ✅ resolved, correctly

stdin now reads on a goroutine feeding a lines channel; the REPL selects on ctx.Done() (Ctrl-C → newline + return), eof (Ctrl-D → return), or a line. The one-read goroutine leak on exit is real but harmless (the process is exiting) and honestly acknowledged in the comment. Mid-turn cancellation and graduation-on-exit both still work.

Finding 1 (#348 coupling) — appropriately deferred with an explicit NOTE

You can't rebase onto an unmerged branch, so the right move is what was done: a prominent in-code NOTE on buildTryEgress documenting that the three constructors are on pre-#348 signatures and must thread allowedPrivateCIDRs when #348 lands, cross-referencing #348's consolidation note. Keeps the landmine visible for whoever merges #348 second.

Verdict: all fixes correct, nothing dangling. Merge-ready (modulo the #348 sequencing the NOTE now tracks).

One optional residual, non-blocking and on an effectively-unreachable branch (the fixed preset always resolves): on the resolve-error path the in-process client is now fail-closed, but it still returns proxyURL="", so skill subprocesses would run unproxied rather than blocked. Fully fail-closing that dead branch would mean refusing to start when egress can't resolve — reasonable to leave as-is behind the NOTE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant