Telemetry: traces are rooted at Cli.serve, so one traceId covers the whole process and every session

[jack-champagne]

Summary

Every span emitted by a running opencode serve process inherits the Cli.serve span, so the entire process lifetime is one trace. A 2.15-hour run produced a single traceId containing 129,839 spans across 4 distinct user sessions, with no root span (because Cli.serve had not ended).

This is the reason the run corpus is hard to use: you cannot open, replay, or reason about a single session's trace.

Evidence

From one production run (opencode 1.18.10), reassembled from 2,593 raw OTLP objects:

spans=129,881   unique spanIds=129,881   duplicates=0
traces=18       largest=129,839 spans (99.97% of the session)
roots present     : 0
missing parents   : 1  -> ['ac1aa69859552660']
orphaned spans    : 2,044
unfinished spans  : 0

All 112 ai.streamText spans share one traceId, spanning four different sessions:

ses_02c8d2f84ffesgKDB29aW9xeK4  ->  c38d28bc11add9ef…  (43 calls)
ses_0353c4111ffeLhfnDa31W27s3E  ->  c38d28bc11add9ef…  (42 calls)
ses_02c7ff990ffeHhvgjYsKJxTU3G  ->  c38d28bc11add9ef…  (15 calls)
ses_02cd9bda7ffeGVkpyM1IsaH1aB  ->  c38d28bc11add9ef…  (12 calls)

The missing root is Cli.serve. Its 2,044 direct children span 50 distinct span names:

1561  http.server GET
 114  InstanceStore.load
  62  Session.get
  60  ToolRegistry.register
  19  SessionHttpApi.get
  ...

Cli.serve (packages/opencode/src/cli/cmd/serve.ts:13, handler: Effect.fn("Cli.serve")(...)) is absent from the trace while Cli.resolveNetworkOptions — one of its children — is present. That is only consistent with Cli.serve being the open, never-exported root.

Consequences

1. No root until the process exits. BatchSpanProcessor flushes on span end. If the process is killed (VS Code quit, crash, container stop), the root is never exported and the trace is permanently rootless.
2. Unbounded trace growth. Span count scales with process uptime, not with work. 130k spans after 2.15 h.
3. No viewer can open it. Phoenix and Langfuse both choke well below this size.
4. Sessions are not separable. Four users' sessions share one traceId; isolating one requires filtering on the ai.telemetry.metadata.sessionId span attribute rather than selecting a trace.

Note this is independent of the per-delta read amplification filed alongside. Fixing that takes the trace to ~35,500 spans — still one unbounded trace covering four sessions.

Suggested direction

SessionPrompt.prompt occurs exactly 8 times in this run — one per user turn across the 4 sessions. It is the natural trace boundary and is already instrumented (packages/opencode/src/session/prompt.ts; SessionPrompt.run at :1083, SessionPrompt.loop at :1491, all present in the trace).

Starting a new root span per prompt rather than inheriting Cli.serve would give:

	today	+ delta fix	+ delta fix + prompt-rooted
traces	1	1	8
spans/trace	129,839	~35,500	~4,400
root closes after	process exit	process exit	one turn

At ~4,400 spans a trace opens in any viewer, the root closes within minutes so a hard kill costs one turn instead of the whole session, and corpus objects map to real sessions.

Unverified: the exact Effect API for starting a detached/root span. This repo is on effect@4.0.0-beta.83 and dependencies are not installed in my checkout, so I could not confirm whether Effect.withSpan(name, { root: true }) (or an equivalent) is available in that beta. That needs checking before implementation — the behaviour above is the requirement, not the call signature.

Notes

• serve.ts and processor.ts are pure upstream — no harmoniqs-authored commits. A fix likely belongs upstream at sst/opencode.
• This changes trace semantics and will need a reviewer conversation; it is not a drop-in like the delta guard.
• Reproduction tooling: a corpus reassembly/inspection script exists that syncs a session prefix, rebuilds the span tree, reports completeness (distinguishing "one missing parent = open root" from "many missing parents = real span loss"), and synthesizes a root so the trace can be viewed today.

[jack-champagne]

Related: #122 (per-delta read amplification). Fixing #122 alone takes this trace from 129,839 to ~35,500 spans but leaves it a single unbounded trace covering 4 sessions — the two are independent. Platform-side: harmoniqs/aws-infra#235, harmoniqs/aws-infra#236.