feat: [AI-7401] stamp launch_surface on serve telemetry events#970
feat: [AI-7401] stamp launch_surface on serve telemetry events#970ralphstodomingo wants to merge 2 commits into
Conversation
Read `ALTIMATE_LAUNCH_SURFACE` from the environment once at telemetry init and attach it as a `launch_surface` property on every App Insights event. The IDE extension sets this to `ide` when it spawns `altimate serve`; direct CLI/TUI invocations leave it unset, so the property is omitted. This lets analytics separate IDE-originated sessions (and their existing session_start / session_end / agent_outcome events) from terminal usage without any new per-session plumbing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aDZXEPBPcBAYNNcrKZomN
📝 WalkthroughWalkthroughTelemetry events now include a ChangesLaunch Surface Telemetry
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
E2E verified — Docker code-server (full chain)Built this branch ( The running serve binary is this branch and embeds the new property (stock 0.8.10 does not): The extension hands it → this build reads that env at telemetry init and stamps Unit coverage (this PR): full telemetry suite 132 pass / 0 fail, including present-when-set + absent-when-unset, asserted against the real outbound envelope JSON built by
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
There was a problem hiding this comment.
Pull request overview
Adds a launch_surface custom dimension to App Insights telemetry emitted by altimate serve, enabling analytics to distinguish IDE-launched sessions (via ALTIMATE_LAUNCH_SURFACE=ide) from CLI/TUI sessions (unset ⇒ omitted).
Changes:
- Read
ALTIMATE_LAUNCH_SURFACEduring telemetry initialization and store it in a module-levellaunchSurface. - Stamp
launch_surfaceonto every emitted App Insights envelope when set. - Add tests for present-when-set vs omitted-when-unset behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/opencode/src/altimate/telemetry/index.ts | Captures ALTIMATE_LAUNCH_SURFACE at init and injects launch_surface into telemetry properties; resets on shutdown. |
| packages/opencode/test/telemetry/telemetry.test.ts | Adds regression tests verifying launch_surface is included only when the env var is set. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Set by the IDE extension when it spawns `altimate serve`; absent for | ||
| // direct CLI/TUI invocations. | ||
| launchSurface = process.env.ALTIMATE_LAUNCH_SURFACE ?? "" |
| Telemetry.track({ type: "session_start", timestamp: 1700000000000, session_id: "sess-ide" }) | ||
| await Telemetry.flush() | ||
|
|
||
| const envelopes = JSON.parse(fetchCalls[0].body) | ||
| expect(envelopes[0].data.baseData.properties.launch_surface).toBe("ide") |
| Telemetry.track({ type: "session_start", timestamp: 1700000000000, session_id: "sess-cli" }) | ||
| await Telemetry.flush() | ||
|
|
||
| const envelopes = JSON.parse(fetchCalls[0].body) | ||
| expect(envelopes[0].data.baseData.properties.launch_surface).toBeUndefined() |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/opencode/test/telemetry/telemetry.test.ts`:
- Line 604: The two Telemetry.track calls for session_start in telemetry.test.ts
are missing required fields for the session_start event. Update both calls at
the relevant test cases to include model_id, provider_id, agent, project_id, os,
arch, and node_version alongside type, timestamp, and session_id, using the
existing test fixtures or mock values already used for
Telemetry.Event["session_start"].
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 728ab0e8-7595-4efd-87c4-9fc1d559294e
📒 Files selected for processing (2)
packages/opencode/src/altimate/telemetry/index.tspackages/opencode/test/telemetry/telemetry.test.ts
| process.env.ALTIMATE_LAUNCH_SURFACE = "ide" | ||
| const { fetchCalls, cleanup } = await initWithMockedFetch() | ||
| try { | ||
| Telemetry.track({ type: "session_start", timestamp: 1700000000000, session_id: "sess-ide" }) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== telemetry test context =="
sed -n '520,650p' packages/opencode/test/telemetry/telemetry.test.ts | cat -n
echo
echo "== Event type / Telemetry.track declaration search =="
rg -n "type Event|interface Event|Telemetry\.track|track\(" packages/opencode -g '!**/dist/**' -g '!**/build/**'
echo
echo "== likely telemetry source files =="
fd -a "telemetry" packages/opencode/src packages/opencode/test packages/opencode -t fRepository: AltimateAI/altimate-code
Length of output: 29640
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== altimate telemetry type definition =="
sed -n '1,220p' packages/opencode/src/altimate/telemetry/index.ts | cat -n
echo
echo "== track implementation area =="
sed -n '1360,1405p' packages/opencode/src/altimate/telemetry/index.ts | cat -n
echo
echo "== any event construction helpers around session_start =="
rg -n '"session_start"|session_start' packages/opencode/src/altimate/telemetry/index.ts packages/opencode/test/telemetry/telemetry.test.ts packages/opencode/test/altimate/telemetry-signals.test.tsRepository: AltimateAI/altimate-code
Length of output: 14026
Add the required session_start fields to both new telemetry calls
Telemetry.Event["session_start"] requires model_id, provider_id, agent, project_id, os, arch, and node_version; the calls at packages/opencode/test/telemetry/telemetry.test.ts:604 and :621 only pass type, timestamp, and session_id.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/opencode/test/telemetry/telemetry.test.ts` at line 604, The two
Telemetry.track calls for session_start in telemetry.test.ts are missing
required fields for the session_start event. Update both calls at the relevant
test cases to include model_id, provider_id, agent, project_id, os, arch, and
node_version alongside type, timestamp, and session_id, using the existing test
fixtures or mock values already used for Telemetry.Event["session_start"].
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 2 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Normalize casing/whitespace and restrict ALTIMATE_LAUNCH_SURFACE to a known set (ide/cli/tui) so a stray value can't inflate the dimension's cardinality. Add tests for normalization and unknown-value drop, plus length assertions before indexing envelopes.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/opencode/test/telemetry/telemetry.test.ts (1)
637-673: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider extracting the repeated env-var setup/restore boilerplate.
Each of the four tests duplicates the same
origSurfacecapture/restore andinitWithMockedFetch/flush/assert pattern. A small helper (e.g.,withLaunchSurface(value, fn)) would reduce duplication across these tests.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/opencode/test/telemetry/telemetry.test.ts` around lines 637 - 673, The telemetry tests repeat the same environment-variable save/restore and mocked fetch setup/teardown logic across the launch_surface cases. Extract that boilerplate into a small helper around Telemetry.track, Telemetry.flush, and initWithMockedFetch, using the existing launch_surface test block to centralize the origSurface handling and cleanup. Keep the helper focused on setting ALTIMATE_LAUNCH_SURFACE, running the test body, then restoring the original value so the individual tests only assert the expected launch_surface outcome.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/opencode/test/telemetry/telemetry.test.ts`:
- Around line 637-673: The telemetry tests repeat the same environment-variable
save/restore and mocked fetch setup/teardown logic across the launch_surface
cases. Extract that boilerplate into a small helper around Telemetry.track,
Telemetry.flush, and initWithMockedFetch, using the existing launch_surface test
block to centralize the origSurface handling and cleanup. Keep the helper
focused on setting ALTIMATE_LAUNCH_SURFACE, running the test body, then
restoring the original value so the individual tests only assert the expected
launch_surface outcome.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: fabd3c77-0bd0-4dd9-a726-5eef7d811877
📒 Files selected for processing (2)
packages/opencode/src/altimate/telemetry/index.tspackages/opencode/test/telemetry/telemetry.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/opencode/src/altimate/telemetry/index.ts
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/telemetry/index.ts">
<violation number="1" location="packages/opencode/src/altimate/telemetry/index.ts:1331">
P3: `KNOWN_LAUNCH_SURFACES` includes `"cli"` and `"tui"` as accepted values, but no code path ever sets them — the IDE extension only ever sets `"ide"` and direct CLI/TUI leaves the env var unset. These are effectively dead allowlist entries that could inflate perceived surface support. Either remove them or add a comment reserving them for future use. If they remain, consider that setting `ALTIMATE_LAUNCH_SURFACE=cli` manually would pass through, so there is no harm — but dead entries risk misleading future maintainers.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| // direct CLI/TUI invocations. Normalize casing/whitespace and restrict to | ||
| // a known set so a stray env value can't inflate the launch_surface | ||
| // dimension's cardinality. New surfaces must be added to KNOWN_LAUNCH_SURFACES. | ||
| const KNOWN_LAUNCH_SURFACES = new Set(["ide", "cli", "tui"]) |
There was a problem hiding this comment.
P3: KNOWN_LAUNCH_SURFACES includes "cli" and "tui" as accepted values, but no code path ever sets them — the IDE extension only ever sets "ide" and direct CLI/TUI leaves the env var unset. These are effectively dead allowlist entries that could inflate perceived surface support. Either remove them or add a comment reserving them for future use. If they remain, consider that setting ALTIMATE_LAUNCH_SURFACE=cli manually would pass through, so there is no harm — but dead entries risk misleading future maintainers.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/altimate/telemetry/index.ts, line 1331:
<comment>`KNOWN_LAUNCH_SURFACES` includes `"cli"` and `"tui"` as accepted values, but no code path ever sets them — the IDE extension only ever sets `"ide"` and direct CLI/TUI leaves the env var unset. These are effectively dead allowlist entries that could inflate perceived surface support. Either remove them or add a comment reserving them for future use. If they remain, consider that setting `ALTIMATE_LAUNCH_SURFACE=cli` manually would pass through, so there is no harm — but dead entries risk misleading future maintainers.</comment>
<file context>
@@ -1325,8 +1325,12 @@ export namespace Telemetry {
+ // direct CLI/TUI invocations. Normalize casing/whitespace and restrict to
+ // a known set so a stray env value can't inflate the launch_surface
+ // dimension's cardinality. New surfaces must be added to KNOWN_LAUNCH_SURFACES.
+ const KNOWN_LAUNCH_SURFACES = new Set(["ide", "cli", "tui"])
+ const rawLaunchSurface = (process.env.ALTIMATE_LAUNCH_SURFACE ?? "").trim().toLowerCase()
+ launchSurface = KNOWN_LAUNCH_SURFACES.has(rawLaunchSurface) ? rawLaunchSurface : ""
</file context>
Code Review SummaryStatus: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Resolved / Verified (click to expand)
Files Reviewed (2 files)
Fix these issues in Kilo Cloud Reviewed by glm-5.2 · Input: 66K · Output: 19.5K · Cached: 735.2K |
🤖 Code Review — OpenCodeReview (Gemini) — No Issues FoundNo comments generated. Looks good to me. |
Goal
Tie IDE sessions to Altimate Code telemetry (AI-7401). The serve harness already emits
session_start,session_end, andagent_outcome(completed/aborted/abandoned/error) plus returning-user identity (machine_id,first_launch). The one missing dimension is launch surface — there is no way to tell an IDE-webview session from a CLI/TUI one. This adds it.Change
packages/opencode/src/altimate/telemetry/index.ts:ALTIMATE_LAUNCH_SURFACEonce at telemetry init into a module-levellaunchSurface.launch_surfaceproperty on every App Insights envelope, alongside the existingmachine_id(omitted when the env var is unset).shutdown().The IDE extension sets
ALTIMATE_LAUNCH_SURFACE=idewhen it spawnsaltimate serve(see paired PRAltimateAI/vscode-altimate-mcp-server#<EXT_PR>). Direct CLI/TUI invocations leave it unset, solaunch_surfaceis absent — queryinglaunch_surface == "ide"selects IDE sessions; everything else is terminal.No new event types and no per-session plumbing: the existing lifecycle events simply gain a filterable dimension.
Tests
Two new cases in
test/telemetry/telemetry.test.ts(present-when-set, omitted-when-unset). Full file: 132 pass / 0 fail.🤖 Generated with Claude Code
https://claude.ai/code/session_013aDZXEPBPcBAYNNcrKZomN
Summary by cubic
Add a
launch_surfaceproperty to all serve telemetry events to distinguish IDE vs terminal sessions (AI-7401). ReadALTIMATE_LAUNCH_SURFACEat init, trim/lowercase, allow onlyide/cli/tui, and omit the field when unset or unrecognized.Written for commit 58adf31. Summary will update on new commits.
Summary by CodeRabbit
New Features
Bug Fixes