feat(telemetry): give every failure an error code and report where a run stopped - #441
Draft
djgould wants to merge 1 commit into
Draft
feat(telemetry): give every failure an error code and report where a run stopped#441djgould wants to merge 1 commit into
djgould wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 6e9f1d3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
…run stopped Errors were collapsing into one bucket: 19 of 87 `CliError` sites carried no code, so 36.5% of init failures reported a generic `cli_error`. Adds 14 `ERROR_CODE` entries and applies them across init, update, deploy, env, users, and switch-env, so agent-mode JSON also gets a code where it previously fell through to plain text. Adds a `stage` dimension recording how far a multi-step command got, emitted on success, error, and abort — a drop-off funnel rather than an error-only field. `clerk init` is instrumented across ten markers. Splits npm registry failures so a bad `--channel` reports a usage error instead of an unreachable registry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
djgould
force-pushed
the
devin/init-telemetry-stages
branch
from
August 22, 2026 17:39
cc1ff63 to
6e9f1d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
clerk initfails on 21.4% of runs (1,596 runs / 30 days), and the telemetry can't say why or where. This makes both legible.CliErrorsites. 19 were uncoded and collapsing into a genericcli_error— 125 of 342 init errors (36.5%). Adds 14ERROR_CODEentries across init, update, deploy, env, users, and switch-env. Agent-mode JSON gates onerror.code, so these failures now emit structured JSON where they previously fell through to plain text.stagedimension recording how far a multi-step command got, emitted on success, error, and abort — a drop-off funnel, not an error-only field.clerk initis instrumented across ten markers:flags → detect → strategy → link → bootstrap → install → scaffold → keys → skills → done, plusalready_set_upfor the clean-rerun short-circuit.clerk update --channel typoreaches npm fine but reportedregistry_unreachable, so agents retried a permanent input error. Nowusage_errorfor a missing dist-tag,update_failedfor a bad response,registry_unreachableonly for transport/timeout.Disclosure notice and README both updated — this collects a new field.
What it does not fix
The 107
unexpected_errorruns (31.3% of init errors) are plainErrorthrows that never reachCliError;stagewill say where they die but the error stays anonymous. Capturingerror.constructor.namewould decompose them — deliberately left out of this PR.Review
Carries a prior multi-model adversarial review (Claude + Codex, 19 agents, 10 findings, 0 refuted). All 8 distinct findings are applied, with one deviation: finding 7 proposed deleting the dead
users/shared.ts; this takes the leaner option the reviewer offered — dropping theNOT_IMPLEMENTEDcode so the PR doesn't mint a public error code for dead code, leaving the out-of-scope file alone.The three new stage tests were mutation-checked: each fails when its marker is removed.
Verification
typecheck,lint,format:checkclean; 2673 pass / 0 fail. E2E not run (needs 1Password-injected secrets).🤖 Generated with Claude Code