Skip to content

[Phase 1] Track auto-upgrade metrics in Monorail#7001

Draft
alfonso-noriega wants to merge 1 commit intomainfrom
03-12-observe-auto-upgrade-metrics
Draft

[Phase 1] Track auto-upgrade metrics in Monorail#7001
alfonso-noriega wants to merge 1 commit intomainfrom
03-12-observe-auto-upgrade-metrics

Conversation

@alfonso-noriega
Copy link
Copy Markdown
Contributor

Summary

  • Adds Monorail tracking for auto-upgrade events (resolves shop/issues-develop#22365)
  • Stacks on [Phase 1] Port auto-upgrade POC to main #6999 (Phase 1: auto-upgrade system)
  • Tracked metrics:
    • cmd_all_auto_upgrade_triggered: whether auto-upgrade was attempted
    • cmd_all_auto_upgrade_package_manager: which package manager was used (npm, yarn, pnpm, brew, etc.)
    • cmd_all_auto_upgrade_skipped_reason: why upgrade was skipped (major_version)
    • cmd_all_auto_upgrade_success: whether the upgrade succeeded or failed

Changes

  • packages/cli-kit/src/public/node/monorail.ts: Added 4 new fields to the public section of Schemas[MONORAIL_COMMAND_TOPIC]
  • packages/cli-kit/src/public/node/hooks/postrun.ts: Updated autoUpgradeIfNeeded() to call addPublicMetadata at each decision point (major bump skip, trigger with package manager, success/failure)
  • packages/cli-kit/src/public/node/hooks/postrun.test.ts: Added 4 new test cases covering metrics recording scenarios

Test plan

  • pnpm vitest run src/public/node/hooks/postrun.test.ts — all 8 tests pass
  • Verify Monorail schema changes are compatible with the pipeline

🤖 Generated with Claude Code

@alfonso-noriega alfonso-noriega changed the base branch from 03-12-port_auto_upgrade_poc_to_main to graphite-base/7001 April 9, 2026 12:15
@alfonso-noriega alfonso-noriega force-pushed the 03-12-observe-auto-upgrade-metrics branch from 2869478 to e33833f Compare April 9, 2026 12:15
@alfonso-noriega alfonso-noriega changed the base branch from graphite-base/7001 to main April 9, 2026 12:15
Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@alfonso-noriega alfonso-noriega force-pushed the 03-12-observe-auto-upgrade-metrics branch from 00ff291 to 5bff8e0 Compare April 9, 2026 16:38
- Adds Monorail tracking for auto-upgrade events
- Tracked metrics: triggered, package_manager, skipped_reason, success
- Updated postrun hook to record metrics at each decision point
- Added tests for all metric recording scenarios

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alfonso-noriega alfonso-noriega force-pushed the 03-12-observe-auto-upgrade-metrics branch from 5bff8e0 to bf2d5b2 Compare April 10, 2026 16:41
@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/analytics.d.ts
@@ -23,6 +23,7 @@ interface EnvironmentData {
     env_auth_method: string;
     env_is_wsl: boolean;
     env_build_repository: string;
+    env_auto_upgrade_enabled: boolean | null;
 }
 export declare function getEnvironmentData(config: Interfaces.Config): Promise<EnvironmentData>;
 export declare function getSensitiveEnvironmentData(config: Interfaces.Config): Promise<{
packages/cli-kit/dist/public/node/metadata.d.ts
@@ -34,7 +34,7 @@ export type SensitiveSchema<T> = T extends RuntimeMetadataManager<infer _TPublic
  * @returns A container for the metadata.
  */
 export declare function createRuntimeMetadataContainer<TPublic extends AnyJson, TSensitive extends AnyJson = Record<string, never>>(defaultPublicMetadata?: Partial<TPublic>): RuntimeMetadataManager<TPublic, TSensitive>;
-type CmdFieldsFromMonorail = PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'>;
+type CmdFieldsFromMonorail = PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'> & PickByPrefix<MonorailEventPublic, 'env_auto_upgrade_'>;
 declare const coreData: RuntimeMetadataManager<CmdFieldsFromMonorail, {
     commandStartOptions: {
         startTime: number;
packages/cli-kit/dist/public/node/monorail.d.ts
@@ -46,6 +46,12 @@ export interface Schemas {
             cmd_all_timing_network_ms?: Optional<number>;
             cmd_all_timing_prompts_ms?: Optional<number>;
             cmd_all_timing_active_ms?: Optional<number>;
+            env_auto_upgrade_enabled?: Optional<boolean>;
+            env_auto_upgrade_accepted?: Optional<boolean>;
+            env_auto_upgrade_triggered?: Optional<boolean>;
+            env_auto_upgrade_package_manager?: Optional<string>;
+            env_auto_upgrade_skipped_reason?: Optional<string>;
+            env_auto_upgrade_success?: Optional<boolean>;
             cmd_extensions_binary_from_source?: Optional<boolean>;
             cmd_scaffold_required_auth?: Optional<boolean>;
             cmd_scaffold_template_custom?: Optional<boolean>;
packages/cli-kit/dist/public/node/upgrade.d.ts
@@ -2,7 +2,7 @@
  * Utility function for generating an install command for the user to run
  * to install an updated version of Shopify CLI.
  *
- * @returns A string with the command to run.
+ * @returns A string with the command to run, or undefined if the package manager cannot be determined.
  */
 export declare function cliInstallCommand(): string | undefined;
 /**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant