From 59525caf31a8387d2868d0c8fff0597600ee8aac Mon Sep 17 00:00:00 2001 From: Spiros Martzoukos Date: Wed, 15 Jul 2026 18:25:07 +0300 Subject: [PATCH 1/4] fix(cli): confirm command echoes only the flags the user typed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Agent-mode write commands print a `confirmCommand` for the user to approve. `buildConfirmCommand` serialized every entry of oclif's parsed flags, which includes everything filled in from `default:`, so a bare `npx checkly deploy` confirmed as: checkly deploy --no-preview --no-verbose --schedule-on-deploy --no-preserve-resources --no-cancel-in-progress-deployment --verify-runtime-dependencies --no-debug-bundle --debug-bundle-output-file="./debug-bundle.json" --force That command does not run. `--no-x` is only a registered flag when the definition sets `allowNo: true`, so oclif rejects the above with "Nonexistent flags: --no-preview, ...". It also leaked two hidden debug flags and buried whichever flag the user had actually typed. Pass oclif's `metadata.flags` through the preview and skip flags marked `setFromDefault`. A bare deploy now confirms as `checkly deploy --force`. The helper was written for `incidents create`, where every flag is user-typed and string-valued, and was later reused for `deploy`, whose flags are mostly booleans with defaults. `deploy` and `destroy` were the two commands producing unrunnable output; `incidents create/update/resolve` were only noisy. Docs: the confirmation protocol lived only in the incidents reference, so an agent running a deploy never read it. Document deploy's two-stage confirm, `--preserve-resources`, and the delete guard in the configure reference. Also resolve the "never auto-append --force" contradiction — the confirmCommand already ends in --force and is meant to run verbatim. Co-Authored-By: Claude Opus 4.8 --- .../src/ai-context/references/communicate.md | 4 +- .../src/ai-context/references/configure.md | 18 +++++++ packages/cli/src/ai-context/skill.md | 6 ++- .../__tests__/confirm-flow-deploy.spec.ts | 47 +++++++++++++++++++ .../__tests__/confirm-flow-destroy.spec.ts | 34 +++++++++++++- packages/cli/src/commands/deploy.ts | 4 +- packages/cli/src/commands/destroy.ts | 3 +- .../incidents/__tests__/confirm-flow.spec.ts | 4 +- .../__tests__/notify-subscribers.spec.ts | 7 ++- packages/cli/src/commands/incidents/create.ts | 3 +- .../cli/src/commands/incidents/resolve.ts | 3 +- packages/cli/src/commands/incidents/update.ts | 3 +- .../helpers/__tests__/command-preview.spec.ts | 18 +++++++ packages/cli/src/helpers/command-preview.ts | 13 ++++- skills/checkly/SKILL.md | 6 ++- 15 files changed, 156 insertions(+), 17 deletions(-) diff --git a/packages/cli/src/ai-context/references/communicate.md b/packages/cli/src/ai-context/references/communicate.md index 745cd505b..1c19ba68b 100644 --- a/packages/cli/src/ai-context/references/communicate.md +++ b/packages/cli/src/ai-context/references/communicate.md @@ -20,11 +20,13 @@ Write commands (`incidents create`, `incidents update`, `incidents resolve`, `de **Rules for agents:** 1. When exit code is 2 and output contains `"status": "confirmation_required"`, **always present the `changes` array to the user** and ask for confirmation. -2. **Never auto-append `--force`** — only run the `confirmCommand` after the user explicitly approves. +2. **Run the `confirmCommand` verbatim, and only after the user explicitly approves.** It is the command you just ran, plus `--force` to skip the second prompt. Don't append `--force` to anything yourself, and don't edit the `confirmCommand` — changing its flags changes what you were authorized to do. 3. This applies to **every** write command, not just the first one. Incident updates and resolutions also require confirmation. 4. Use `--dry-run` to preview what a command will do without executing or prompting. 5. Read-only commands (`incidents list`, `status-pages list`) execute immediately without confirmation. +The `confirmCommand` echoes back only the flags that were actually passed — flags left at their default are omitted, so a bare `npx checkly deploy` confirms as `checkly deploy --force`. Treat every flag you see there as deliberate. + ## Available Commands Parse and read further reference documentation when tasked with any of the following: diff --git a/packages/cli/src/ai-context/references/configure.md b/packages/cli/src/ai-context/references/configure.md index a45c2e1b4..624c2b840 100644 --- a/packages/cli/src/ai-context/references/configure.md +++ b/packages/cli/src/ai-context/references/configure.md @@ -75,3 +75,21 @@ Run `npx checkly skills manage plan` for the full reference. ## Deploying - Deploy checks using the `npx checkly deploy` command. Use `--output` to see the created, updated, and deleted resources. Use `--verbose` to also include each resource's name and physical ID (UUID), which is useful for programmatically referencing deployed resources (e.g. `npx checkly checks get `). +- Use `--preview` to see what a deploy would change without applying it. + +### Deleted resources + +A deploy makes the account match the code. **Any resource that was deployed before and is no longer in the code gets deleted, along with its run history.** Pass `--preserve-resources` to keep those resources and their history in the Checkly account instead, where the user can manage them from the web app. + +This matters when the local project isn't the whole picture — a partial checkout, or a project whose checks were also edited elsewhere. If you're not sure the code is the complete source of truth, say so before deploying. + +### Confirmation + +`deploy` is a write command: without `--force` it returns exit code 2 and a `confirmation_required` envelope. Present its `changes` to the user and run the `confirmCommand` verbatim only after they approve. + +Deploy confirms **twice**, and the second one is the one that matters: + +1. Up front, before the project is parsed — describes the deploy, and whether checks will be scheduled and resources preserved. +2. After parsing, **only if resources would actually be deleted** — lists each one by name. This is the point where run history is at stake, so show the user that list rather than a summary of it. + +Run `npx checkly skills communicate` for the full protocol. diff --git a/packages/cli/src/ai-context/skill.md b/packages/cli/src/ai-context/skill.md index 9f28eac82..c24f11267 100644 --- a/packages/cli/src/ai-context/skill.md +++ b/packages/cli/src/ai-context/skill.md @@ -34,9 +34,11 @@ Run `npx checkly skills manage` for the full reference. ## Confirmation Protocol -Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** Never auto-append `--force`. This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. +Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -Run `npx checkly skills communicate` for the full protocol details. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. + +Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms. ## API Pass-Through (fallback for any endpoint) diff --git a/packages/cli/src/commands/__tests__/confirm-flow-deploy.spec.ts b/packages/cli/src/commands/__tests__/confirm-flow-deploy.spec.ts index 9e85e5e1b..ce37f1c1f 100644 --- a/packages/cli/src/commands/__tests__/confirm-flow-deploy.spec.ts +++ b/packages/cli/src/commands/__tests__/confirm-flow-deploy.spec.ts @@ -1,3 +1,4 @@ +import { Parser } from '@oclif/core' import { beforeEach, describe, expect, it, vi } from 'vitest' vi.mock('../../helpers/cli-mode', () => ({ @@ -39,11 +40,23 @@ vi.mock('prompts', () => ({ })) import { detectCliMode } from '../../helpers/cli-mode.js' +import { buildConfirmCommand } from '../../helpers/command-preview.js' import * as api from '../../rest/api.js' import { parseProject } from '../../services/project-parser.js' import { AuthCommand } from '../authCommand.js' import Deploy from '../deploy.js' +/** Turn a generated confirmCommand back into argv, so oclif can re-parse it. */ +function flagArgv (confirmCommand: string): string[] { + return [...confirmCommand.matchAll(/--[a-zA-Z0-9-]+(?:="[^"]*")?/g)] + .map(([token]) => token.replace(/="(.*)"$/, '=$1')) +} + +async function confirmCommandFor (argv: string[]): Promise { + const { flags, metadata } = await Parser.parse(argv, { flags: Deploy.flags, strict: true }) + return buildConfirmCommand('deploy', flags, undefined, metadata.flags) +} + function createConfirmContext () { const logged: string[] = [] let exitCodeValue: number | undefined @@ -156,6 +169,17 @@ describe('deploy confirmation flow', () => { 'debug-bundle': false, 'debug-bundle-output-file': './debug-bundle.json', }, + metadata: { + flags: { + 'preview': { setFromDefault: true }, + 'output': { setFromDefault: true }, + 'verbose': { setFromDefault: true }, + 'schedule-on-deploy': { setFromDefault: true }, + 'verify-runtime-dependencies': { setFromDefault: true }, + 'debug-bundle': { setFromDefault: true }, + 'debug-bundle-output-file': { setFromDefault: true }, + }, + }, }) await expect( @@ -172,5 +196,28 @@ describe('deploy confirmation flow', () => { expect(output.command).toBe('deploy') expect(output.classification.idempotent).toBe(true) expect(output.confirmCommand).toContain('--force') + expect(output.confirmCommand).not.toContain('--no-preview') + }) +}) + +describe('deploy confirmCommand', () => { + it('echoes only the flags the user typed', async () => { + expect(await confirmCommandFor([])).toBe('checkly deploy --force') + expect(await confirmCommandFor(['--preserve-resources'])).toBe('checkly deploy --preserve-resources --force') + }) + + it('keeps an explicit --no- for flags that allow it', async () => { + expect(await confirmCommandFor(['--no-schedule-on-deploy'])) + .toBe('checkly deploy --no-schedule-on-deploy --force') + }) + + it('generates a command oclif can parse back', async () => { + for (const argv of [[], ['--preserve-resources'], ['--no-schedule-on-deploy'], ['--verbose']]) { + const confirmCommand = await confirmCommandFor(argv) + await expect( + Parser.parse(flagArgv(confirmCommand), { flags: Deploy.flags, strict: true }), + `confirmCommand for "checkly deploy ${argv.join(' ')}" must be runnable: ${confirmCommand}`, + ).resolves.toBeDefined() + } }) }) diff --git a/packages/cli/src/commands/__tests__/confirm-flow-destroy.spec.ts b/packages/cli/src/commands/__tests__/confirm-flow-destroy.spec.ts index c4f65c381..bb41fce53 100644 --- a/packages/cli/src/commands/__tests__/confirm-flow-destroy.spec.ts +++ b/packages/cli/src/commands/__tests__/confirm-flow-destroy.spec.ts @@ -1,3 +1,4 @@ +import { Parser } from '@oclif/core' import { beforeEach, describe, expect, it, vi } from 'vitest' vi.mock('../../helpers/cli-mode', () => ({ @@ -29,14 +30,26 @@ vi.mock('prompts', () => ({ import { detectCliMode } from '../../helpers/cli-mode.js' import prompts from 'prompts' import * as api from '../../rest/api.js' +import { buildConfirmCommand } from '../../helpers/command-preview.js' import { AuthCommand } from '../authCommand.js' import Destroy from '../destroy.js' -function createCommandContext (parsed: unknown) { +/** Turn a generated confirmCommand back into argv, so oclif can re-parse it. */ +function flagArgv (confirmCommand: string): string[] { + return [...confirmCommand.matchAll(/--[a-zA-Z0-9-]+(?:="[^"]*")?/g)] + .map(([token]) => token.replace(/="(.*)"$/, '=$1')) +} + +async function confirmCommandFor (argv: string[]): Promise { + const { flags, metadata } = await Parser.parse(argv, { flags: Destroy.flags, strict: true }) + return buildConfirmCommand('destroy', flags, undefined, metadata.flags) +} + +function createCommandContext (parsed: { flags: Record, metadata?: unknown }) { const logged: string[] = [] let exitCodeValue: number | undefined return { - parse: vi.fn().mockResolvedValue(parsed), + parse: vi.fn().mockResolvedValue({ metadata: { flags: {} }, ...parsed }), log: vi.fn((msg?: string) => { if (msg) logged.push(msg) }), @@ -172,3 +185,20 @@ describe('destroy confirmation flow', () => { expect(Destroy.idempotent).toBe(false) }) }) + +describe('destroy confirmCommand', () => { + it('echoes only the flags the user typed', async () => { + expect(await confirmCommandFor([])).toBe('checkly destroy --force') + expect(await confirmCommandFor(['--preserve-resources'])).toBe('checkly destroy --preserve-resources --force') + }) + + it('generates a command oclif can parse back', async () => { + for (const argv of [[], ['--preserve-resources'], ['--cancel-in-progress-deployment']]) { + const confirmCommand = await confirmCommandFor(argv) + await expect( + Parser.parse(flagArgv(confirmCommand), { flags: Destroy.flags, strict: true }), + `confirmCommand for "checkly destroy ${argv.join(' ')}" must be runnable: ${confirmCommand}`, + ).resolves.toBeDefined() + } + }) +}) diff --git a/packages/cli/src/commands/deploy.ts b/packages/cli/src/commands/deploy.ts index c09554cbe..8bd258395 100644 --- a/packages/cli/src/commands/deploy.ts +++ b/packages/cli/src/commands/deploy.ts @@ -110,7 +110,7 @@ export default class Deploy extends AuthCommand { } async run (): Promise { - const { flags } = await this.parse(Deploy) + const { flags, metadata } = await this.parse(Deploy) const { force, preview, @@ -146,6 +146,7 @@ export default class Deploy extends AuthCommand { : 'Delete any resources removed from code, losing their run history. Pass --preserve-resources to keep them in your Checkly account instead', ], flags, + flagMetadata: metadata.flags, classification: { readOnly: Deploy.readOnly, destructive: Deploy.destructive, @@ -316,6 +317,7 @@ export default class Deploy extends AuthCommand { `Delete ${PRETTY_RESOURCE_TYPES[resourceType] ?? resourceType}: ${logicalId}`), ], flags, + flagMetadata: metadata.flags, classification: { readOnly: Deploy.readOnly, destructive: Deploy.destructive, diff --git a/packages/cli/src/commands/destroy.ts b/packages/cli/src/commands/destroy.ts index 3054c8581..4ec85eaaf 100644 --- a/packages/cli/src/commands/destroy.ts +++ b/packages/cli/src/commands/destroy.ts @@ -31,7 +31,7 @@ export default class Destroy extends AuthCommand { } async run (): Promise { - const { flags } = await this.parse(Destroy) + const { flags, metadata } = await this.parse(Destroy) const { config: configFilename, 'preserve-resources': preserveResources, @@ -52,6 +52,7 @@ export default class Destroy extends AuthCommand { : `PERMANENTLY delete ALL resources associated with the project "${checklyConfig.projectName}" in account "${account.name}"`, ], flags, + flagMetadata: metadata.flags, classification: { readOnly: Destroy.readOnly, destructive: Destroy.destructive, diff --git a/packages/cli/src/commands/incidents/__tests__/confirm-flow.spec.ts b/packages/cli/src/commands/incidents/__tests__/confirm-flow.spec.ts index 0dad4ee1c..193e3f9c7 100644 --- a/packages/cli/src/commands/incidents/__tests__/confirm-flow.spec.ts +++ b/packages/cli/src/commands/incidents/__tests__/confirm-flow.spec.ts @@ -50,12 +50,12 @@ const incidentFixture = { services: [{ id: 'svc-1', name: 'Checkout API', accountId: 'acc-1' }], } -function createCommandContext (parsed: unknown) { +function createCommandContext (parsed: { flags: Record, args?: unknown, metadata?: unknown }) { const logged: string[] = [] let didExit = false let exitCodeValue: number | undefined return { - parse: vi.fn().mockResolvedValue(parsed), + parse: vi.fn().mockResolvedValue({ metadata: { flags: {} }, ...parsed }), log: vi.fn((msg?: string) => { if (msg) logged.push(msg) }), diff --git a/packages/cli/src/commands/incidents/__tests__/notify-subscribers.spec.ts b/packages/cli/src/commands/incidents/__tests__/notify-subscribers.spec.ts index c00d3fe67..7dcabf76a 100644 --- a/packages/cli/src/commands/incidents/__tests__/notify-subscribers.spec.ts +++ b/packages/cli/src/commands/incidents/__tests__/notify-subscribers.spec.ts @@ -43,10 +43,13 @@ const statusPageFixture: StatusPage = { updated_at: '2026-02-25T10:00:00.000Z', } -function createCommandContext (parsed: unknown, CommandClass: any = IncidentsCreate) { +function createCommandContext ( + parsed: { flags: Record, args?: unknown, metadata?: unknown }, + CommandClass: any = IncidentsCreate, +) { const logged: string[] = [] return { - parse: vi.fn().mockResolvedValue(parsed), + parse: vi.fn().mockResolvedValue({ metadata: { flags: {} }, ...parsed }), log: vi.fn((msg?: string) => { if (msg) logged.push(msg) }), diff --git a/packages/cli/src/commands/incidents/create.ts b/packages/cli/src/commands/incidents/create.ts index 88dcebf1a..490c76348 100644 --- a/packages/cli/src/commands/incidents/create.ts +++ b/packages/cli/src/commands/incidents/create.ts @@ -51,7 +51,7 @@ export default class IncidentsCreate extends AuthCommand { } async run (): Promise { - const { flags } = await this.parse(IncidentsCreate) + const { flags, metadata } = await this.parse(IncidentsCreate) this.style.outputFormat = flags.output const statusPage = await api.statusPages.get(flags['status-page-id']) @@ -73,6 +73,7 @@ export default class IncidentsCreate extends AuthCommand { : 'Subscribers will NOT be notified', ], flags, + flagMetadata: metadata.flags, classification: { readOnly: IncidentsCreate.readOnly, destructive: IncidentsCreate.destructive, diff --git a/packages/cli/src/commands/incidents/resolve.ts b/packages/cli/src/commands/incidents/resolve.ts index d76b8997c..16039842c 100644 --- a/packages/cli/src/commands/incidents/resolve.ts +++ b/packages/cli/src/commands/incidents/resolve.ts @@ -33,7 +33,7 @@ export default class IncidentsResolve extends AuthCommand { } async run (): Promise { - const { args, flags } = await this.parse(IncidentsResolve) + const { args, flags, metadata } = await this.parse(IncidentsResolve) this.style.outputFormat = flags.output const incident = await api.incidents.get(args.id) @@ -48,6 +48,7 @@ export default class IncidentsResolve extends AuthCommand { : 'Subscribers will NOT be notified', ], flags, + flagMetadata: metadata.flags, args: { id: args.id }, classification: { readOnly: IncidentsResolve.readOnly, diff --git a/packages/cli/src/commands/incidents/update.ts b/packages/cli/src/commands/incidents/update.ts index c798c7771..1c87f8c2f 100644 --- a/packages/cli/src/commands/incidents/update.ts +++ b/packages/cli/src/commands/incidents/update.ts @@ -50,7 +50,7 @@ export default class IncidentsUpdate extends AuthCommand { } async run (): Promise { - const { args, flags } = await this.parse(IncidentsUpdate) + const { args, flags, metadata } = await this.parse(IncidentsUpdate) this.style.outputFormat = flags.output const incident = await api.incidents.get(args.id) @@ -69,6 +69,7 @@ export default class IncidentsUpdate extends AuthCommand { description: 'Post progress update to incident', changes, flags, + flagMetadata: metadata.flags, args: { id: args.id }, classification: { readOnly: IncidentsUpdate.readOnly, diff --git a/packages/cli/src/helpers/__tests__/command-preview.spec.ts b/packages/cli/src/helpers/__tests__/command-preview.spec.ts index 93ea1b707..32a1dffb4 100644 --- a/packages/cli/src/helpers/__tests__/command-preview.spec.ts +++ b/packages/cli/src/helpers/__tests__/command-preview.spec.ts @@ -124,4 +124,22 @@ describe('buildConfirmCommand', () => { const result = buildConfirmCommand('incidents update', { message: 'Fix deployed' }, { id: 'inc-123' }) expect(result).toBe('checkly incidents update inc-123 --message="Fix deployed" --force') }) + + it('omits flags oclif filled in from their default', () => { + const result = buildConfirmCommand('deploy', { + 'preview': false, + 'schedule-on-deploy': true, + 'preserve-resources': true, + }, undefined, { + 'preview': { setFromDefault: true }, + 'schedule-on-deploy': { setFromDefault: true }, + 'preserve-resources': { setFromDefault: false }, + }) + expect(result).toBe('checkly deploy --preserve-resources --force') + }) + + it('keeps default-valued flags when no metadata is given', () => { + const result = buildConfirmCommand('deploy', { 'schedule-on-deploy': true }) + expect(result).toBe('checkly deploy --schedule-on-deploy --force') + }) }) diff --git a/packages/cli/src/helpers/command-preview.ts b/packages/cli/src/helpers/command-preview.ts index 7d59c7060..64f8368d1 100644 --- a/packages/cli/src/helpers/command-preview.ts +++ b/packages/cli/src/helpers/command-preview.ts @@ -4,11 +4,18 @@ export type CommandClassification = { idempotent: boolean } +/** + * The subset of oclif's parse metadata we rely on: which flags were filled in from + * `default:` rather than typed by the user. Pass `metadata.flags` from `this.parse()`. + */ +export type FlagMetadata = Record + export type CommandPreview = { command: string description: string changes: string[] flags: Record + flagMetadata?: FlagMetadata args?: Record classification: CommandClassification } @@ -28,6 +35,7 @@ export function buildConfirmCommand ( command: string, flags: Record, args?: Record, + flagMetadata?: FlagMetadata, ): string { const parts = ['checkly', command] @@ -40,6 +48,9 @@ export function buildConfirmCommand ( for (const [key, value] of Object.entries(flags)) { if (OMITTED_FLAGS.has(key)) continue if (value === undefined || value === null) continue + // Defaults are not part of what the user asked for, and a default `false` renders as + // `--no-x`, which only parses when the flag sets `allowNo: true`. + if (flagMetadata?.[key]?.setFromDefault) continue if (Array.isArray(value)) { for (const item of value) { @@ -66,7 +77,7 @@ export function formatPreviewForAgent ( description: preview.description, classification: preview.classification, changes: preview.changes, - confirmCommand: buildConfirmCommand(preview.command, preview.flags, preview.args), + confirmCommand: buildConfirmCommand(preview.command, preview.flags, preview.args, preview.flagMetadata), } } diff --git a/skills/checkly/SKILL.md b/skills/checkly/SKILL.md index 0c1e17744..c44432618 100644 --- a/skills/checkly/SKILL.md +++ b/skills/checkly/SKILL.md @@ -34,9 +34,11 @@ Run `npx checkly skills manage` for the full reference. ## Confirmation Protocol -Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** Never auto-append `--force`. This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. +Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -Run `npx checkly skills communicate` for the full protocol details. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. + +Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms. ## API Pass-Through (fallback for any endpoint) From a72224635f5b95293c65bfdced5299c96e1a0d2e Mon Sep 17 00:00:00 2001 From: Spiros Martzoukos Date: Wed, 15 Jul 2026 18:57:30 +0300 Subject: [PATCH 2/4] docs(cli): correct deploy's delete-guard guidance in the skill The previous commit told agents that deploy confirms twice and to present the itemised list of resources that would be deleted. That list is unreachable for an agent: the whole delete-guard block is gated on `!force` (deploy.ts:288), and `--force` is exactly what the confirmCommand carries. The guard is an interactive-only prompt. Point agents at `npx checkly deploy --preview` instead, which dry-runs and prints the would-be deletions without confirming or applying anything. Co-Authored-By: Claude Opus 4.8 --- packages/cli/src/ai-context/references/configure.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/packages/cli/src/ai-context/references/configure.md b/packages/cli/src/ai-context/references/configure.md index 624c2b840..c32c53b84 100644 --- a/packages/cli/src/ai-context/references/configure.md +++ b/packages/cli/src/ai-context/references/configure.md @@ -87,9 +87,14 @@ This matters when the local project isn't the whole picture — a partial checko `deploy` is a write command: without `--force` it returns exit code 2 and a `confirmation_required` envelope. Present its `changes` to the user and run the `confirmCommand` verbatim only after they approve. -Deploy confirms **twice**, and the second one is the one that matters: +That confirmation happens **before the project is parsed**, so it cannot tell you which resources would be deleted — its `changes` only warn that deletion is possible. There is a second, itemised guard that lists each doomed resource by name, but it is skipped by `--force`, and `--force` is exactly what the `confirmCommand` carries. **An agent following the confirmation protocol never sees that list.** It is a prompt for humans deploying by hand. -1. Up front, before the project is parsed — describes the deploy, and whether checks will be scheduled and resources preserved. -2. After parsing, **only if resources would actually be deleted** — lists each one by name. This is the point where run history is at stake, so show the user that list rather than a summary of it. +So when resources may have been removed from the code, don't rely on the confirmation to surface it — preview first: + +```bash +npx checkly deploy --preview +``` + +This is a dry run: nothing is applied and nothing is confirmed. It prints the resources that would be created, updated, and deleted (add `--verbose` for names and IDs). Show the user what would be deleted, and only then deploy. Run `npx checkly skills communicate` for the full protocol. From f181e5bc0da82110399371848f4e3d7c935b44ac Mon Sep 17 00:00:00 2001 From: Spiros Martzoukos Date: Wed, 15 Jul 2026 19:12:58 +0300 Subject: [PATCH 3/4] docs(cli): allow for a pinned target in the confirmCommand contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The confirmation docs promised that the `confirmCommand` is the command you ran plus `--force`, and that it "echoes back only the flags that were actually passed". That is true of every command on main today: the two callers that reshape their preview flags (`members delete`, `members update`) only drop empty ones, they never add. It stops being true the moment #1402 lands. `import commit` and `import cancel` resolve their plan and pin it into the preview, so a bare `checkly import commit` confirms as: checkly import commit --plan-id="abc123" --force The pin is the right call — it stops the approved run from re-resolving to a plan the user was never shown — but it makes an absolute claim in this reference false, and #1402 merges first. Widen the contract to allow a resolved target, and keep "treat every flag you see there as deliberate", which was always the operative rule and covers the pinned flag too. Worded to hold under either merge order: it describes the mechanism without pointing at the import section that arrives with #1402. Co-Authored-By: Claude Opus 4.8 --- packages/cli/src/ai-context/references/communicate.md | 4 ++-- packages/cli/src/ai-context/skill.md | 2 +- skills/checkly/SKILL.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/cli/src/ai-context/references/communicate.md b/packages/cli/src/ai-context/references/communicate.md index 1c19ba68b..8cf77ea1c 100644 --- a/packages/cli/src/ai-context/references/communicate.md +++ b/packages/cli/src/ai-context/references/communicate.md @@ -20,12 +20,12 @@ Write commands (`incidents create`, `incidents update`, `incidents resolve`, `de **Rules for agents:** 1. When exit code is 2 and output contains `"status": "confirmation_required"`, **always present the `changes` array to the user** and ask for confirmation. -2. **Run the `confirmCommand` verbatim, and only after the user explicitly approves.** It is the command you just ran, plus `--force` to skip the second prompt. Don't append `--force` to anything yourself, and don't edit the `confirmCommand` — changing its flags changes what you were authorized to do. +2. **Run the `confirmCommand` verbatim, and only after the user explicitly approves.** It is normally the command you just ran, plus `--force` to skip the second prompt. Don't append `--force` to anything yourself, and don't edit the `confirmCommand` — changing its flags changes what you were authorized to do. 3. This applies to **every** write command, not just the first one. Incident updates and resolutions also require confirmation. 4. Use `--dry-run` to preview what a command will do without executing or prompting. 5. Read-only commands (`incidents list`, `status-pages list`) execute immediately without confirmation. -The `confirmCommand` echoes back only the flags that were actually passed — flags left at their default are omitted, so a bare `npx checkly deploy` confirms as `checkly deploy --force`. Treat every flag you see there as deliberate. +The `confirmCommand` omits flags left at their default, so a bare `npx checkly deploy` confirms as `checkly deploy --force` rather than echoing back every boolean the parser filled in. It can also carry a flag you did not pass: a command that resolves a target for you pins that target into the `confirmCommand`, so the approved run cannot drift onto a different one. Treat every flag you see there as deliberate. ## Available Commands diff --git a/packages/cli/src/ai-context/skill.md b/packages/cli/src/ai-context/skill.md index c24f11267..90dcbf3eb 100644 --- a/packages/cli/src/ai-context/skill.md +++ b/packages/cli/src/ai-context/skill.md @@ -36,7 +36,7 @@ Run `npx checkly skills manage` for the full reference. Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed, already ends in `--force`, and may pin a target the CLI resolved for you. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms. diff --git a/skills/checkly/SKILL.md b/skills/checkly/SKILL.md index c44432618..1a4493fb0 100644 --- a/skills/checkly/SKILL.md +++ b/skills/checkly/SKILL.md @@ -36,7 +36,7 @@ Run `npx checkly skills manage` for the full reference. Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed, already ends in `--force`, and may pin a target the CLI resolved for you. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms. From 4ed9aca060dd45aa5d417127292b3c86d6e36c47 Mon Sep 17 00:00:00 2001 From: Spiros Martzoukos Date: Thu, 16 Jul 2026 10:28:35 +0300 Subject: [PATCH 4/4] docs(cli): drop the pinned-target claim until the behavior lands The confirmation protocol told agents that a command resolving a target pins it into the confirmCommand. No command on main does that: every call site passes raw user input, and members delete drops its email/id disambiguation before confirming. The pinning behavior arrives with import commit/cancel in #1402, which documents it there. Co-Authored-By: Claude Opus 4.8 --- packages/cli/src/ai-context/references/communicate.md | 2 +- packages/cli/src/ai-context/skill.md | 2 +- skills/checkly/SKILL.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/cli/src/ai-context/references/communicate.md b/packages/cli/src/ai-context/references/communicate.md index 8cf77ea1c..506b42448 100644 --- a/packages/cli/src/ai-context/references/communicate.md +++ b/packages/cli/src/ai-context/references/communicate.md @@ -25,7 +25,7 @@ Write commands (`incidents create`, `incidents update`, `incidents resolve`, `de 4. Use `--dry-run` to preview what a command will do without executing or prompting. 5. Read-only commands (`incidents list`, `status-pages list`) execute immediately without confirmation. -The `confirmCommand` omits flags left at their default, so a bare `npx checkly deploy` confirms as `checkly deploy --force` rather than echoing back every boolean the parser filled in. It can also carry a flag you did not pass: a command that resolves a target for you pins that target into the `confirmCommand`, so the approved run cannot drift onto a different one. Treat every flag you see there as deliberate. +The `confirmCommand` omits flags left at their default, so a bare `npx checkly deploy` confirms as `checkly deploy --force` rather than echoing back every boolean the parser filled in. Treat every flag you see there as deliberate. ## Available Commands diff --git a/packages/cli/src/ai-context/skill.md b/packages/cli/src/ai-context/skill.md index 90dcbf3eb..c24f11267 100644 --- a/packages/cli/src/ai-context/skill.md +++ b/packages/cli/src/ai-context/skill.md @@ -36,7 +36,7 @@ Run `npx checkly skills manage` for the full reference. Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed, already ends in `--force`, and may pin a target the CLI resolved for you. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms. diff --git a/skills/checkly/SKILL.md b/skills/checkly/SKILL.md index 1a4493fb0..c44432618 100644 --- a/skills/checkly/SKILL.md +++ b/skills/checkly/SKILL.md @@ -36,7 +36,7 @@ Run `npx checkly skills manage` for the full reference. Write commands (e.g. `incidents create`, `deploy`, `destroy`) return exit code 2 with a `confirmation_required` JSON envelope instead of executing. **Always present the `changes` to the user and wait for approval before running the `confirmCommand`.** This applies to every write command individually — updates and resolutions need confirmation too, not just the initial create. -The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed, already ends in `--force`, and may pin a target the CLI resolved for you. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. +The `confirmCommand` is the approved command, ready to run verbatim: it repeats the flags you passed and already ends in `--force`. Run it as-is once the user approves — don't add `--force` to a command yourself, and don't add flags the user didn't ask for. Run `npx checkly skills communicate` for the full protocol details, or `npx checkly skills configure` for what `deploy` confirms.