From a08cfa1aaa1de42fe7c8d6b5495e37a0ec3027ec Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Thu, 21 May 2026 20:11:51 -0700 Subject: [PATCH 1/4] feat: first-run install prompt on bare `failproofai` invocation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PostHog showed only ~10% of npm-installed users ever ran `failproofai policies --install` (470 unique installers → 48 over 90d). The no-args dashboard launch now detects "zero hooks installed across any detected CLI" and offers to run the existing interactive policy-selection inline. Non-TTY falls through to the dashboard with a short hint. - src/hooks/first-run-nudge.ts: opt-out via FAILPROOFAI_NO_FIRST_RUN=1, walks every detected CLI/scope to skip if anything is already set up, emits four PostHog events (first_run_nudge_{shown,accepted,declined, skipped_noninteractive}) so the uplift is measurable. - bin/failproofai.mjs: args.length === 0 guard before launch("start"), try/catch-wrapped so the nudge cannot block the dashboard. - scripts/postinstall.mjs: "Next steps" block for !configured && !registered (the brand-new-user case the existing printHooksWarning doesn't cover). Co-Authored-By: Claude Opus 4.7 --- CHANGELOG.md | 4 + __tests__/hooks/first-run-nudge.test.ts | 282 ++++++++++++++++++++++++ __tests__/scripts/postinstall.test.ts | 191 ++++++++++++++++ bin/failproofai.mjs | 11 + scripts/postinstall.mjs | 9 + src/hooks/first-run-nudge.ts | 146 ++++++++++++ 6 files changed, 643 insertions(+) create mode 100644 __tests__/hooks/first-run-nudge.test.ts create mode 100644 __tests__/scripts/postinstall.test.ts create mode 100644 src/hooks/first-run-nudge.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 87b2ddb..4b7e2ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,11 @@ ## 0.0.11-beta.2 — 2026-05-21 ### Features +<<<<<<< HEAD - Expand PostHog telemetry coverage to close the 16 server-side and 12 web-UI gaps surfaced by the May audit (#376). Server-side adds `cli_install_success` / `cli_install_failure` / `cli_uninstall_success` / `cli_uninstall_failure` / `cli_list_invoked` / `cli_parse_error` / `cli_unexpected_error` / `hook_dispatch_error` (CLI lifecycle outcomes in `bin/failproofai.mjs`), `hook_stdin_error` / `hook_payload_parse_error` (hook handler input errors in `src/hooks/handler.ts`), `policy_evaluation_error` (builtin policy crashes in `src/hooks/policy-evaluator.ts`, distinct from the existing `custom_hook_error`), `custom_policy_validation_failed` / `custom_hooks_load_error` / `policy_params_validation_warning` / `scope_validation_failed` / `hook_write_failed` / `multi_scope_warning_shown` / `cli_detection_summary` / `beta_policies_installed` (manager / loader / install-prompt internals), and `first_install` / `version_changed` (lifecycle detection in `scripts/postinstall.mjs` via a new `~/.failproofai/last-version` file). Web-UI adds `policies_tab_switched` / `activity_filter_changed` (debounced) / `activity_row_toggled` / `activity_copy_clicked` / `activity_pagination_changed` / `cli_selection_toggled` / `cli_install_remove_submitted` / `cli_reinstall_submitted` / `policy_config_modal_opened` / `policy_config_modal_closed` / `action_error_displayed` / `hooks_install_from_error_clicked` via `usePostHog()` in `app/policies/hooks-client.tsx`. The deny-/instruct-only condition at `handler.ts:344` (allow-path tracking) is intentionally left unchanged. All events go through the existing helpers (`trackHookEvent`, `trackInstallEvent`, `captureClientEvent`) and honor `FAILPROOFAI_TELEMETRY_DISABLED=1`. +======= +- Add a first-run install prompt on bare `failproofai` invocations. PostHog showed only ~10% of npm-installed users ever ran `failproofai policies --install`; the no-args dashboard launch now detects "zero hooks installed across any detected CLI" and offers to run the existing interactive policy-selection inline (covering all of Claude Code, Codex, Copilot, Cursor, OpenCode, Pi, Gemini). Non-TTY contexts (CI, piped invocations) print a short stderr hint and fall through to the dashboard. New `src/hooks/first-run-nudge.ts` module, a guard in `bin/failproofai.mjs` before `launch("start")`, plus four new PostHog events (`first_run_nudge_shown`, `_accepted`, `_declined`, `_skipped_noninteractive`) so the uplift is measurable. Postinstall message extended with a "Next steps" block when the brand-new-user case is detected (`!configured && !registered`). Opt-out via `FAILPROOFAI_NO_FIRST_RUN=1`. +>>>>>>> c19621a (feat: first-run install prompt on bare `failproofai` invocation) ### Breaking - Remove the undocumented cloud auth + event relay subsystem ahead of a from-scratch redesign. Deletes `src/auth/` (OAuth 2.0 device-flow login against `api.befailproof.ai`, `~/.failproofai/auth.json` token store) and `src/relay/` (WebSocket event relay daemon, sanitized JSONL queue at `~/.failproofai/cache/server-queue/`, PID tracking). Strips the `failproofai login` / `logout` / `whoami` / `relay start|stop|status` / `sync` subcommands and the internal `--relay-daemon` mode from `bin/failproofai.mjs`, along with their `--help` entries and "did you mean" suggestions. Removes the fire-and-forget `appendToServerQueue` + `ensureRelayRunning` calls from `src/hooks/handler.ts` so hook evaluation no longer enqueues events or lazy-spawns a daemon. The whole subsystem had zero references in `README.md`, `docs/`, `examples/`, or `__tests__/`, and only had internal cross-imports — `tsc`, `eslint`, `vitest` (1623 tests), and the `bun run build` bundles all stay green. Users who ran `failproofai login` should also wipe `~/.failproofai/{auth.json,cache/server-queue,relay.pid}` and stop any running relay daemon by hand; new auth/cloud surface will land in a follow-up. diff --git a/__tests__/hooks/first-run-nudge.test.ts b/__tests__/hooks/first-run-nudge.test.ts new file mode 100644 index 0000000..e5008bd --- /dev/null +++ b/__tests__/hooks/first-run-nudge.test.ts @@ -0,0 +1,282 @@ +// @vitest-environment node +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import { PassThrough } from "node:stream"; + +vi.mock("../../src/hooks/integrations", () => ({ + detectInstalledClis: vi.fn(), + getIntegration: vi.fn(), +})); + +vi.mock("../../src/hooks/manager", () => ({ + installHooks: vi.fn(async () => undefined), +})); + +vi.mock("../../src/hooks/hook-telemetry", () => ({ + trackHookEvent: vi.fn(async () => undefined), +})); + +vi.mock("../../lib/telemetry-id", () => ({ + getInstanceId: vi.fn(() => "test-distinct-id"), +})); + +function makeIntegration(displayName: string, scopes: readonly string[], installed: boolean) { + return { + id: displayName.toLowerCase(), + displayName, + scopes, + eventTypes: [], + getSettingsPath: vi.fn(), + readSettings: vi.fn(), + writeSettings: vi.fn(), + buildHookEntry: vi.fn(), + isFailproofaiHook: vi.fn(), + writeHookEntries: vi.fn(), + removeHooksFromFile: vi.fn(), + hooksInstalledInSettings: vi.fn(() => installed), + detectInstalled: vi.fn(), + }; +} + +interface IO { + stdin: PassThrough & { isTTY?: boolean }; + stdout: PassThrough & { isTTY?: boolean }; + output: string; +} + +function makeIO(isTTY: boolean): IO { + const stdin = new PassThrough() as PassThrough & { isTTY?: boolean }; + const stdout = new PassThrough() as PassThrough & { isTTY?: boolean }; + stdin.isTTY = isTTY; + stdout.isTTY = isTTY; + let output = ""; + stdout.on("data", (chunk) => { + output += chunk.toString(); + }); + const io = { stdin, stdout } as IO; + Object.defineProperty(io, "output", { get: () => output }); + return io; +} + +async function importModule() { + return await import("../../src/hooks/first-run-nudge"); +} + +async function importMocks() { + const integrations = await import("../../src/hooks/integrations"); + const manager = await import("../../src/hooks/manager"); + const telemetry = await import("../../src/hooks/hook-telemetry"); + return { integrations, manager, telemetry }; +} + +describe("hooks/first-run-nudge", () => { + let exitSpy: ReturnType; + + beforeEach(() => { + vi.clearAllMocks(); + vi.resetModules(); + delete process.env.FAILPROOFAI_NO_FIRST_RUN; + exitSpy = vi.spyOn(process, "exit").mockImplementation(((code?: number) => { + throw new Error(`exit:${code ?? 0}`); + }) as never); + }); + + afterEach(() => { + exitSpy.mockRestore(); + }); + + it("returns immediately when FAILPROOFAI_NO_FIRST_RUN=1 (no detection, no telemetry)", async () => { + process.env.FAILPROOFAI_NO_FIRST_RUN = "1"; + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + + await maybeRunFirstRunNudge(makeIO(true)); + + expect(integrations.detectInstalledClis).not.toHaveBeenCalled(); + expect(manager.installHooks).not.toHaveBeenCalled(); + expect(telemetry.trackHookEvent).not.toHaveBeenCalled(); + expect(exitSpy).not.toHaveBeenCalled(); + }); + + it("returns when no CLIs are detected", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue([]); + + await maybeRunFirstRunNudge(makeIO(true)); + + expect(manager.installHooks).not.toHaveBeenCalled(); + expect(telemetry.trackHookEvent).not.toHaveBeenCalled(); + expect(exitSpy).not.toHaveBeenCalled(); + }); + + it("returns when any detected CLI already has hooks installed in any scope", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude", "codex"] as never); + const claudeInt = makeIntegration("Claude Code", ["user", "project", "local"], false); + const codexInt = makeIntegration("Codex", ["user", "project"], true); + vi.mocked(integrations.getIntegration).mockImplementation( + (id: string) => (id === "claude" ? claudeInt : codexInt) as never, + ); + + await maybeRunFirstRunNudge(makeIO(true)); + + expect(manager.installHooks).not.toHaveBeenCalled(); + expect(telemetry.trackHookEvent).not.toHaveBeenCalled(); + }); + + it("non-TTY: prints hint and fires _skipped_noninteractive", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude"] as never); + vi.mocked(integrations.getIntegration).mockReturnValue( + makeIntegration("Claude Code", ["user"], false) as never, + ); + + const io = makeIO(false); + await maybeRunFirstRunNudge(io); + + expect(io.output).toContain("No policies are installed"); + expect(io.output).toContain("Launching dashboard"); + expect(manager.installHooks).not.toHaveBeenCalled(); + expect(telemetry.trackHookEvent).toHaveBeenCalledWith( + "test-distinct-id", + "first_run_nudge_skipped_noninteractive", + { detected_clis: ["claude"], detected_count: 1 }, + ); + }); + + it("TTY accept (Y): fires _shown then _accepted, calls installHooks with detected CLIs, exits 0", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude", "codex"] as never); + const intMap: Record> = { + claude: makeIntegration("Claude Code", ["user"], false), + codex: makeIntegration("Codex", ["user"], false), + }; + vi.mocked(integrations.getIntegration).mockImplementation( + (id: string) => intMap[id] as never, + ); + + const io = makeIO(true); + setTimeout(() => io.stdin.write("y\n"), 10); + + await expect(maybeRunFirstRunNudge(io)).rejects.toThrow("exit:0"); + + expect(io.output).toContain("Failproof AI — first-run setup"); + expect(io.output).toContain("Claude Code, Codex"); + + const calls = vi.mocked(telemetry.trackHookEvent).mock.calls; + const events = calls.map((c) => c[1]); + expect(events).toEqual(["first_run_nudge_shown", "first_run_nudge_accepted"]); + expect(calls[1][2]).toMatchObject({ + detected_clis: ["claude", "codex"], + detected_count: 2, + target_scope: "user", + source: "first-run-nudge", + }); + + expect(manager.installHooks).toHaveBeenCalledWith( + undefined, + "user", + undefined, + false, + "first-run-nudge", + undefined, + false, + ["claude", "codex"], + ); + }); + + it("TTY accept on empty Enter (default Y): runs installHooks", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude"] as never); + vi.mocked(integrations.getIntegration).mockReturnValue( + makeIntegration("Claude Code", ["user"], false) as never, + ); + + const io = makeIO(true); + setTimeout(() => io.stdin.write("\n"), 10); + + await expect(maybeRunFirstRunNudge(io)).rejects.toThrow("exit:0"); + expect(manager.installHooks).toHaveBeenCalled(); + }); + + it("TTY decline (n): fires _declined with reason user_no, does NOT call installHooks, does NOT exit", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude"] as never); + vi.mocked(integrations.getIntegration).mockReturnValue( + makeIntegration("Claude Code", ["user"], false) as never, + ); + + const io = makeIO(true); + setTimeout(() => io.stdin.write("n\n"), 10); + + await maybeRunFirstRunNudge(io); + + const events = vi.mocked(telemetry.trackHookEvent).mock.calls.map((c) => c[1]); + expect(events).toEqual(["first_run_nudge_shown", "first_run_nudge_declined"]); + const declined = vi + .mocked(telemetry.trackHookEvent) + .mock.calls.find((c) => c[1] === "first_run_nudge_declined")?.[2]; + expect(declined).toMatchObject({ reason: "user_no" }); + expect(manager.installHooks).not.toHaveBeenCalled(); + expect(exitSpy).not.toHaveBeenCalled(); + }); + + it("TTY SIGINT: fires _declined with reason sigint and exits 130", async () => { + // Mock readline so we can trigger the SIGINT handler directly — emulating + // ^C through a PassThrough is brittle across Node versions. + vi.doMock("node:readline", () => ({ + createInterface: () => { + const handlers: Record void> = {}; + return { + on: (ev: string, cb: () => void) => { + handlers[ev] = cb; + }, + question: (_q: string, _cb: () => void) => { + setImmediate(() => handlers["SIGINT"]?.()); + }, + close: () => {}, + }; + }, + })); + + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager, telemetry } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude"] as never); + vi.mocked(integrations.getIntegration).mockReturnValue( + makeIntegration("Claude Code", ["user"], false) as never, + ); + + const io = makeIO(true); + await expect(maybeRunFirstRunNudge(io)).rejects.toThrow("exit:130"); + + const declined = vi + .mocked(telemetry.trackHookEvent) + .mock.calls.find((c) => c[1] === "first_run_nudge_declined")?.[2]; + expect(declined).toMatchObject({ reason: "sigint" }); + expect(manager.installHooks).not.toHaveBeenCalled(); + vi.doUnmock("node:readline"); + }); + + it("survives a broken integration.hooksInstalledInSettings (treats it as not-installed)", async () => { + const { maybeRunFirstRunNudge } = await importModule(); + const { integrations, manager } = await importMocks(); + vi.mocked(integrations.detectInstalledClis).mockReturnValue(["claude"] as never); + const broken = makeIntegration("Claude Code", ["user"], false); + broken.hooksInstalledInSettings = vi.fn(() => { + throw new Error("boom"); + }); + vi.mocked(integrations.getIntegration).mockReturnValue(broken as never); + + const io = makeIO(true); + setTimeout(() => io.stdin.write("n\n"), 10); + + await maybeRunFirstRunNudge(io); + + expect(manager.installHooks).not.toHaveBeenCalled(); + }); +}); diff --git a/__tests__/scripts/postinstall.test.ts b/__tests__/scripts/postinstall.test.ts new file mode 100644 index 0000000..1dc77b2 --- /dev/null +++ b/__tests__/scripts/postinstall.test.ts @@ -0,0 +1,191 @@ +// @vitest-environment node +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import { resolve } from "node:path"; + +const FAKE_HOME = "/fake/home"; +const FAKE_CWD = "/fake/project"; +const HOOKS_CONFIG = resolve(FAKE_HOME, ".failproofai", "policies-config.json"); +const USER_SETTINGS = resolve(FAKE_HOME, ".claude", "settings.json"); +const SERVER_JS = resolve(FAKE_CWD, ".next", "standalone", "server.js"); + +vi.mock("node:fs", () => ({ + existsSync: vi.fn(), + readFileSync: vi.fn(), +})); + +vi.mock("node:os", () => ({ + homedir: vi.fn(() => FAKE_HOME), + platform: vi.fn(() => "linux"), + arch: vi.fn(() => "x64"), + release: vi.fn(() => "5.15.0"), + hostname: vi.fn(() => "test-host"), +})); + +vi.mock("../../scripts/install-telemetry.mjs", () => ({ + trackInstallEvent: vi.fn(() => Promise.resolve()), +})); + +vi.mock("../../scripts/install-diagnosis.mjs", () => ({ + diagnoseShadow: vi.fn(() => ({ shadowed: false })), +})); + +const CONFIG_WITH_TWO_POLICIES = JSON.stringify({ + enabledPolicies: ["block-sudo", "block-rm-rf"], +}); + +const SETTINGS_WITH_MARKED_HOOK = JSON.stringify({ + hooks: { + PreToolUse: [ + { + hooks: [ + { type: "command", command: "failproofai --hook PreToolUse", __failproofai_hook__: true }, + ], + }, + ], + }, +}); + +const SETTINGS_WITHOUT_MARKED_HOOK = JSON.stringify({ hooks: {} }); + +describe("postinstall script", () => { + let cwdSpy: ReturnType; + let exitSpy: ReturnType; + let consoleLogSpy: ReturnType; + let consoleWarnSpy: ReturnType; + + beforeEach(() => { + vi.resetModules(); + vi.clearAllMocks(); + cwdSpy = vi.spyOn(process, "cwd").mockReturnValue(FAKE_CWD); + exitSpy = vi.spyOn(process, "exit").mockImplementation((_code?: string | number | null) => { + throw new Error("process.exit called"); + }); + consoleLogSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + consoleWarnSpy = vi.spyOn(console, "warn").mockImplementation(() => {}); + // Must differ from cwd so the dev-context guard doesn't early-exit + process.env.INIT_CWD = "/some/other/dir"; + }); + + afterEach(() => { + delete process.env.INIT_CWD; + cwdSpy.mockRestore(); + exitSpy.mockRestore(); + consoleLogSpy.mockRestore(); + consoleWarnSpy.mockRestore(); + }); + + async function runPostinstall(fs: { + hooksConfigExists?: boolean; + settingsExists?: boolean; + hooksConfigContent?: string; + settingsContent?: string; + }) { + const { existsSync, readFileSync } = await import("node:fs"); + vi.mocked(existsSync).mockImplementation((p) => { + if (p === SERVER_JS) return true; + if (p === HOOKS_CONFIG) return fs.hooksConfigExists ?? false; + if (p === USER_SETTINGS) return fs.settingsExists ?? false; + return false; + }); + vi.mocked(readFileSync).mockImplementation(((p: string) => { + if (p === HOOKS_CONFIG) return fs.hooksConfigContent ?? ""; + if (p === USER_SETTINGS) return fs.settingsContent ?? ""; + // package.json read for shadow diagnosis — return a minimal stub + return JSON.stringify({ version: "0.0.0-test" }); + }) as never); + + await import("../../scripts/postinstall.mjs"); + } + + const allLogs = () => + consoleLogSpy.mock.calls.map((c: unknown[]) => String(c[0] ?? "")).join("\n"); + + describe("brand-new user (no config, no settings)", () => { + it("prints the Next steps block", async () => { + await runPostinstall({ hooksConfigExists: false }); + expect(allLogs()).toContain("Next steps"); + expect(allLogs()).toContain("failproofai policies --install"); + expect(allLogs()).toContain("FAILPROOFAI_NO_FIRST_RUN=1"); + }); + + it("does NOT print the existing hooks-not-registered warning", async () => { + await runPostinstall({ hooksConfigExists: false }); + expect(allLogs()).not.toContain("hooks config exists with enabled policies"); + }); + + it("fires package_installed with hooks_configured=false, hooks_registered=false", async () => { + const { trackInstallEvent } = await import("../../scripts/install-telemetry.mjs"); + await runPostinstall({ hooksConfigExists: false }); + expect(trackInstallEvent).toHaveBeenCalledWith( + "package_installed", + expect.objectContaining({ + hooks_configured: false, + hooks_registered: false, + enabled_policy_count: 0, + }), + ); + }); + }); + + describe("fully installed user (config + settings + registered)", () => { + it("prints nothing — no warning, no Next steps", async () => { + await runPostinstall({ + hooksConfigExists: true, + hooksConfigContent: CONFIG_WITH_TWO_POLICIES, + settingsExists: true, + settingsContent: SETTINGS_WITH_MARKED_HOOK, + }); + expect(allLogs()).not.toContain("Next steps"); + expect(allLogs()).not.toContain("hooks config exists with enabled policies"); + }); + + it("fires package_installed with hooks_configured=true, hooks_registered=true", async () => { + const { trackInstallEvent } = await import("../../scripts/install-telemetry.mjs"); + await runPostinstall({ + hooksConfigExists: true, + hooksConfigContent: CONFIG_WITH_TWO_POLICIES, + settingsExists: true, + settingsContent: SETTINGS_WITH_MARKED_HOOK, + }); + expect(trackInstallEvent).toHaveBeenCalledWith( + "package_installed", + expect.objectContaining({ + hooks_configured: true, + hooks_registered: true, + enabled_policy_count: 2, + }), + ); + }); + }); + + describe("config exists but hooks not registered", () => { + it("prints printHooksWarning, NOT the Next steps block", async () => { + await runPostinstall({ + hooksConfigExists: true, + hooksConfigContent: CONFIG_WITH_TWO_POLICIES, + settingsExists: true, + settingsContent: SETTINGS_WITHOUT_MARKED_HOOK, + }); + expect(allLogs()).toContain("hooks config exists with enabled policies"); + expect(allLogs()).not.toContain("Next steps"); + }); + + it("fires package_installed with hooks_configured=true, hooks_registered=false", async () => { + const { trackInstallEvent } = await import("../../scripts/install-telemetry.mjs"); + await runPostinstall({ + hooksConfigExists: true, + hooksConfigContent: CONFIG_WITH_TWO_POLICIES, + settingsExists: true, + settingsContent: SETTINGS_WITHOUT_MARKED_HOOK, + }); + expect(trackInstallEvent).toHaveBeenCalledWith( + "package_installed", + expect.objectContaining({ + hooks_configured: true, + hooks_registered: false, + enabled_policy_count: 2, + }), + ); + }); + }); +}); diff --git a/bin/failproofai.mjs b/bin/failproofai.mjs index abf7e21..d0b7454 100755 --- a/bin/failproofai.mjs +++ b/bin/failproofai.mjs @@ -492,6 +492,17 @@ EXAMPLES ); } + // First-run nudge — only on truly bare `failproofai` invocations. Best-effort: + // any thrown error must not block the dashboard from launching. + if (args.length === 0) { + try { + const { maybeRunFirstRunNudge } = await import("../src/hooks/first-run-nudge"); + await maybeRunFirstRunNudge(); + } catch { + // Nudge is non-critical; fall through to dashboard. + } + } + // Dashboard launch — always production mode const { launch } = await import("../scripts/launch"); launch("start"); diff --git a/scripts/postinstall.mjs b/scripts/postinstall.mjs index e61b933..23bd43d 100644 --- a/scripts/postinstall.mjs +++ b/scripts/postinstall.mjs @@ -134,6 +134,15 @@ try { // Non-critical — don't fail the install } +if (!hooksResult.configured && !hooksResult.registered) { + console.log( + `\n[failproofai] Installed. Next steps:\n` + + ` 1. Run \`failproofai policies --install\` to enable safety policies.\n` + + ` 2. Run \`failproofai\` to open the dashboard (or just \`failproofai\` to start now — it'll offer to set up policies for you).\n` + + ` Disable first-run prompt: FAILPROOFAI_NO_FIRST_RUN=1\n` + ); +} + // First-run + version_changed detection. The presence of ~/.failproofai/last-version // is a stable signal: written on every postinstall, absent before the first one. // Cannot piggy-back on instance-id because most users hit Tier 2 (OS machine ID) diff --git a/src/hooks/first-run-nudge.ts b/src/hooks/first-run-nudge.ts new file mode 100644 index 0000000..9000a52 --- /dev/null +++ b/src/hooks/first-run-nudge.ts @@ -0,0 +1,146 @@ +/** + * First-run nudge for `failproofai` (no-args invocation). + * + * Fires when a user runs the bare CLI without having installed any policies on + * any detected agent CLI. PostHog data showed only ~10% of npm-installed users + * ran `failproofai policies --install`; this prompt closes the awareness gap + * by offering to run that install inline. + * + * The hooks themselves are the sentinel — if any are installed for any + * detected CLI, we never prompt again. No separate state file needed. + * + * Honors `FAILPROOFAI_NO_FIRST_RUN=1` for opt-out, falls back to a short + * stderr hint in non-TTY contexts (CI, piped invocations). + */ +import * as readline from "node:readline"; + +import { detectInstalledClis, getIntegration } from "./integrations"; +import { installHooks } from "./manager"; +import { trackHookEvent } from "./hook-telemetry"; +import { getInstanceId } from "../../lib/telemetry-id"; +import type { IntegrationType } from "./types"; + +type TTYIn = NodeJS.ReadableStream & { isTTY?: boolean }; +type TTYOut = NodeJS.WritableStream & { isTTY?: boolean }; + +export interface FirstRunNudgeOptions { + stdin?: TTYIn; + stdout?: TTYOut; +} + +async function emit(event: string, props: Record): Promise { + try { + await trackHookEvent(getInstanceId(), event, props); + } catch { + // Telemetry must never break first-run UX. + } +} + +function anyHooksInstalled(detected: IntegrationType[]): boolean { + for (const id of detected) { + const integration = getIntegration(id); + for (const scope of integration.scopes) { + try { + if (integration.hooksInstalledInSettings(scope)) return true; + } catch { + // A broken settings file shouldn't suppress the nudge. + } + } + } + return false; +} + +function clisLabel(detected: IntegrationType[]): string { + return detected.map((id) => getIntegration(id).displayName).join(", "); +} + +async function promptYesNo( + stdin: TTYIn, + stdout: TTYOut, +): Promise<"yes" | "no" | "sigint"> { + return new Promise((resolve) => { + const rl = readline.createInterface({ input: stdin, output: stdout }); + let settled = false; + const finish = (answer: "yes" | "no" | "sigint") => { + if (settled) return; + settled = true; + rl.close(); + resolve(answer); + }; + rl.on("SIGINT", () => finish("sigint")); + rl.question("Install policies now? [Y/n] ", (raw) => { + const a = (raw ?? "").trim().toLowerCase(); + if (a === "" || a === "y" || a === "yes") finish("yes"); + else finish("no"); + }); + }); +} + +export async function maybeRunFirstRunNudge(opts: FirstRunNudgeOptions = {}): Promise { + if (process.env.FAILPROOFAI_NO_FIRST_RUN === "1") return; + + const stdin: TTYIn = opts.stdin ?? process.stdin; + const stdout: TTYOut = opts.stdout ?? process.stdout; + + let detected: IntegrationType[]; + try { + detected = detectInstalledClis(); + } catch { + return; + } + if (detected.length === 0) return; + + if (anyHooksInstalled(detected)) return; + + const detectedProps = { detected_clis: detected, detected_count: detected.length }; + + if (!stdin.isTTY || !stdout.isTTY) { + stdout.write( + `\n[failproofai] No policies are installed. Run \`failproofai policies --install\` to set them up.\n` + + `[failproofai] Launching dashboard…\n\n`, + ); + await emit("first_run_nudge_skipped_noninteractive", detectedProps); + return; + } + + stdout.write( + `\n┌─ Failproof AI — first-run setup ────────────────────────────────────\n` + + `│ Detected agent CLI(s): ${clisLabel(detected)}\n` + + `│ Policies block unsafe actions (sudo, rm -rf /, secret leaks, …)\n` + + `│ before your agent runs them. Nothing is installed yet.\n` + + `└──────────────────────────────────────────────────────────────────────\n\n` + + ` Disable this prompt anytime: FAILPROOFAI_NO_FIRST_RUN=1\n\n`, + ); + + await emit("first_run_nudge_shown", detectedProps); + + const answer = await promptYesNo(stdin, stdout); + + if (answer === "sigint") { + await emit("first_run_nudge_declined", { ...detectedProps, reason: "sigint" }); + process.exit(130); + } + + if (answer === "no") { + await emit("first_run_nudge_declined", { ...detectedProps, reason: "user_no" }); + return; + } + + await emit("first_run_nudge_accepted", { + ...detectedProps, + target_scope: "user", + source: "first-run-nudge", + }); + + await installHooks( + undefined, + "user", + undefined, + false, + "first-run-nudge", + undefined, + false, + detected, + ); + process.exit(0); +} From 59f51dfe730fcb441c517df833aca0da0316a217 Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Thu, 21 May 2026 20:13:48 -0700 Subject: [PATCH 2/4] docs: document the first-run install prompt + FAILPROOFAI_NO_FIRST_RUN MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reflect the new no-args behavior shipped in the previous commit. Quickstart in README and introduction.mdx call out that `failproofai policies --install` is now optional — running bare `failproofai` will offer to do it. Env-vars reference gets a new First-run prompt section for FAILPROOFAI_NO_FIRST_RUN. Chinese mirror (docs/zh/introduction.mdx) and the 14 translated env-vars files are intentionally left for the translation-sync PR pattern (see #371). Co-Authored-By: Claude Opus 4.7 --- README.md | 4 ++-- docs/cli/environment-variables.mdx | 6 ++++++ docs/introduction.mdx | 2 +- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e384130..89c0e0a 100644 --- a/README.md +++ b/README.md @@ -81,11 +81,11 @@ before they become incidents. Zero latency. Runs locally. ```sh npm install -g failproofai -failproofai policies --install +failproofai policies --install # or just run `failproofai` and accept the first-run prompt failproofai ``` -30 built-in policies activate immediately. Dashboard at `localhost:8020`. +30 built-in policies activate immediately. Dashboard at `localhost:8020`. Disable the first-run prompt with `FAILPROOFAI_NO_FIRST_RUN=1`. --- diff --git a/docs/cli/environment-variables.mdx b/docs/cli/environment-variables.mdx index d440c3f..e43e501 100644 --- a/docs/cli/environment-variables.mdx +++ b/docs/cli/environment-variables.mdx @@ -25,6 +25,12 @@ description: "Configure failproofai behavior with environment variables" |----------|-------------| | `FAILPROOFAI_TELEMETRY_DISABLED=1` | Disable anonymous usage telemetry | +## First-run prompt + +| Variable | Description | +|----------|-------------| +| `FAILPROOFAI_NO_FIRST_RUN=1` | Skip the prompt that offers to install policies on the first bare `failproofai` invocation | + ## LLM (for policy evaluation) | Variable | Description | diff --git a/docs/introduction.mdx b/docs/introduction.mdx index b1e9582..2d1989c 100644 --- a/docs/introduction.mdx +++ b/docs/introduction.mdx @@ -50,7 +50,7 @@ bun add -g failproofai ```bash -failproofai policies --install # enable policies +failproofai policies --install # enable policies (or skip — `failproofai` will offer to set them up on first run) failproofai # launch the dashboard ``` From 04b06978ed8e994e3ceedbbd74044fc2109c6d57 Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Thu, 21 May 2026 20:14:07 -0700 Subject: [PATCH 3/4] docs: add CHANGELOG entry for first-run prompt docs update Co-Authored-By: Claude Opus 4.7 --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4b7e2ab..6677b74 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,9 @@ - Add a first-run install prompt on bare `failproofai` invocations. PostHog showed only ~10% of npm-installed users ever ran `failproofai policies --install`; the no-args dashboard launch now detects "zero hooks installed across any detected CLI" and offers to run the existing interactive policy-selection inline (covering all of Claude Code, Codex, Copilot, Cursor, OpenCode, Pi, Gemini). Non-TTY contexts (CI, piped invocations) print a short stderr hint and fall through to the dashboard. New `src/hooks/first-run-nudge.ts` module, a guard in `bin/failproofai.mjs` before `launch("start")`, plus four new PostHog events (`first_run_nudge_shown`, `_accepted`, `_declined`, `_skipped_noninteractive`) so the uplift is measurable. Postinstall message extended with a "Next steps" block when the brand-new-user case is detected (`!configured && !registered`). Opt-out via `FAILPROOFAI_NO_FIRST_RUN=1`. >>>>>>> c19621a (feat: first-run install prompt on bare `failproofai` invocation) +### Docs +- Document the new first-run prompt in the README and `docs/introduction.mdx` quickstart snippets (calling out that `failproofai policies --install` is now optional — running bare `failproofai` will offer to do it), and add a new "First-run prompt" section to `docs/cli/environment-variables.mdx` for `FAILPROOFAI_NO_FIRST_RUN=1`. Chinese mirror and the 14 translated env-vars files left for the translation-sync workflow. + ### Breaking - Remove the undocumented cloud auth + event relay subsystem ahead of a from-scratch redesign. Deletes `src/auth/` (OAuth 2.0 device-flow login against `api.befailproof.ai`, `~/.failproofai/auth.json` token store) and `src/relay/` (WebSocket event relay daemon, sanitized JSONL queue at `~/.failproofai/cache/server-queue/`, PID tracking). Strips the `failproofai login` / `logout` / `whoami` / `relay start|stop|status` / `sync` subcommands and the internal `--relay-daemon` mode from `bin/failproofai.mjs`, along with their `--help` entries and "did you mean" suggestions. Removes the fire-and-forget `appendToServerQueue` + `ensureRelayRunning` calls from `src/hooks/handler.ts` so hook evaluation no longer enqueues events or lazy-spawns a daemon. The whole subsystem had zero references in `README.md`, `docs/`, `examples/`, or `__tests__/`, and only had internal cross-imports — `tsc`, `eslint`, `vitest` (1623 tests), and the `bun run build` bundles all stay green. Users who ran `failproofai login` should also wipe `~/.failproofai/{auth.json,cache/server-queue,relay.pid}` and stop any running relay daemon by hand; new auth/cloud surface will land in a follow-up. From 25fdda1f9a038afa37115b16065227dda5468d65 Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Thu, 21 May 2026 20:17:14 -0700 Subject: [PATCH 4/4] fix: drop conflict markers left in CHANGELOG by rebase onto main The rebase auto-merge for #376's Features entry left literal <<<<<<< markers around the two coexisting bullets. Both entries are valid; they're now side-by-side under 0.0.11-beta.2. Co-Authored-By: Claude Opus 4.7 --- CHANGELOG.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6677b74..3d00edf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,11 +3,8 @@ ## 0.0.11-beta.2 — 2026-05-21 ### Features -<<<<<<< HEAD - Expand PostHog telemetry coverage to close the 16 server-side and 12 web-UI gaps surfaced by the May audit (#376). Server-side adds `cli_install_success` / `cli_install_failure` / `cli_uninstall_success` / `cli_uninstall_failure` / `cli_list_invoked` / `cli_parse_error` / `cli_unexpected_error` / `hook_dispatch_error` (CLI lifecycle outcomes in `bin/failproofai.mjs`), `hook_stdin_error` / `hook_payload_parse_error` (hook handler input errors in `src/hooks/handler.ts`), `policy_evaluation_error` (builtin policy crashes in `src/hooks/policy-evaluator.ts`, distinct from the existing `custom_hook_error`), `custom_policy_validation_failed` / `custom_hooks_load_error` / `policy_params_validation_warning` / `scope_validation_failed` / `hook_write_failed` / `multi_scope_warning_shown` / `cli_detection_summary` / `beta_policies_installed` (manager / loader / install-prompt internals), and `first_install` / `version_changed` (lifecycle detection in `scripts/postinstall.mjs` via a new `~/.failproofai/last-version` file). Web-UI adds `policies_tab_switched` / `activity_filter_changed` (debounced) / `activity_row_toggled` / `activity_copy_clicked` / `activity_pagination_changed` / `cli_selection_toggled` / `cli_install_remove_submitted` / `cli_reinstall_submitted` / `policy_config_modal_opened` / `policy_config_modal_closed` / `action_error_displayed` / `hooks_install_from_error_clicked` via `usePostHog()` in `app/policies/hooks-client.tsx`. The deny-/instruct-only condition at `handler.ts:344` (allow-path tracking) is intentionally left unchanged. All events go through the existing helpers (`trackHookEvent`, `trackInstallEvent`, `captureClientEvent`) and honor `FAILPROOFAI_TELEMETRY_DISABLED=1`. -======= - Add a first-run install prompt on bare `failproofai` invocations. PostHog showed only ~10% of npm-installed users ever ran `failproofai policies --install`; the no-args dashboard launch now detects "zero hooks installed across any detected CLI" and offers to run the existing interactive policy-selection inline (covering all of Claude Code, Codex, Copilot, Cursor, OpenCode, Pi, Gemini). Non-TTY contexts (CI, piped invocations) print a short stderr hint and fall through to the dashboard. New `src/hooks/first-run-nudge.ts` module, a guard in `bin/failproofai.mjs` before `launch("start")`, plus four new PostHog events (`first_run_nudge_shown`, `_accepted`, `_declined`, `_skipped_noninteractive`) so the uplift is measurable. Postinstall message extended with a "Next steps" block when the brand-new-user case is detected (`!configured && !registered`). Opt-out via `FAILPROOFAI_NO_FIRST_RUN=1`. ->>>>>>> c19621a (feat: first-run install prompt on bare `failproofai` invocation) ### Docs - Document the new first-run prompt in the README and `docs/introduction.mdx` quickstart snippets (calling out that `failproofai policies --install` is now optional — running bare `failproofai` will offer to do it), and add a new "First-run prompt" section to `docs/cli/environment-variables.mdx` for `FAILPROOFAI_NO_FIRST_RUN=1`. Chinese mirror and the 14 translated env-vars files left for the translation-sync workflow.