diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 3746f46..e18e8bd 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -110,15 +110,16 @@ Every artifact in the bundle shares one `snapshotId`; the CLI will not mix a fai ```bash testsprite agent install claude # install the skill for Claude Code testsprite agent install codex # install into AGENTS.md for Codex (managed-section) +testsprite agent install gemini # install into GEMINI.md for Gemini CLI (managed-section) testsprite agent install cursor # .cursor/rules/testsprite-verify.mdc testsprite agent install cline # .clinerules/testsprite-verify.md testsprite agent install antigravity # .agents/skills/testsprite-verify/SKILL.md -testsprite agent list # list all 5 targets with status + mode + path +testsprite agent list # list all 6 targets with status + mode + path ``` -Supported targets: `claude` (GA), `codex` (experimental), `cursor` (experimental), `cline` (experimental), `antigravity` (experimental). +Supported targets: `claude` (GA), `codex` (experimental), `gemini` (experimental), `cursor` (experimental), `cline` (experimental), `antigravity` (experimental). -The `codex` target uses **managed-section mode** — it writes only a sentinel-delimited section inside your existing `AGENTS.md`, so your project instructions are never clobbered. Re-running without `--force` replaces the section in-place; user content outside the sentinels is always preserved. +The `codex` and `gemini` targets use **managed-section mode** — they write only a sentinel-delimited section inside your existing `AGENTS.md` or `GEMINI.md`, so your project instructions are never clobbered. Re-running without `--force` replaces the section in-place; user content outside the sentinels is always preserved. Re-running with `--force` on **own-file targets** (claude, cursor, cline, antigravity) backs up the existing file to `.bak` first. @@ -180,11 +181,11 @@ testsprite test get test_xxxxxxxx --dry-run --output json #### `testsprite test code get ` -Print the generated test source. With `--out `, write it to a file instead of stdout (text mode writes the source body; JSON mode writes the wire envelope). +Print the generated test source. TestSprite test code is **Python**: frontend tests are Playwright (`playwright.async_api`, async), backend tests use `requests` with `pytest`-style assertions. With `--out `, write it to a file instead of stdout (text mode writes the source body; JSON mode writes the wire envelope). ```bash testsprite test code get test_xxxxxxxx -testsprite test code get test_xxxxxxxx --out ./test_xxxxxxxx.spec.ts +testsprite test code get test_xxxxxxxx --out ./test_xxxxxxxx.py testsprite test code get test_xxxxxxxx --dry-run --output json ``` @@ -300,12 +301,12 @@ testsprite test delete test_xxxxxxxx --dry-run --output json #### `testsprite test code put ` -Replace the generated test code with a new file. The CLI uses an etag (`codeVersion`) for optimistic-concurrency control: it auto-fetches the current version, or pass `--expected-version` to pin one, or `--force` to skip the guard. +Replace the generated test code with a new file. **The replacement must be Python** — the execution engine runs the stored code with Python `exec()` (frontend: Playwright `playwright.async_api`; backend: `requests` + assertions), so a TypeScript/JavaScript file would fail at run time with a `SyntaxError`. The CLI uses an etag (`codeVersion`) for optimistic-concurrency control: it auto-fetches the current version, or pass `--expected-version` to pin one, or `--force` to skip the guard. ```bash -testsprite test code put test_xxxxxxxx --code-file ./test.spec.ts -testsprite test code put test_xxxxxxxx --code-file ./test.spec.ts --expected-version v3 -testsprite test code put test_xxxxxxxx --code-file ./test.spec.ts --dry-run --output json +testsprite test code put test_xxxxxxxx --code-file ./test.py +testsprite test code put test_xxxxxxxx --code-file ./test.py --expected-version v3 +testsprite test code put test_xxxxxxxx --code-file ./test.py --dry-run --output json ``` #### `testsprite test plan put ` diff --git a/README.md b/README.md index 2d90012..c9009d8 100644 --- a/README.md +++ b/README.md @@ -61,13 +61,13 @@ npm install -g @testsprite/testsprite-cli testsprite setup ``` -`testsprite setup` prompts for your [API key](https://www.testsprite.com), verifies it, and installs the verification-loop skill for your coding agent (`claude`, `cursor`, `cline`, `antigravity`, `codex`, etc.) — one command, so your agent is wired to verify its own work. Non-interactive (CI / onboarding scripts): +`testsprite setup` prompts for your [API key](https://www.testsprite.com), verifies it, and installs the verification-loop skill for your coding agent (`claude`, `cursor`, `cline`, `antigravity`, `codex`, `gemini`, etc.) — one command, so your agent is wired to verify its own work. Non-interactive (CI / onboarding scripts): ```bash TESTSPRITE_API_KEY=sk-... testsprite setup --from-env --yes --agent claude ``` -> **Pointing a coding agent (Claude Code, Cursor, Codex, Cline, …) at TestSprite?** Have it run `testsprite setup` first — that installs the verification skill, so the agent knows how to create, run, and triage tests on its own (instead of guessing from this README). New here? Start with the **[getting-started overview](https://docs.testsprite.com/cli/getting-started/overview)**. +> **Pointing a coding agent (Claude Code, Gemini CLI, Cursor, Codex, Cline, …) at TestSprite?** Have it run `testsprite setup` first — that installs the verification skill, so the agent knows how to create, run, and triage tests on its own (instead of guessing from this README). New here? Start with the **[getting-started overview](https://docs.testsprite.com/cli/getting-started/overview)**. From there, the loop runs on its own — an example session, typed by the coding agent: diff --git a/docs/cli-v1-agent-install/onboard-skill-template.md b/docs/cli-v1-agent-install/onboard-skill-template.md new file mode 100644 index 0000000..49add5a --- /dev/null +++ b/docs/cli-v1-agent-install/onboard-skill-template.md @@ -0,0 +1,178 @@ +--- +name: testsprite-onboard +description: Stand up a complete, runnable TestSprite test suite for the current repo at first use — create a project (with a target URL and auth), derive a coherent set of tests from the codebase, batch-create them, and smoke-run a few to a green verdict so the user immediately has something worth running. Use ONLY when a repo has no TestSprite tests yet (a fresh project), right after `testsprite setup`, or when the user asks to "set up / bootstrap / seed tests". This is first-run setup, NOT change verification — once a project already has tests, use the testsprite-verify skill instead. +--- + + + +# TestSprite: onboard a repo with a seed test suite + +Your job is to take a repo that has **no TestSprite tests yet** and leave it with a +**coherent, runnable suite** plus a couple of **already-green** smoke tests — in one pass. +A new user who can immediately run a real, passing test is an activated user; an empty +project is the #1 drop-off. + +This skill only uses shipped CLI commands. It works the same whether the user is on the V2 +or V3 backend — the CLI routes internally. Do **not** call backend APIs directly. + +## When to use + +- Right after `testsprite setup`, or any time the active project has 0–1 tests. +- The user says "set up tests", "bootstrap", "seed a suite", "get me started", or similar. + +## When NOT to use + +- The project already has tests — that's the `testsprite-verify` skill's job, not this one. +- The user only changed code and wants it checked — again, `testsprite-verify`. + +## Prerequisites + +`testsprite setup` has run (an API key is configured). If `testsprite project list` errors on +auth, stop and tell the user to run `testsprite setup` first — don't try to configure for them. + +## Steps + +### 1. Understand the app (don't skip — this is where coverage quality comes from) + +Read the repo to establish, concretely: + +- **Frontend**: the deployed/local **URL**, the 4–8 most important user flows (auth, core + CRUD, checkout, search, settings…), and whether flows need **login**. +- **Backend**: the key **API endpoints** and their success/error contracts. + +Prefer **code-derived** routes/handlers over guessing — you have the source; use it. This +beats a blind crawl. + +### 2. Create the project (FE must have a URL) + +Frontend: + +```bash +testsprite project create --type frontend --name "" --url \ + [--username --password-file ] +``` + +Backend: + +```bash +testsprite project create --type backend --name "" +``` + +Capture the returned `projectId`. + +> **Critical for FE**: a frontend test with no resolvable target URL fails immediately with +> `No environment URL configured` — the suite goes all-red. Always pass `--url`. If flows need +> login, pass `--username/--password-file` now so authenticated pages are reachable. + +### 3. Author the tests (quality over quantity) + +**Frontend** — one JSON plan file per flow, in a directory (e.g. `./testsprite-plans/`). +Each file is a COMPLETE plan and must include `projectId` (from step 2), `type: "frontend"`, +`name`, and `planSteps` — `create-batch` reads the project from each file, not from a flag: + +```json +{ + "projectId": "", + "type": "frontend", + "name": "Checkout — guest can complete a purchase", + "planSteps": [ + { "type": "action", "description": "Navigate to /products and open the first product" }, + { "type": "action", "description": "Click 'Add to cart', then go to /cart" }, + { + "type": "assertion", + "description": "The cart shows exactly 1 line item with the product's name and price" + }, + { "type": "action", "description": "Proceed to checkout as guest and submit the test payment" }, + { "type": "assertion", "description": "A confirmation page appears showing an order number" } + ] +} +``` + +**Backend** — one `.py` file per endpoint, using `requests` with concrete assertions on +status code and response body. + +**Assertion rule (this is the whole game for FE):** every `assertion` step must name a +**concrete, observable** outcome — an element, text, URL, count, or status. Never write +`"verify it works"`, `"check the page loads"`, or other narrative that an AI judge can +rubber-stamp. Vague assertions are how false-PASS sneaks in. + +Aim for ~8–15 tests covering the core flows. Don't pad. + +### 4. Batch-create + +Frontend (one call, up to 50 plans from the directory — `create-batch` is FE-only and has +**no `--project` flag**; the project comes from each plan file's `projectId`): + +```bash +testsprite test create-batch --plan-from-dir ./testsprite-plans +``` + +Backend (one call per file — `create-batch` is FE-only; `--name` is required): + +```bash +testsprite test create --type backend --name "" \ + --code-file ./tests/.py --project +``` + +Capture the created `testId`s from the output. + +### 5. Smoke-run a few — NOT all (protect credits) + +Pick the **2–3 highest-value happy-path** tests (prefer ones you're most confident are green) +and run only those: + +```bash +testsprite test run --wait +``` + +Do **not** run the whole suite automatically — a 20-test FE suite is ~40 credits and a free +account only has 150. Running the full suite is the user's explicit choice. + +### 6. Report + +Tell the user, plainly: + +- "Your project now has **N** tests covering: ." +- "I smoke-ran **M** — here's the result: ." +- "To run the rest (≈X credits — state the cost so they choose knowingly): + - frontend — run each remaining test by id: `testsprite test run --wait` (there is + **no `--all` for frontend**); + - backend — `testsprite test run --all --project ` (wave-ordered, runs every BE test)." + +## Quality checklist (self-check before reporting done) + +- [ ] FE project has a real `--url`; login configured if the app needs it. +- [ ] Every FE assertion names a concrete, observable outcome (no "verify it works"). +- [ ] Tests cover the core flows you found in the code, not just one page. +- [ ] Smoke-ran 2–3 happy-path tests, not the whole suite. +- [ ] Reported test count, smoke result + dashboard link, and the cost to run the rest. + +## Don'ts + +- Don't auto-run the full suite (credit wall / surprise 402). +- Don't write narrative assertions an AI judge can't fail. +- Don't call backend endpoints directly — only the `testsprite` CLI. +- Don't create a FE project without a URL. +- Don't re-seed a project that already has tests — that's not this skill's job. + +## Hand off to verify + +This skill's job ends once the project has a seeded suite and a first green run. From here on, +the **`testsprite-verify`** skill takes over: after the user changes code, it runs the tests +covering that change before they report the work done. Onboard once; verify continuously. diff --git a/skills/testsprite-onboard.skill.md b/skills/testsprite-onboard.skill.md new file mode 100644 index 0000000..cc9342c --- /dev/null +++ b/skills/testsprite-onboard.skill.md @@ -0,0 +1,162 @@ + + +# TestSprite: onboard a repo with a seed test suite + +Your job is to take a repo that has **no TestSprite tests yet** and leave it with a +**coherent, runnable suite** plus a couple of **already-green** smoke tests — in one pass. +A new user who can immediately run a real, passing test is an activated user; an empty +project is the #1 drop-off. + +This skill only uses shipped CLI commands. It works the same whether the user is on the V2 +or V3 backend — the CLI routes internally. Do **not** call backend APIs directly. + +## When to use + +- Right after `testsprite setup`, or any time the active project has 0–1 tests. +- The user says "set up tests", "bootstrap", "seed a suite", "get me started", or similar. + +## When NOT to use + +- The project already has tests — that's the `testsprite-verify` skill's job, not this one. +- The user only changed code and wants it checked — again, `testsprite-verify`. + +## Prerequisites + +`testsprite setup` has run (an API key is configured). If `testsprite project list` errors on +auth, stop and tell the user to run `testsprite setup` first — don't try to configure for them. + +## Steps + +### 1. Understand the app (don't skip — this is where coverage quality comes from) + +Read the repo to establish, concretely: + +- **Frontend**: the deployed/local **URL**, the 4–8 most important user flows (auth, core + CRUD, checkout, search, settings…), and whether flows need **login**. +- **Backend**: the key **API endpoints** and their success/error contracts. + +Prefer **code-derived** routes/handlers over guessing — you have the source; use it. This +beats a blind crawl. + +### 2. Create the project (FE must have a URL) + +Frontend: + +```bash +testsprite project create --type frontend --name "" --url \ + [--username --password-file ] +``` + +Backend: + +```bash +testsprite project create --type backend --name "" +``` + +Capture the returned `projectId`. + +> **Critical for FE**: a frontend test with no resolvable target URL fails immediately with +> `No environment URL configured` — the suite goes all-red. Always pass `--url`. If flows need +> login, pass `--username/--password-file` now so authenticated pages are reachable. + +### 3. Author the tests (quality over quantity) + +**Frontend** — one JSON plan file per flow, in a directory (e.g. `./testsprite-plans/`). +Each file is a COMPLETE plan and must include `projectId` (from step 2), `type: "frontend"`, +`name`, and `planSteps` — `create-batch` reads the project from each file, not from a flag: + +```json +{ + "projectId": "", + "type": "frontend", + "name": "Checkout — guest can complete a purchase", + "planSteps": [ + { "type": "action", "description": "Navigate to /products and open the first product" }, + { "type": "action", "description": "Click 'Add to cart', then go to /cart" }, + { + "type": "assertion", + "description": "The cart shows exactly 1 line item with the product's name and price" + }, + { "type": "action", "description": "Proceed to checkout as guest and submit the test payment" }, + { "type": "assertion", "description": "A confirmation page appears showing an order number" } + ] +} +``` + +**Backend** — one `.py` file per endpoint, using `requests` with concrete assertions on +status code and response body. + +**Assertion rule (this is the whole game for FE):** every `assertion` step must name a +**concrete, observable** outcome — an element, text, URL, count, or status. Never write +`"verify it works"`, `"check the page loads"`, or other narrative that an AI judge can +rubber-stamp. Vague assertions are how false-PASS sneaks in. + +Aim for ~8–15 tests covering the core flows. Don't pad. + +### 4. Batch-create + +Frontend (one call, up to 50 plans from the directory — `create-batch` is FE-only and has +**no `--project` flag**; the project comes from each plan file's `projectId`): + +```bash +testsprite test create-batch --plan-from-dir ./testsprite-plans +``` + +Backend (one call per file — `create-batch` is FE-only; `--name` is required): + +```bash +testsprite test create --type backend --name "" \ + --code-file ./tests/.py --project +``` + +Capture the created `testId`s from the output. + +### 5. Smoke-run a few — NOT all (protect credits) + +Pick the **2–3 highest-value happy-path** tests (prefer ones you're most confident are green) +and run only those: + +```bash +testsprite test run --wait +``` + +Do **not** run the whole suite automatically — a 20-test FE suite is ~40 credits and a free +account only has 150. Running the full suite is the user's explicit choice. + +### 6. Report + +Tell the user, plainly: + +- "Your project now has **N** tests covering: ." +- "I smoke-ran **M** — here's the result: ." +- "To run the rest (≈X credits — state the cost so they choose knowingly): + - frontend — run each remaining test by id: `testsprite test run --wait` (there is + **no `--all` for frontend**); + - backend — `testsprite test run --all --project ` (wave-ordered, runs every BE test)." + +## Quality checklist (self-check before reporting done) + +- [ ] FE project has a real `--url`; login configured if the app needs it. +- [ ] Every FE assertion names a concrete, observable outcome (no "verify it works"). +- [ ] Tests cover the core flows you found in the code, not just one page. +- [ ] Smoke-ran 2–3 happy-path tests, not the whole suite. +- [ ] Reported test count, smoke result + dashboard link, and the cost to run the rest. + +## Don'ts + +- Don't auto-run the full suite (credit wall / surprise 402). +- Don't write narrative assertions an AI judge can't fail. +- Don't call backend endpoints directly — only the `testsprite` CLI. +- Don't create a FE project without a URL. +- Don't re-seed a project that already has tests — that's not this skill's job. + +## Hand off to verify + +This skill's job ends once the project has a seeded suite and a first green run. From here on, +the **`testsprite-verify`** skill takes over: after the user changes code, it runs the tests +covering that change before they report the work done. Onboard once; verify continuously. diff --git a/skills/testsprite-verify.codex.md b/skills/testsprite-verify.codex.md index c641526..f2c2456 100644 --- a/skills/testsprite-verify.codex.md +++ b/skills/testsprite-verify.codex.md @@ -64,6 +64,8 @@ testsprite test run --all --project [--filter ] \ already have the change deployed (e.g. a CI preview deploy) — the CLI tests a deployed URL, it doesn't host your environment. Running earlier verifies the previous build. +- Backend `--code-file`: the runner executes the file top-to-bottom (not `pytest`), so **call your `test_*` function(s) at the end of the file** — a defined-but-uncalled test silently passes. +- Backend sandbox has only stdlib + `requests` + `pytest` + `numpy` + `scipy`. Test the API over HTTP with `requests`; do **not** `import` the project's own source modules or other packages (e.g. `torch`) — they aren't installed and the test won't run. - `--wait` long-polls until terminal. Do not wrap it in a retry loop. - Exit `0` = passed; `1` = failed/blocked; `7` = timeout (resume with `test wait `). - BE dependency flags (`--produces`/`--needs`/`--category`) are backend-only and diff --git a/skills/testsprite-verify.skill.md b/skills/testsprite-verify.skill.md index 34a7dcc..650a88b 100644 --- a/skills/testsprite-verify.skill.md +++ b/skills/testsprite-verify.skill.md @@ -135,7 +135,10 @@ language; you don't write browser code. **Backend — write the Python yourself and use `--code-file`.** There is no server-side codegen on the CLI. Read the API surface that changed (OpenAPI, the route handler, request/response shapes) and write a pytest-style assertion script -to a tempfile: +to a tempfile. **End the file by calling your `test_*` function(s)** — the runner +executes the file top-to-bottom and does NOT auto-discover/collect test functions +the way `pytest` does, so a test that is only _defined_ (never called) silently +passes regardless of its assertions: ```python # /tmp/login-empty-password.py — runs against the project's target URL, creds injected. @@ -145,8 +148,23 @@ def test_login_rejects_empty_password(): r = requests.post(f"{TARGET_URL}/login", json={"email": "a@b.c", "password": ""}) assert r.status_code == 400 assert r.json().get("error") == "invalid password" + +# Required: actually invoke the test so its assertions run. +test_login_rejects_empty_password() ``` +**Execution environment (backend).** The code runs in a locked-down sandbox with +only the Python **standard library + `requests` + `pytest` + `numpy` + `scipy`** +(plus `requests`' own deps like `urllib3`). So: + +- **Test the API over HTTP** with `requests` against the target URL — that's what a + backend test verifies. +- **Do NOT `import` the project's own source modules** (e.g. `from app.services import …`, + `import core`, `import model`) or other third-party/ML packages (e.g. `torch`, + `pandas`, `django`). They are not installed, so the test fails to even run. +- Get values from the API's responses (and captured variables), not by importing and + calling the app's internals. + **Backend tests that share state declare dependencies at create time.** For a one-off verification, prefer a single self-contained script (log in inside the same file). But when the coverage set splits naturally into producer → consumer diff --git a/src/commands/agent.test.ts b/src/commands/agent.test.ts index bb558f3..41897d3 100644 --- a/src/commands/agent.test.ts +++ b/src/commands/agent.test.ts @@ -4,8 +4,14 @@ import path from 'node:path'; import { describe, expect, it, vi } from 'vitest'; import { ApiError, CLIError } from '../lib/errors.js'; import { + DEFAULT_SKILLS, + GEMINI_MANAGED_SECTION_BEGIN, + GEMINI_MANAGED_SECTION_END, MANAGED_SECTION_BEGIN, MANAGED_SECTION_END, + ONBOARD_CODEX_LINE, + SKILLS, + pathFor, renderForTarget, TARGETS, type AgentTarget, @@ -135,12 +141,13 @@ describe('runInstall — fresh install', () => { debug: false, dryRun: false, target: [t], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, ); - const { path: relPath, content } = renderForTarget(t); + const { path: relPath, content } = renderForTarget(t, 'testsprite-verify'); const abs = path.resolve(CWD, relPath); // File was written to store @@ -165,12 +172,13 @@ describe('runInstall — fresh install', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, ); - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(CWD, relPath); expect(store.get(abs)).toBe(content); }); @@ -186,6 +194,7 @@ describe('runInstall — fresh install', () => { debug: false, dryRun: false, target: ['claude', 'antigravity'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -208,6 +217,7 @@ describe('runInstall — fresh install', () => { debug: false, dryRun: false, target: ['cline'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -228,6 +238,7 @@ describe('runInstall — fresh install', () => { debug: false, dryRun: false, target: ['cursor'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -248,7 +259,7 @@ describe('runInstall — idempotency (skipped)', () => { const { capture, deps } = makeCapture(); // Pre-seed with the canonical content - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(CWD, relPath); seedFile(abs, content); @@ -261,6 +272,7 @@ describe('runInstall — idempotency (skipped)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -283,7 +295,7 @@ describe('runInstall — conflict (blocked)', () => { const { store, fs: agentFs, writeCalls, seedFile } = makeMemFs(); const { capture, deps } = makeCapture(); - const { path: relPath } = renderForTarget('claude'); + const { path: relPath } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(CWD, relPath); seedFile(abs, 'DIFFERENT CONTENT'); @@ -298,6 +310,7 @@ describe('runInstall — conflict (blocked)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -329,7 +342,7 @@ describe('runInstall — --force', () => { const { store, fs: agentFs, writeCalls, seedFile } = makeMemFs(); const { capture, deps } = makeCapture(); - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(CWD, relPath); const oldContent = 'OLD CONTENT'; seedFile(abs, oldContent); @@ -341,6 +354,7 @@ describe('runInstall — --force', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: true, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -363,7 +377,7 @@ describe('runInstall — --force', () => { const { store, fs: agentFs, seedFile } = makeMemFs(); const { deps: deps1 } = makeCapture(); - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(CWD, relPath); const firstEdit = 'FIRST EDIT'; seedFile(abs, firstEdit); @@ -376,6 +390,7 @@ describe('runInstall — --force', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: true, }, { cwd: CWD, fs: agentFs, ...deps1 }, @@ -398,6 +413,7 @@ describe('runInstall — --force', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: true, }, { cwd: CWD, fs: agentFs, ...deps2 }, @@ -427,6 +443,7 @@ describe('runInstall — --dry-run', () => { debug: false, dryRun: true, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -468,13 +485,14 @@ describe('runInstall — --dir override', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: customDir, }, { cwd: CWD, fs: agentFs, ...deps }, ); - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(customDir, relPath); expect(store.get(abs)).toBe(content); // Not written under CWD @@ -534,6 +552,7 @@ describe('runInstall — multi-target', () => { debug: false, dryRun: false, target: ['claude', 'cursor'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -555,6 +574,7 @@ describe('runInstall — multi-target', () => { debug: false, dryRun: false, target: ['claude,cursor'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -581,6 +601,7 @@ describe('runInstall — multi-target', () => { debug: false, dryRun: false, target: ['claude', 'cursor'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -617,6 +638,7 @@ describe('runInstall — multi-target', () => { debug: false, dryRun: false, target: ['claude', 'claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -665,7 +687,15 @@ describe('runInstall — empty target', () => { const promptFn = vi.fn().mockResolvedValue('claude'); await runInstall( - { profile: 'default', output: 'text', debug: false, dryRun: false, target: [], force: false }, + { + profile: 'default', + output: 'text', + debug: false, + dryRun: false, + target: [], + skills: ['testsprite-verify'], + force: false, + }, { cwd: CWD, fs: agentFs, isTTY: true, prompt: promptFn, ...deps }, ); @@ -681,7 +711,15 @@ describe('runInstall — empty target', () => { const promptFn = vi.fn().mockResolvedValue(''); // empty => default to claude await runInstall( - { profile: 'default', output: 'text', debug: false, dryRun: false, target: [], force: false }, + { + profile: 'default', + output: 'text', + debug: false, + dryRun: false, + target: [], + skills: ['testsprite-verify'], + force: false, + }, { cwd: CWD, fs: agentFs, isTTY: true, prompt: promptFn, ...deps }, ); @@ -695,7 +733,7 @@ describe('runInstall — empty target', () => { // --------------------------------------------------------------------------- describe('runList', () => { - it('returns all five targets with correct status', async () => { + it('returns all six targets with correct status', async () => { const { capture, deps } = makeCapture(); await runList({ profile: 'default', output: 'text', debug: false, dryRun: false }, deps); @@ -706,6 +744,7 @@ describe('runList', () => { expect(out).toContain('cline'); expect(out).toContain('antigravity'); expect(out).toContain('codex'); + expect(out).toContain('gemini'); expect(out).toContain('ga'); expect(out).toContain('experimental'); // All matrix paths present @@ -714,28 +753,37 @@ describe('runList', () => { expect(out).toContain(TARGETS.cline.path); expect(out).toContain(TARGETS.antigravity.path); expect(out).toContain(TARGETS.codex.path); + expect(out).toContain(TARGETS.gemini.path); }); - it('JSON mode emits array of {target, status, path, mode}', async () => { + it('JSON mode emits array of {target, skill, status, path, mode}', async () => { const { capture, deps } = makeCapture(); await runList({ profile: 'default', output: 'json', debug: false, dryRun: false }, deps); const json = JSON.parse(capture.stdout.join('\n')) as ListResult[]; expect(Array.isArray(json)).toBe(true); - expect(json).toHaveLength(5); + // 6 targets × 2 default skills = 12 rows + expect(json).toHaveLength(12); const targets = json.map(r => r.target); expect(targets).toContain('claude'); expect(targets).toContain('cursor'); expect(targets).toContain('cline'); expect(targets).toContain('antigravity'); expect(targets).toContain('codex'); - const claudeEntry = json.find(r => r.target === 'claude'); + expect(targets).toContain('gemini'); + // skill field present on each row + const skills = json.map(r => r.skill); + expect(skills).toContain('testsprite-verify'); + expect(skills).toContain('testsprite-onboard'); + const claudeEntry = json.find(r => r.target === 'claude' && r.skill === 'testsprite-verify'); expect(claudeEntry?.status).toBe('ga'); expect(claudeEntry?.path).toBe(TARGETS.claude.path); // codex entry has mode: managed-section - const codexEntry = json.find(r => r.target === 'codex'); + const codexEntry = json.find(r => r.target === 'codex' && r.skill === 'testsprite-verify'); expect(codexEntry?.mode).toBe('managed-section'); + const geminiEntry = json.find(r => r.target === 'gemini' && r.skill === 'testsprite-verify'); + expect(geminiEntry?.mode).toBe('managed-section'); }); it('text mode has a header row', async () => { @@ -745,6 +793,7 @@ describe('runList', () => { const lines = capture.stdout.join('\n').split('\n'); expect(lines[0]).toMatch(/TARGET/i); + expect(lines[0]).toMatch(/SKILL/i); expect(lines[0]).toMatch(/STATUS/i); expect(lines[0]).toMatch(/PATH/i); }); @@ -755,7 +804,7 @@ describe('runList', () => { // --------------------------------------------------------------------------- describe('runInstall — output modes', () => { - it('JSON mode emits array of {target, path, action}', async () => { + it('JSON mode emits array of {target, path, action, skills}', async () => { const { fs: agentFs } = makeMemFs(); const { capture, deps } = makeCapture(); @@ -766,6 +815,7 @@ describe('runInstall — output modes', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -773,7 +823,11 @@ describe('runInstall — output modes', () => { const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; expect(Array.isArray(json)).toBe(true); - expect(json[0]).toMatchObject({ target: 'claude', action: 'written' }); + expect(json[0]).toMatchObject({ + target: 'claude', + action: 'written', + skills: ['testsprite-verify'], + }); expect(json[0]?.path).toBe(TARGETS.claude.path); }); @@ -788,6 +842,7 @@ describe('runInstall — output modes', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -872,6 +927,7 @@ describe('runInstall — all four own-file targets', () => { debug: false, dryRun: false, target: ['claude', 'cursor', 'cline', 'antigravity'], + skills: ['testsprite-verify'], force: false, }, { cwd: CWD, fs: agentFs, ...deps }, @@ -892,7 +948,7 @@ describe('runInstall — all four own-file targets', () => { // --------------------------------------------------------------------------- describe('runInstall — dry-run all own-file targets', () => { - it('writes nothing for any of the four own-file targets', async () => { + it('writes nothing for any of the four own-file targets (default 2 skills = 8 would-write lines)', async () => { const { store, fs: agentFs } = makeMemFs(); const { capture, deps } = makeCapture(); @@ -912,9 +968,9 @@ describe('runInstall — dry-run all own-file targets', () => { const stderrOut = capture.stderr.join('\n'); // Banner appears once expect(stderrOut).toContain('[dry-run] no files written'); - // Four would-write lines + // 4 targets × 2 default skills = 8 would-write lines const wouldWriteLines = stderrOut.split('\n').filter(l => l.includes('would write')); - expect(wouldWriteLines.length).toBe(4); + expect(wouldWriteLines.length).toBe(8); }); }); @@ -934,13 +990,14 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: tmpRoot, }, { ...deps }, // no fs injected → uses defaultAgentFs ); - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(tmpRoot, relPath); // File exists on real disk expect(readFileSync(abs, 'utf8')).toBe(content); @@ -959,6 +1016,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: tmpRoot, }, @@ -974,6 +1032,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: tmpRoot, }, @@ -995,6 +1054,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: tmpRoot, }, @@ -1002,7 +1062,7 @@ describe('runInstall — default AgentFs (real disk)', () => { ); // Mutate the file - const { path: relPath, content } = renderForTarget('claude'); + const { path: relPath, content } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(tmpRoot, relPath); const oldContent = 'MODIFIED BY USER'; // Use default fs to write the modified content @@ -1018,6 +1078,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: true, dir: tmpRoot, }, @@ -1047,6 +1108,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: false, dir: tmpRoot, }, @@ -1065,7 +1127,7 @@ describe('runInstall — default AgentFs (real disk)', () => { it('refuses to overwrite a symlinked target file (real disk) with --force — exit 5', async () => { const tmpRoot = mkdtempSync(path.join(tmpdir(), 'agent-test-symlink-target-')); const outsideDir = mkdtempSync(path.join(tmpdir(), 'agent-test-outside-target-')); - const { path: relPath } = renderForTarget('claude'); + const { path: relPath } = renderForTarget('claude', 'testsprite-verify'); const abs = path.resolve(tmpRoot, relPath); const nodeFs = await import('node:fs/promises'); await nodeFs.mkdir(path.dirname(abs), { recursive: true }); @@ -1084,6 +1146,7 @@ describe('runInstall — default AgentFs (real disk)', () => { debug: false, dryRun: false, target: ['claude'], + skills: ['testsprite-verify'], force: true, dir: tmpRoot, }, @@ -1109,6 +1172,7 @@ const BASE_OPTS = { output: 'text' as const, debug: false, dryRun: false, + skills: ['testsprite-verify'], }; describe('runInstall — path safety', () => { @@ -1259,7 +1323,7 @@ describe('runInstall — symlink safety', () => { const abs = path.resolve(CWD, TARGETS.claude.path); seedFile(abs, 'DIFFERENT CONTENT'); // real file that differs -> overwrite seedSymlink(`${abs}.bak`); // a planted symlink at the default .bak slot - const { content } = renderForTarget('claude'); + const { content } = renderForTarget('claude', 'testsprite-verify'); const { deps } = makeCapture(); await runInstall( @@ -1284,7 +1348,7 @@ describe('runInstall — backup collision', () => { const abs = path.resolve(CWD, TARGETS.claude.path); seedFile(abs, 'CURRENT EDIT'); seedFile(`${abs}.bak`, 'PRECIOUS USER BACKUP'); // a backup the user already has - const { content } = renderForTarget('claude'); + const { content } = renderForTarget('claude', 'testsprite-verify'); const { capture, deps } = makeCapture(); await runInstall( @@ -1353,6 +1417,29 @@ describe('runInstall — codex managed-section: create (AGENTS.md absent)', () = }); }); +describe('runInstall — gemini managed-section: create (GEMINI.md absent)', () => { + it('creates GEMINI.md with Gemini sentinels when file is absent', async () => { + const { store, fs: agentFs, writeCalls } = makeMemFs(); + const { capture, deps } = makeCapture(); + + await runInstall( + { ...BASE_OPTS, target: ['gemini'], force: false }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const abs = path.resolve(CWD, TARGETS.gemini.path); + expect(store.has(abs)).toBe(true); + const written = store.get(abs)!; + expect(written).toContain(GEMINI_MANAGED_SECTION_BEGIN); + expect(written).toContain(GEMINI_MANAGED_SECTION_END); + expect(written).toContain('testsprite test run'); + const out = capture.stdout.join('\n'); + expect(out).toContain('gemini'); + expect(out).toContain('section-installed'); + expect(writeCalls.some(p => p === abs)).toBe(true); + }); +}); + describe('runInstall — codex managed-section: append (AGENTS.md exists, no sentinels)', () => { it('appends the section to existing AGENTS.md content', async () => { const { store, fs: agentFs, seedFile } = makeMemFs(); @@ -1400,6 +1487,38 @@ describe('runInstall — codex managed-section: append (AGENTS.md exists, no sen }); }); +describe('runInstall — gemini managed-section: append (GEMINI.md exists, no sentinels)', () => { + it('appends the section to existing GEMINI.md content and is idempotent', async () => { + const { store, fs: agentFs, seedFile } = makeMemFs(); + const { capture, deps } = makeCapture(); + + const geminiAbs = path.resolve(CWD, TARGETS.gemini.path); + const existingContent = '# Project Instructions\n\nKeep existing Gemini CLI notes.\n'; + seedFile(geminiAbs, existingContent); + + await runInstall( + { ...BASE_OPTS, target: ['gemini'], force: false }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const written = store.get(geminiAbs)!; + expect(written).toContain('# Project Instructions'); + expect(written).toContain('Keep existing Gemini CLI notes.'); + expect(written).toContain(GEMINI_MANAGED_SECTION_BEGIN); + expect(written).toContain(GEMINI_MANAGED_SECTION_END); + expect(capture.stdout.join('\n')).toContain('section-installed'); + + await runInstall( + { ...BASE_OPTS, target: ['gemini'], force: false }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const rewritten = store.get(geminiAbs)!; + expect(rewritten.split(GEMINI_MANAGED_SECTION_BEGIN).length - 1).toBe(1); + expect(rewritten.split(GEMINI_MANAGED_SECTION_END).length - 1).toBe(1); + }); +}); + describe('runInstall — codex managed-section: replace (sentinels already present)', () => { it('replaces the section when content differs, preserves surrounding text', async () => { const { store, fs: agentFs, seedFile } = makeMemFs(); @@ -1955,3 +2074,374 @@ describe('[P3 round-2] codex --dry-run: composed-size precision + read-failure s ).rejects.toMatchObject({ exitCode: 5 }); }); }); + +// --------------------------------------------------------------------------- +// Multi-skill behavior — new tests covering the SKILLS registry refactor +// --------------------------------------------------------------------------- + +describe('runInstall — multi-skill: default install writes BOTH skills (own-file target)', () => { + it('default claude install produces 2 results: verify + onboard, both action:written', async () => { + const { store, fs: agentFs, writeCalls } = makeMemFs(); + const { capture, deps } = makeCapture(); + + // No skills option → DEFAULT_SKILLS (both verify and onboard); use json output to parse results + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: undefined, + target: ['claude'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + // Both skill files should be written + const verifyAbs = path.resolve(CWD, pathFor('claude', 'testsprite-verify')); + const onboardAbs = path.resolve(CWD, pathFor('claude', 'testsprite-onboard')); + expect(store.has(verifyAbs)).toBe(true); + expect(store.has(onboardAbs)).toBe(true); + + // Both writes recorded + expect(writeCalls).toContain(verifyAbs); + expect(writeCalls).toContain(onboardAbs); + + // JSON output contains 2 results + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + + // There must be a result for testsprite-verify + const verifyResult = json.find(r => r.skills.includes('testsprite-verify')); + expect(verifyResult).toBeDefined(); + expect(verifyResult?.target).toBe('claude'); + expect(verifyResult?.action).toBe('written'); + expect(verifyResult?.path).toBe(pathFor('claude', 'testsprite-verify')); + + // There must be a result for testsprite-onboard + const onboardResult = json.find(r => r.skills.includes('testsprite-onboard')); + expect(onboardResult).toBeDefined(); + expect(onboardResult?.target).toBe('claude'); + expect(onboardResult?.action).toBe('written'); + expect(onboardResult?.path).toBe(pathFor('claude', 'testsprite-onboard')); + }); + + it('onboard skill file content contains the onboard H1 heading', async () => { + const { store, fs: agentFs } = makeMemFs(); + const { deps } = makeCapture(); + + await runInstall( + { ...BASE_OPTS, dryRun: false, skills: undefined, target: ['claude'], force: false }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const onboardAbs = path.resolve(CWD, pathFor('claude', 'testsprite-onboard')); + const content = store.get(onboardAbs)!; + // The onboard skill body must contain its H1 + expect(content).toContain('# TestSprite: onboard a repo'); + }); + + it('each result has skills:[skill] (one-element array) for own-file targets', async () => { + const { fs: agentFs } = makeMemFs(); + const { capture, deps } = makeCapture(); + + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: undefined, + target: ['claude'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + expect(json.length).toBe(2); + for (const r of json) { + expect(Array.isArray(r.skills)).toBe(true); + expect(r.skills.length).toBe(1); + } + }); + + it('text output FORMAT is unchanged: one row per result (target padEnd(12) action padEnd(12) path), 2 rows for default install', async () => { + const { fs: agentFs } = makeMemFs(); + const { capture, deps } = makeCapture(); + + await runInstall( + { + ...BASE_OPTS, + dryRun: false, + output: 'text', + skills: undefined, + target: ['claude'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const lines = capture.stdout.join('\n').split('\n').filter(Boolean); + // 2 results → 2 lines + expect(lines.length).toBe(2); + // Each line has target, action, and path + for (const line of lines) { + expect(line).toContain('claude'); + expect(line).toContain('written'); + } + // One line for each skill path + expect(lines.some(l => l.includes(pathFor('claude', 'testsprite-verify')))).toBe(true); + expect(lines.some(l => l.includes(pathFor('claude', 'testsprite-onboard')))).toBe(true); + }); +}); + +describe('runInstall — multi-skill: default codex install aggregates BOTH skills in ONE section', () => { + it('creates ONE AGENTS.md managed section containing verify H1 AND onboard one-liner', async () => { + const { store, fs: agentFs, writeCalls } = makeMemFs(); + const { capture, deps } = makeCapture(); + + // Default (no skills opt) → both skills; json output for result parsing + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: undefined, + target: ['codex'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const agentsAbs = path.resolve(CWD, TARGETS.codex.path); + expect(writeCalls).toContain(agentsAbs); + + const written = store.get(agentsAbs)!; + // Exactly ONE BEGIN sentinel (not two) + const beginCount = written.split(MANAGED_SECTION_BEGIN).length - 1; + expect(beginCount).toBe(1); + + // Section contains the verify H1 + expect(written).toContain('# TestSprite Verification Loop'); + + // Section contains the onboard one-liner + expect(written).toContain('**First-time setup:**'); + + // The result is a single codex result with skills = both + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + expect(json.length).toBe(1); + const codexResult = json[0]!; + expect(codexResult.target).toBe('codex'); + expect(codexResult.action).toBe('section-installed'); + expect(codexResult.skills).toContain('testsprite-verify'); + expect(codexResult.skills).toContain('testsprite-onboard'); + }); + + it('single-skill codex install produces a section byte-identical to old single-skill behavior', async () => { + // A ['testsprite-verify']-only codex install should produce the same section + // content as the pre-refactor behavior (single-skill codex body). + const { store: storeA, fs: fsA } = makeMemFs(); + const { deps: depsA } = makeCapture(); + await runInstall( + { + ...BASE_OPTS, + dryRun: false, + skills: ['testsprite-verify'], + target: ['codex'], + force: false, + }, + { cwd: CWD, fs: fsA, ...depsA }, + ); + const verifyOnlyContent = storeA.get(path.resolve(CWD, TARGETS.codex.path))!; + + // The section must contain the verify H1 but NOT the onboard line + expect(verifyOnlyContent).toContain('# TestSprite Verification Loop'); + expect(verifyOnlyContent).not.toContain('**First-time setup:**'); + // Exactly one BEGIN sentinel + expect(verifyOnlyContent.split(MANAGED_SECTION_BEGIN).length - 1).toBe(1); + }); +}); + +describe('runInstall — multi-skill: --skill subset installs only the named skill', () => { + it('skills:[testsprite-onboard] installs ONLY the onboard file (1 result, 1 write)', async () => { + const { store, fs: agentFs, writeCalls } = makeMemFs(); + const { capture, deps } = makeCapture(); + + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: ['testsprite-onboard'], + target: ['claude'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const onboardAbs = path.resolve(CWD, pathFor('claude', 'testsprite-onboard')); + const verifyAbs = path.resolve(CWD, pathFor('claude', 'testsprite-verify')); + + // Only onboard written; verify NOT written + expect(store.has(onboardAbs)).toBe(true); + expect(store.has(verifyAbs)).toBe(false); + expect(writeCalls).toContain(onboardAbs); + expect(writeCalls).not.toContain(verifyAbs); + + // Exactly 1 result + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + expect(json.length).toBe(1); + expect(json[0]?.skills).toEqual(['testsprite-onboard']); + expect(json[0]?.action).toBe('written'); + }); + + it('skills:[testsprite-verify] installs ONLY the verify file (1 result)', async () => { + const { store, fs: agentFs } = makeMemFs(); + const { capture, deps } = makeCapture(); + + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: ['testsprite-verify'], + target: ['claude'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const verifyAbs = path.resolve(CWD, pathFor('claude', 'testsprite-verify')); + const onboardAbs = path.resolve(CWD, pathFor('claude', 'testsprite-onboard')); + + expect(store.has(verifyAbs)).toBe(true); + expect(store.has(onboardAbs)).toBe(false); + + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + expect(json.length).toBe(1); + expect(json[0]?.skills).toEqual(['testsprite-verify']); + }); +}); + +describe('runInstall — multi-skill: unknown --skill exits 5', () => { + it('skills:[bogus] → localValidationError exit 5 with documented message', async () => { + const { fs: agentFs, writeCalls } = makeMemFs(); + const { deps } = makeCapture(); + + let thrown: unknown; + try { + await runInstall( + { ...BASE_OPTS, dryRun: false, skills: ['bogus'], target: ['claude'], force: false }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + } catch (err) { + thrown = err; + } + + expect(thrown).toBeInstanceOf(ApiError); + expect((thrown as ApiError).exitCode).toBe(5); + // The nextAction must contain the exact documented message format + const nextAction = (thrown as ApiError).nextAction ?? ''; + expect(nextAction).toContain('unknown skill "bogus"'); + expect(nextAction).toContain('testsprite-verify'); + expect(nextAction).toContain('testsprite-onboard'); + // Nothing written + expect(writeCalls.length).toBe(0); + }); + + it('unknown skill via createAgentCommand parseAsync → exit 5', async () => { + const { fs: agentFs } = makeMemFs(); + const { deps } = makeCapture(); + + const command = createAgentCommand({ cwd: CWD, fs: agentFs, ...deps }); + const parent = new (await import('commander')).Command('testsprite'); + parent.option('--output ', 'output', 'text'); + parent.option('--profile ', 'profile', 'default'); + parent.option('--endpoint-url '); + parent.option('--debug', 'debug', false); + parent.option('--verbose', 'verbose', false); + parent.option('--dry-run', 'dry-run', false); + parent.addCommand(command); + + let thrown: unknown; + try { + await parent.parseAsync([ + 'node', + 'ts', + 'agent', + 'install', + '--target=claude', + '--skill=bogus', + `--dir=${CWD}`, + ]); + } catch (err) { + thrown = err; + } + + expect(thrown).toBeDefined(); + const isValidationErr = + (thrown instanceof ApiError && thrown.exitCode === 5) || + (thrown instanceof CLIError && thrown.exitCode === 5); + expect(isValidationErr).toBe(true); + }); +}); + +describe('runInstall — multi-skill: multi-target own-file with default skills', () => { + it('default install to claude + cursor writes 4 files (2 targets × 2 skills)', async () => { + const { store, fs: agentFs, writeCalls } = makeMemFs(); + const { capture, deps } = makeCapture(); + + // No skills → default both; json output for result parsing + await runInstall( + { + ...BASE_OPTS, + output: 'json', + dryRun: false, + skills: undefined, + target: ['claude', 'cursor'], + force: false, + }, + { cwd: CWD, fs: agentFs, ...deps }, + ); + + const expectedPaths = [ + path.resolve(CWD, pathFor('claude', 'testsprite-verify')), + path.resolve(CWD, pathFor('claude', 'testsprite-onboard')), + path.resolve(CWD, pathFor('cursor', 'testsprite-verify')), + path.resolve(CWD, pathFor('cursor', 'testsprite-onboard')), + ]; + + for (const p of expectedPaths) { + expect(store.has(p)).toBe(true); + expect(writeCalls).toContain(p); + } + + const json = JSON.parse(capture.stdout.join('\n')) as InstallResult[]; + expect(json.length).toBe(4); + expect(json.every(r => r.action === 'written')).toBe(true); + }); +}); + +describe('runInstall — SKILLS registry / DEFAULT_SKILLS contract', () => { + it('DEFAULT_SKILLS contains exactly testsprite-verify and testsprite-onboard', () => { + expect(DEFAULT_SKILLS).toContain('testsprite-verify'); + expect(DEFAULT_SKILLS).toContain('testsprite-onboard'); + expect(DEFAULT_SKILLS.length).toBe(2); + }); + + it('SKILLS registry contains both skills with required fields', () => { + expect(SKILLS['testsprite-verify']).toBeDefined(); + expect(SKILLS['testsprite-onboard']).toBeDefined(); + for (const [name, spec] of Object.entries(SKILLS)) { + expect(spec.name).toBe(name); + expect(typeof spec.description).toBe('string'); + expect(spec.description.length).toBeGreaterThan(0); + expect(typeof spec.bodyFile).toBe('string'); + expect(spec.codex).toBeDefined(); + } + }); + + it('ONBOARD_CODEX_LINE is the one-liner used in the codex section', () => { + expect(typeof ONBOARD_CODEX_LINE).toBe('string'); + expect(ONBOARD_CODEX_LINE).toContain('**First-time setup:**'); + }); +}); diff --git a/src/commands/agent.ts b/src/commands/agent.ts index c27e2b8..6236189 100644 --- a/src/commands/agent.ts +++ b/src/commands/agent.ts @@ -8,12 +8,14 @@ import { GLOBAL_OPTS_HINT, Output } from '../lib/output.js'; import { promptText } from '../lib/prompt.js'; import { type AgentTarget, + type ManagedSectionSpec, TARGETS, - loadSkillBody, - loadCodexSkillBody, + SKILLS, + DEFAULT_SKILLS, + pathFor, + loadSkillBodyFor, + buildCodexAggregate, renderForTarget, - MANAGED_SECTION_BEGIN, - MANAGED_SECTION_END, } from '../lib/agent-targets.js'; // --------------------------------------------------------------------------- @@ -143,15 +145,15 @@ async function writeBackup(agentFs: AgentFs, abs: string, existing: string): Pro } // --------------------------------------------------------------------------- -// Managed-section helpers (codex target) +// Managed-section helpers (root instruction file targets) // --------------------------------------------------------------------------- /** * Build the section block to inject (sentinels + body + trailing newline). * Uses \n throughout; the caller handles CRLF normalisation. */ -function buildSection(body: string): string { - return `${MANAGED_SECTION_BEGIN}\n${body.trimEnd()}\n${MANAGED_SECTION_END}\n`; +function buildSection(body: string, managed: ManagedSectionSpec): string { + return `${managed.begin}\n${body.trimEnd()}\n${managed.end}\n`; } /** @@ -171,7 +173,7 @@ type SectionState = | { kind: 'corrupt' }; /** - * Inspect an existing AGENTS.md and classify the managed-section state. + * Inspect an existing root instruction file and classify the managed-section state. * * Sentinel-matching rules (P2 hardening): * - Only STANDALONE sentinel lines count (a line that consists solely of the @@ -182,7 +184,11 @@ type SectionState = * - CRLF files are handled by stripping trailing \r from each line before * comparison. */ -function classifySection(existing: string, section: string): SectionState { +function classifySection( + existing: string, + section: string, + managed: ManagedSectionSpec, +): SectionState { // Split on LF; strip trailing CR so CRLF files normalise correctly. const lines = existing.split('\n'); @@ -193,8 +199,8 @@ function classifySection(existing: string, section: string): SectionState { for (let i = 0; i < lines.length; i++) { const stripped = (lines[i] ?? '').trimEnd(); - if (stripped === MANAGED_SECTION_BEGIN) beginLines.push(i); - else if (stripped === MANAGED_SECTION_END) endLines.push(i); + if (stripped === managed.begin) beginLines.push(i); + else if (stripped === managed.end) endLines.push(i); } const hasBegin = beginLines.length > 0; @@ -316,6 +322,12 @@ export interface InstallResult { target: AgentTarget; path: string; // repo-relative matrix path action: InstallAction; + /** + * Skill(s) this result covers. Own-file targets produce one result per skill + * (`[skill]`); managed-section targets produce ONE result whose + * section aggregates every installed skill (`[...skills]`). + */ + skills: string[]; } // --------------------------------------------------------------------------- @@ -326,6 +338,8 @@ type CommonOptions = FactoryCommonOptions; interface InstallOptions extends CommonOptions { target: string[]; + /** Skill subset to install; empty/absent → {@link DEFAULT_SKILLS}. */ + skills?: string[]; dir?: string; force: boolean; } @@ -385,13 +399,51 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr return true; }) as AgentTarget[]; + // 2b. Resolve + validate the skill set (empty/absent → DEFAULT_SKILLS). + // Accepts comma-separated or repeated --skill values, same shape as --target. + const rawSkills = (opts.skills ?? []) + .flatMap(s => s.split(',')) + .map(s => s.trim()) + .filter(Boolean); + const validSkills = Object.keys(SKILLS); + for (const s of rawSkills) { + if (!validSkills.includes(s)) { + throw localValidationError( + 'skill', + `unknown skill "${s}"; supported: ${validSkills.join(', ')}`, + ); + } + } + const seenSkill = new Set(); + const skills = (rawSkills.length > 0 ? rawSkills : [...DEFAULT_SKILLS]).filter(s => { + if (seenSkill.has(s)) return false; + seenSkill.add(s); + return true; + }); + // 3. Resolve dir const dir = opts.dir ?? deps.cwd ?? process.cwd(); const root = path.resolve(dir); - // 4. Load skill bodies (lazy — only touch disk if a target actually needs it) - let ownFileBody: string | undefined; - let codexBody: string | undefined; + // 4. Lazy asset loaders — only touch disk if a target actually needs it. + // own-file bodies are per-skill (cached); managed-section targets aggregate EVERY + // installed skill's contribution into ONE managed section. + const skillBodyCache = new Map(); + const bodyForSkill = (skill: string): string => { + let b = skillBodyCache.get(skill); + if (b === undefined) { + b = loadSkillBodyFor(skill); + skillBodyCache.set(skill, b); + } + return b; + }; + let managedSectionBodyCache: string | undefined; + const getManagedSection = (managed: ManagedSectionSpec): string => { + if (managedSectionBodyCache === undefined) { + managedSectionBodyCache = buildCodexAggregate(skills); + } + return buildSection(managedSectionBodyCache, managed); + }; const results: InstallResult[] = []; @@ -401,20 +453,22 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr // 5. Process each target for (const t of targets) { const spec = TARGETS[t]; - const relPath = spec.path; - const abs = path.resolve(root, relPath); - - // Path safety: ensure abs is inside root (defense against .. in relPath or dir) - if (abs !== root && !abs.startsWith(root + path.sep)) { - throw new CLIError(`refusing to write outside --dir: ${relPath}`, 5); - } // ----------------------------------------------------------------------- - // managed-section mode (codex target) + // managed-section mode — ONE section aggregating all skills // ----------------------------------------------------------------------- if (spec.mode === 'managed-section') { - if (codexBody === undefined) codexBody = loadCodexSkillBody(); - const section = buildSection(codexBody); + const managedConfig = spec.managedSection; + if (managedConfig === undefined) { + throw new CLIError(`managed-section target "${t}" is missing sentinel metadata`, 5); + } + const relPath = spec.path; // skill-independent (all skills merge here) + const abs = path.resolve(root, relPath); + // Path safety: ensure abs is inside root (defense against .. in relPath or dir) + if (abs !== root && !abs.startsWith(root + path.sep)) { + throw new CLIError(`refusing to write outside --dir: ${relPath}`, 5); + } + const section = getManagedSection(managedConfig); if (opts.dryRun) { // Dry-run: report what would happen without writing disk. @@ -441,7 +495,6 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr // replace path and misses the append separator. Read failures other // than ENOENT are surfaced (EACCES/EIO must not read as "absent" — // absence is already represented by dryRunSt === null). - const bytes = Buffer.byteLength(section, 'utf8'); let wouldBeContent = section; if (dryRunSt !== null) { let existing: string | null = null; @@ -458,7 +511,7 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr } } if (existing !== null) { - const state = classifySection(existing, section); + const state = classifySection(existing, section, managedConfig); if (state.kind === 'corrupt') { // The real install would refuse with exit 5 — dry-run reports // the same outcome rather than a misleading success. @@ -477,13 +530,16 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr } } const wouldBeBytes = Buffer.byteLength(wouldBeContent, 'utf8'); - if (wouldBeBytes > AGENTS_MD_CODEX_BUDGET_BYTES) { + if ( + managedConfig.loadBudgetBytes !== undefined && + wouldBeBytes > managedConfig.loadBudgetBytes + ) { stderrFn( - `[warn] ${relPath} will be ${wouldBeBytes} bytes after this write — Codex may not load content beyond its 32 KiB (${AGENTS_MD_CODEX_BUDGET_BYTES} byte) budget. Trim AGENTS.md to stay within the limit.`, + `[warn] ${relPath} will be ${wouldBeBytes} bytes after this write — ${managedConfig.loadBudgetLabel ?? `the target agent may not load content beyond its ${managedConfig.loadBudgetBytes} byte budget`}. Trim ${relPath} to stay within the limit.`, ); } - dryRunLines.push({ abs, bytes, note: 'managed section' }); - results.push({ target: t, path: relPath, action: 'dry-run' }); + dryRunLines.push({ abs, bytes: wouldBeBytes, note: 'managed section' }); + results.push({ target: t, path: relPath, action: 'dry-run', skills: [...skills] }); continue; } @@ -498,21 +554,23 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr } /** - * [P2] Emit a stderr warn when the would-be file content exceeds Codex's - * 32 KiB load budget. We still write — this is a warn, not a refusal — - * but the operator needs early visibility so they can trim AGENTS.md. + * Emit a stderr warning when this target has a documented load budget and + * the would-be file content exceeds it. We still write — this is visibility, + * not a refusal. */ + const budgetBytes = managedConfig.loadBudgetBytes; + const budgetLabel = managedConfig.loadBudgetLabel; function warnIfOverBudget(wouldBeContent: string): void { const byteLen = Buffer.byteLength(wouldBeContent, 'utf8'); - if (byteLen > AGENTS_MD_CODEX_BUDGET_BYTES) { + if (budgetBytes !== undefined && byteLen > budgetBytes) { stderrFn( - `[warn] ${relPath} will be ${byteLen} bytes after this write — Codex may not load content beyond its 32 KiB (${AGENTS_MD_CODEX_BUDGET_BYTES} byte) budget. Trim AGENTS.md to stay within the limit.`, + `[warn] ${relPath} will be ${byteLen} bytes after this write — ${budgetLabel ?? `the target agent may not load content beyond its ${budgetBytes} byte budget`}. Trim ${relPath} to stay within the limit.`, ); } } if (st === null) { - // File absent → create AGENTS.md containing just the section. + // File absent → create the target instruction file containing just the section. warnIfOverBudget(section); await agentFs.mkdir(path.dirname(abs)); try { @@ -526,10 +584,15 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr } throw err; } - results.push({ target: t, path: relPath, action: 'section-installed' }); + results.push({ + target: t, + path: relPath, + action: 'section-installed', + skills: [...skills], + }); } else { const existing = await agentFs.readFile(abs); - const state = classifySection(existing, section); + const state = classifySection(existing, section, managedConfig); if (state.kind === 'corrupt') { // BEGIN without matching END (or vice-versa) — never destroy user content. @@ -541,12 +604,22 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr } if (state.kind === 'unchanged') { - results.push({ target: t, path: relPath, action: 'section-unchanged' }); + results.push({ + target: t, + path: relPath, + action: 'section-unchanged', + skills: [...skills], + }); } else if (state.kind === 'create') { // Shouldn't happen (st !== null means file exists), but guard anyway. warnIfOverBudget(section); await agentFs.writeFile(abs, section); - results.push({ target: t, path: relPath, action: 'section-installed' }); + results.push({ + target: t, + path: relPath, + action: 'section-installed', + skills: [...skills], + }); } else { // 'append' or 'replace' — write the new content. // --force has no special meaning for managed-section: we always merge @@ -557,72 +630,82 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr await agentFs.writeFile(abs, newContent); const action: InstallAction = state.kind === 'append' ? 'section-installed' : 'section-updated'; - results.push({ target: t, path: relPath, action }); + results.push({ target: t, path: relPath, action, skills: [...skills] }); } } continue; } // ----------------------------------------------------------------------- - // own-file mode (all other targets) + // own-file mode (all other targets) — one file per skill // ----------------------------------------------------------------------- - if (ownFileBody === undefined) ownFileBody = loadSkillBody(); - const content = renderForTarget(t, ownFileBody).content; + for (const skill of skills) { + const relPath = pathFor(t, skill); + const abs = path.resolve(root, relPath); + // Path safety: ensure abs is inside root (defense against .. in relPath or dir) + if (abs !== root && !abs.startsWith(root + path.sep)) { + throw new CLIError(`refusing to write outside --dir: ${relPath}`, 5); + } + const content = renderForTarget(t, skill, bodyForSkill(skill)).content; - if (opts.dryRun) { - const bytes = Buffer.byteLength(content, 'utf8'); - dryRunLines.push({ abs, bytes, note: '' }); - results.push({ target: t, path: relPath, action: 'dry-run' }); - continue; - } + if (opts.dryRun) { + const bytes = Buffer.byteLength(content, 'utf8'); + dryRunLines.push({ abs, bytes, note: '' }); + results.push({ target: t, path: relPath, action: 'dry-run', skills: [skill] }); + continue; + } - // Inspect the target path: refuse to traverse or write through a symlink - // (fs writes follow symlinks, which would let a planted symlink escape - // --dir), and reject a non-regular-file landing path. The lexical guard - // above is necessary but not sufficient — it cannot see symlinks. - const st = await inspectTargetPath(agentFs, root, relPath); + // Inspect the target path: refuse to traverse or write through a symlink + // (fs writes follow symlinks, which would let a planted symlink escape + // --dir), and reject a non-regular-file landing path. The lexical guard + // above is necessary but not sufficient — it cannot see symlinks. + const st = await inspectTargetPath(agentFs, root, relPath); - if (st !== null && !st.isFile) { - throw new CLIError(`${relPath} exists but is not a regular file — remove it and re-run.`, 5); - } + if (st !== null && !st.isFile) { + throw new CLIError( + `${relPath} exists but is not a regular file — remove it and re-run.`, + 5, + ); + } - if (st === null) { - // Path does not exist — create it. inspectTargetPath verified every - // existing ancestor is a real directory; exclusive create (wx) then - // ensures a file or symlink that races in after the check is not followed - // or silently overwritten. - await agentFs.mkdir(path.dirname(abs)); - try { - await agentFs.writeFile(abs, content, { exclusive: true }); - } catch (err) { - if (err instanceof Error && (err as NodeJS.ErrnoException).code === 'EEXIST') { - throw new CLIError( - `${relPath} appeared after the path check — re-run, or pass --force to overwrite.`, - 6, - ); + if (st === null) { + // Path does not exist — create it. inspectTargetPath verified every + // existing ancestor is a real directory; exclusive create (wx) then + // ensures a file or symlink that races in after the check is not followed + // or silently overwritten. + await agentFs.mkdir(path.dirname(abs)); + try { + await agentFs.writeFile(abs, content, { exclusive: true }); + } catch (err) { + if (err instanceof Error && (err as NodeJS.ErrnoException).code === 'EEXIST') { + throw new CLIError( + `${relPath} appeared after the path check — re-run, or pass --force to overwrite.`, + 6, + ); + } + throw err; } - throw err; - } - results.push({ target: t, path: relPath, action: 'written' }); - } else { - const existing = await agentFs.readFile(abs); - if (existing === content) { - // Byte-identical — skip - results.push({ target: t, path: relPath, action: 'skipped' }); - } else if (!opts.force) { - // Differs and no --force → blocked - results.push({ target: t, path: relPath, action: 'blocked' }); + results.push({ target: t, path: relPath, action: 'written', skills: [skill] }); } else { - // Differs and --force → back up the current bytes to a fresh slot - // (never clobbering an existing backup or following a symlink), then - // overwrite. The overwrite itself can follow a symlink swapped in after - // the check — an accepted TOCTOU residual for a local, single-user CLI. - const backupPath = await writeBackup(agentFs, abs, existing); - await agentFs.writeFile(abs, content); - if (opts.output === 'text') { - stderrFn(`backed up ${relPath} to ${path.relative(root, backupPath)}`); + const existing = await agentFs.readFile(abs); + if (existing === content) { + // Byte-identical — skip + results.push({ target: t, path: relPath, action: 'skipped', skills: [skill] }); + } else if (!opts.force) { + // Differs and no --force → blocked + results.push({ target: t, path: relPath, action: 'blocked', skills: [skill] }); + } else { + // Differs and --force → back up the current bytes to a fresh slot + // (never clobbering an existing backup or following a symlink), then + // overwrite. The overwrite itself can follow a symlink swapped in after + // the check — an accepted TOCTOU residual for a local, single-user CLI. + const backupPath = await writeBackup(agentFs, abs, existing); + await agentFs.writeFile(abs, content); + if (opts.output === 'text') { + stderrFn(`backed up ${relPath} to ${path.relative(root, backupPath)}`); + } + results.push({ target: t, path: relPath, action: 'updated', skills: [skill] }); } - results.push({ target: t, path: relPath, action: 'updated' }); } } } @@ -666,6 +749,7 @@ export async function runInstall(opts: InstallOptions, deps: AgentDeps = {}): Pr export interface ListResult { target: AgentTarget; + skill: string; status: string; mode: string; path: string; @@ -674,20 +758,32 @@ export interface ListResult { export async function runList(opts: CommonOptions, deps: AgentDeps = {}): Promise { const out = makeOutput(opts.output, deps); - const results: ListResult[] = ( - Object.entries(TARGETS) as [AgentTarget, { status: string; mode: string; path: string }][] - ).map(([t, spec]) => ({ - target: t, - status: spec.status, - mode: spec.mode, - path: spec.path, - })); + // One row per (target × default skill). Own-file targets land each skill at a + // distinct path; the codex managed-section target merges all skills into the + // single AGENTS.md (so every codex row shares that path — truthful, since both + // skills' content lands there). + const results: ListResult[] = []; + for (const [t, spec] of Object.entries(TARGETS) as [ + AgentTarget, + { status: string; mode: string }, + ][]) { + for (const skill of DEFAULT_SKILLS) { + results.push({ + target: t, + skill, + status: spec.status, + mode: spec.mode, + path: pathFor(t, skill), + }); + } + } out.print(results, data => { const items = data as ListResult[]; - const header = `${'TARGET'.padEnd(14)} ${'STATUS'.padEnd(12)} ${'MODE'.padEnd(18)} PATH`; + const header = `${'TARGET'.padEnd(14)} ${'SKILL'.padEnd(20)} ${'STATUS'.padEnd(12)} ${'MODE'.padEnd(18)} PATH`; const rows = items.map( - r => `${r.target.padEnd(14)} ${r.status.padEnd(12)} ${r.mode.padEnd(18)} ${r.path}`, + r => + `${r.target.padEnd(14)} ${r.skill.padEnd(20)} ${r.status.padEnd(12)} ${r.mode.padEnd(18)} ${r.path}`, ); return [header, ...rows].join('\n'); }); @@ -703,17 +799,23 @@ function collect(v: string, prev: string[]): string[] { export function createAgentCommand(deps: AgentDeps = {}): Command { const agent = new Command('agent').description( - 'Install TestSprite guidance into coding-agent config (Claude Code, Cursor, Cline, Antigravity, Codex)', + 'Install TestSprite guidance into coding-agent config (Claude Code, Cursor, Cline, Antigravity, Codex, Gemini CLI)', ); agent .command('install') .description( - 'Write the TestSprite verification-loop skill file into a project for a coding agent', + 'Write the TestSprite agent skills (verification loop + first-run onboarding) into a project for a coding agent', ) .option( '--target ', - 'Agent target(s): claude, cursor, cline, antigravity, codex (comma-separated or repeated)', + 'Agent target(s): claude, cursor, cline, antigravity, codex, gemini (comma-separated or repeated)', + collect, + [], + ) + .option( + '--skill ', + `Skill(s) to install: ${Object.keys(SKILLS).join(', ')} (comma-separated or repeated; default: all)`, collect, [], ) @@ -721,15 +823,19 @@ export function createAgentCommand(deps: AgentDeps = {}): Command { .option( '--force', 'For own-file targets: overwrite existing file (a .bak backup is kept). ' + - 'For codex (managed-section): replaces the section unconditionally; user content outside the section is never destroyed.', + 'For managed-section targets: replaces the section unconditionally; user content outside the section is never destroyed.', ) .addHelpText('after', GLOBAL_OPTS_HINT) .action( - async (cmdOpts: { target: string[]; dir?: string; force?: boolean }, command: Command) => { + async ( + cmdOpts: { target: string[]; skill: string[]; dir?: string; force?: boolean }, + command: Command, + ) => { await runInstall( { ...resolveCommonOptions(command), target: cmdOpts.target, + skills: cmdOpts.skill, dir: cmdOpts.dir, force: Boolean(cmdOpts.force), }, @@ -740,7 +846,7 @@ export function createAgentCommand(deps: AgentDeps = {}): Command { agent .command('list') - .description('List supported agent targets, their status, and landing paths') + .description('List supported agent targets and skills, their status, and landing paths') .addHelpText('after', GLOBAL_OPTS_HINT) .action(async (_o, command: Command) => { await runList(resolveCommonOptions(command), deps); diff --git a/src/commands/init.ts b/src/commands/init.ts index dc3effd..6a432c2 100644 --- a/src/commands/init.ts +++ b/src/commands/init.ts @@ -19,7 +19,7 @@ import type { AuthDeps, MeResponse } from './auth.js'; import { runConfigure, runWhoami } from './auth.js'; import type { AgentDeps, AgentFs, InstallResult } from './agent.js'; import { runInstall } from './agent.js'; -import { TARGETS, type AgentTarget } from '../lib/agent-targets.js'; +import { TARGETS, DEFAULT_SKILLS, type AgentTarget } from '../lib/agent-targets.js'; import type { FetchImpl } from '../lib/http.js'; import { readProfile } from '../lib/credentials.js'; @@ -96,10 +96,27 @@ export interface InitSummary { env: string; email?: string; scopes: string[]; - agent: { target: string; action: string } | null; + /** + * Agent skill install outcome. `action` is an AGGREGATE across the installed + * skills (setup installs {@link DEFAULT_SKILLS}); `skills` lists which skills + * landed. `null` when --no-agent. + */ + agent: { target: string; action: string; skills?: string[] } | null; status: 'initialized'; } +/** + * Collapse the per-skill install actions into one representative action for the + * init summary. Precedence: a real change (updated) outranks a fresh install, + * which outranks a no-op. `blocked` never reaches here — runInstall throws first. + */ +function aggregateInstallAction(actions: string[]): string { + if (actions.some(a => a === 'updated' || a === 'section-updated')) return 'updated'; + if (actions.some(a => a === 'written' || a === 'section-installed')) return 'installed'; + if (actions.some(a => a === 'dry-run')) return 'dry-run'; + return 'skipped'; // all skipped / section-unchanged +} + // --------------------------------------------------------------------------- // Helpers to split deps into the two primitive shapes // --------------------------------------------------------------------------- @@ -227,7 +244,9 @@ export async function runInit(opts: InitOptions, deps: InitDeps = {}): Promise { try { const parsed = JSON.parse(line) as InstallResult[]; - if (Array.isArray(parsed) && parsed.length > 0 && parsed[0]) { - capturedInstallResult = parsed[0]; + if (Array.isArray(parsed) && parsed.length > 0) { + capturedInstallResults = parsed; } } catch { // ignore non-JSON lines (shouldn't happen in json mode, but be safe) @@ -302,6 +324,7 @@ export async function runInit(opts: InitOptions, deps: InitDeps = {}): Promise 0 + ? aggregateInstallAction(capturedInstallResults.map(r => r.action)) + : 'installed'; + // De-dupe skills across results, preserving first-seen order. + installedSkills = [...new Set(capturedInstallResults.flatMap(r => r.skills ?? []))]; } catch (installErr) { // Fix 6: credentials were already saved (Step 1+2 above succeeded). // Emit a clear summary line BEFORE re-throwing so the user knows their @@ -330,7 +356,11 @@ export async function runInit(opts: InitOptions, deps: InitDeps = {}): Promise 0 ? installedSkills : [...DEFAULT_SKILLS], + }; const summary: InitSummary = { profile: opts.profile, @@ -365,6 +395,9 @@ function renderInitText(data: unknown): string { lines.push(''); if (s.agent) { lines.push(` agent: ${s.agent.target} (${s.agent.action})`); + if (s.agent.skills && s.agent.skills.length > 0) { + lines.push(` skills: ${s.agent.skills.join(', ')}`); + } } else { lines.push(' agent: skipped (--no-agent)'); } @@ -408,7 +441,7 @@ function parseRequestTimeoutFlag(raw: string | undefined): number | undefined { } const SETUP_DESCRIPTION = - 'Set up TestSprite: configure your API key and install the verification skill for your coding agent'; + 'Set up TestSprite: configure your API key and install the TestSprite agent skills for your coding agent'; /** Raw Commander options shared by `setup` and the deprecated `init` alias. */ interface SetupCmdOpts { diff --git a/src/lib/agent-targets.test.ts b/src/lib/agent-targets.test.ts index 2a2262c..c3eb136 100644 --- a/src/lib/agent-targets.test.ts +++ b/src/lib/agent-targets.test.ts @@ -1,13 +1,22 @@ import { readFileSync } from 'node:fs'; import { describe, expect, it } from 'vitest'; import { + DEFAULT_SKILLS, + GEMINI_MANAGED_SECTION_BEGIN, + GEMINI_MANAGED_SECTION_END, MANAGED_SECTION_BEGIN, MANAGED_SECTION_END, + ONBOARD_CODEX_LINE, SKILL_DESCRIPTION, SKILL_NAME, + SKILLS, TARGETS, + buildCodexAggregate, + codexContentFor, loadCodexSkillBody, loadSkillBody, + loadSkillBodyFor, + pathFor, renderForTarget, } from './agent-targets.js'; @@ -44,6 +53,13 @@ function parseFrontmatterDescription(content: string): string | undefined { const templateRaw = readFileSync('docs/cli-v1-agent-install/skill-template.md', 'utf8'); const templateDescription = parseFrontmatterDescription(templateRaw); +// Load onboard-skill-template.md from repo root. +const onboardTemplateRaw = readFileSync( + 'docs/cli-v1-agent-install/onboard-skill-template.md', + 'utf8', +); +const onboardTemplateDescription = parseFrontmatterDescription(onboardTemplateRaw); + // Stub body for unit tests that don't need the real file, so tests are fast // and deterministic regardless of asset path resolution. const STUB_BODY = `# TestSprite Verification Loop @@ -60,20 +76,21 @@ testsprite test artifact get --out ./out/ // --------------------------------------------------------------------------- describe('TARGETS', () => { - it('has all five required keys', () => { + it('has all six required keys', () => { const keys = Object.keys(TARGETS).sort(); - expect(keys).toEqual(['antigravity', 'claude', 'cline', 'codex', 'cursor']); + expect(keys).toEqual(['antigravity', 'claude', 'cline', 'codex', 'cursor', 'gemini']); }); it('claude is GA', () => { expect(TARGETS.claude.status).toBe('ga'); }); - it('cursor, cline, antigravity, and codex are experimental', () => { + it('cursor, cline, antigravity, codex, and gemini are experimental', () => { expect(TARGETS.cursor.status).toBe('experimental'); expect(TARGETS.cline.status).toBe('experimental'); expect(TARGETS.antigravity.status).toBe('experimental'); expect(TARGETS.codex.status).toBe('experimental'); + expect(TARGETS.gemini.status).toBe('experimental'); }); it('each target has a non-empty POSIX path', () => { @@ -90,13 +107,18 @@ describe('TARGETS', () => { expect(TARGETS.cline.mode).toBe('own-file'); }); - it('codex target has mode managed-section', () => { + it('root instruction targets have mode managed-section', () => { expect(TARGETS.codex.mode).toBe('managed-section'); + expect(TARGETS.gemini.mode).toBe('managed-section'); }); it('codex target path is AGENTS.md', () => { expect(TARGETS.codex.path).toBe('AGENTS.md'); }); + + it('gemini target path is GEMINI.md', () => { + expect(TARGETS.gemini.path).toBe('GEMINI.md'); + }); }); // --------------------------------------------------------------------------- @@ -151,7 +173,7 @@ describe('loadSkillBody', () => { // --------------------------------------------------------------------------- describe('renderForTarget("claude")', () => { - const result = renderForTarget('claude', STUB_BODY); + const result = renderForTarget('claude', 'testsprite-verify', STUB_BODY); it('returns the correct path', () => { expect(result.path).toBe('.claude/skills/testsprite-verify/SKILL.md'); @@ -171,7 +193,7 @@ describe('renderForTarget("claude")', () => { }); describe('renderForTarget("antigravity")', () => { - const result = renderForTarget('antigravity', STUB_BODY); + const result = renderForTarget('antigravity', 'testsprite-verify', STUB_BODY); it('returns the correct path', () => { expect(result.path).toBe('.agents/skills/testsprite-verify/SKILL.md'); @@ -188,8 +210,8 @@ describe('renderForTarget("antigravity")', () => { describe('renderForTarget("claude") vs renderForTarget("antigravity")', () => { it('produce the same frontmatter lines (name + description)', () => { - const claude = renderForTarget('claude', STUB_BODY); - const antigravity = renderForTarget('antigravity', STUB_BODY); + const claude = renderForTarget('claude', 'testsprite-verify', STUB_BODY); + const antigravity = renderForTarget('antigravity', 'testsprite-verify', STUB_BODY); // Extract the frontmatter block from each const extractFrontmatter = (content: string): string => { @@ -201,8 +223,8 @@ describe('renderForTarget("claude") vs renderForTarget("antigravity")', () => { }); it('differ only in their landing path', () => { - const claude = renderForTarget('claude', STUB_BODY); - const antigravity = renderForTarget('antigravity', STUB_BODY); + const claude = renderForTarget('claude', 'testsprite-verify', STUB_BODY); + const antigravity = renderForTarget('antigravity', 'testsprite-verify', STUB_BODY); expect(claude.path).not.toBe(antigravity.path); // Body content should be identical @@ -211,7 +233,7 @@ describe('renderForTarget("claude") vs renderForTarget("antigravity")', () => { }); describe('renderForTarget("cursor")', () => { - const result = renderForTarget('cursor', STUB_BODY); + const result = renderForTarget('cursor', 'testsprite-verify', STUB_BODY); it('returns the correct path', () => { expect(result.path).toBe('.cursor/rules/testsprite-verify.mdc'); @@ -240,7 +262,7 @@ describe('renderForTarget("cursor")', () => { }); describe('renderForTarget("cline")', () => { - const result = renderForTarget('cline', STUB_BODY); + const result = renderForTarget('cline', 'testsprite-verify', STUB_BODY); it('returns the correct path', () => { expect(result.path).toBe('.clinerules/testsprite-verify.md'); @@ -270,7 +292,7 @@ describe('content integrity — own-file targets', () => { // Use the real body for these checks, since we're guarding against trimming. for (const target of ownFileTargets) { describe(`target: ${target}`, () => { - const result = renderForTarget(target); + const result = renderForTarget(target, 'testsprite-verify'); it('contains the TestSprite Verification Loop H1', () => { // The skill body opens with the renamed H1. @@ -334,6 +356,13 @@ describe('MANAGED_SECTION sentinels', () => { expect(MANAGED_SECTION_BEGIN.toLowerCase()).toContain('testsprite'); expect(MANAGED_SECTION_END.toLowerCase()).toContain('testsprite'); }); + + it('gemini sentinels are distinct HTML comments', () => { + expect(GEMINI_MANAGED_SECTION_BEGIN.startsWith('')).toBe(true); + expect(GEMINI_MANAGED_SECTION_BEGIN).toContain('gemini'); + expect(GEMINI_MANAGED_SECTION_BEGIN).not.toBe(MANAGED_SECTION_BEGIN); + }); }); // --------------------------------------------------------------------------- @@ -359,14 +388,14 @@ describe('content integrity — codex target (testsprite-verify.codex.md)', () = it('renderForTarget("codex") path is AGENTS.md', () => { const STUB_CODEX_BODY = '# TestSprite Verification Loop\ntestsprite test run\n--wait\ntest artifact get\n'; - const result = renderForTarget('codex', STUB_CODEX_BODY); + const result = renderForTarget('codex', 'testsprite-verify', STUB_CODEX_BODY); expect(result.path).toBe('AGENTS.md'); }); it('renderForTarget("codex") content is the body unwrapped (no frontmatter)', () => { const STUB_CODEX_BODY = '# TestSprite Verification Loop\ntestsprite test run\n--wait\ntest artifact get\n'; - const result = renderForTarget('codex', STUB_CODEX_BODY); + const result = renderForTarget('codex', 'testsprite-verify', STUB_CODEX_BODY); // codex wrap is identity — no frontmatter fences expect(result.content).toBe(STUB_CODEX_BODY); expect(result.content).not.toContain('---'); @@ -374,9 +403,337 @@ describe('content integrity — codex target (testsprite-verify.codex.md)', () = it('renderForTarget("codex") without body arg uses codex asset (not full skill body)', () => { // The real codex asset is trimmed (no acronym line). - const result = renderForTarget('codex'); + const result = renderForTarget('codex', 'testsprite-verify'); // Plain Markdown; no frontmatter fences from own-file wraps expect(result.content).not.toContain('name: testsprite-verify'); expect(result.content).not.toContain('alwaysApply:'); }); }); + +// --------------------------------------------------------------------------- +// content integrity — gemini target +// --------------------------------------------------------------------------- + +describe('content integrity — gemini target (GEMINI.md managed-section body)', () => { + it('renderForTarget("gemini") path is GEMINI.md', () => { + const STUB_GEMINI_BODY = + '# TestSprite Verification Loop\ntestsprite test run\n--wait\ntest artifact get\n'; + const result = renderForTarget('gemini', 'testsprite-verify', STUB_GEMINI_BODY); + expect(result.path).toBe('GEMINI.md'); + }); + + it('renderForTarget("gemini") content is the body unwrapped (no frontmatter)', () => { + const STUB_GEMINI_BODY = + '# TestSprite Verification Loop\ntestsprite test run\n--wait\ntest artifact get\n'; + const result = renderForTarget('gemini', 'testsprite-verify', STUB_GEMINI_BODY); + expect(result.content).toBe(STUB_GEMINI_BODY); + expect(result.content).not.toContain('---'); + }); + + it('renderForTarget("gemini") without body arg uses the managed-section asset', () => { + const result = renderForTarget('gemini', 'testsprite-verify'); + expect(result.content).toContain('testsprite test run'); + expect(result.content).not.toContain('name: testsprite-verify'); + expect(result.content).not.toContain('alwaysApply:'); + }); +}); + +// --------------------------------------------------------------------------- +// SKILLS registry +// --------------------------------------------------------------------------- + +describe('SKILLS registry', () => { + it('has testsprite-verify key', () => { + expect(SKILLS['testsprite-verify']).toBeDefined(); + }); + + it('has testsprite-onboard key', () => { + expect(SKILLS['testsprite-onboard']).toBeDefined(); + }); + + it('testsprite-verify description is ≤ 1536 characters', () => { + expect(SKILLS['testsprite-verify']!.description.length).toBeLessThanOrEqual(1536); + }); + + it('testsprite-onboard description is ≤ 1536 characters', () => { + expect(SKILLS['testsprite-onboard']!.description.length).toBeLessThanOrEqual(1536); + }); + + it('testsprite-verify description is byte-identical to skill-template.md frontmatter description', () => { + expect(templateDescription).toBeDefined(); + expect(SKILLS['testsprite-verify']!.description).toBe(templateDescription); + }); + + it('testsprite-onboard description is byte-identical to onboard-skill-template.md frontmatter description', () => { + expect(onboardTemplateDescription).toBeDefined(); + expect(SKILLS['testsprite-onboard']!.description).toBe(onboardTemplateDescription); + }); + + it('testsprite-verify has bodyFile testsprite-verify.skill.md', () => { + expect(SKILLS['testsprite-verify']!.bodyFile).toBe('testsprite-verify.skill.md'); + }); + + it('testsprite-onboard has bodyFile testsprite-onboard.skill.md', () => { + expect(SKILLS['testsprite-onboard']!.bodyFile).toBe('testsprite-onboard.skill.md'); + }); + + it('testsprite-verify codex kind is full', () => { + const codex = SKILLS['testsprite-verify']!.codex; + expect(codex.kind).toBe('full'); + }); + + it('testsprite-onboard codex kind is line', () => { + const codex = SKILLS['testsprite-onboard']!.codex; + expect(codex.kind).toBe('line'); + }); +}); + +// --------------------------------------------------------------------------- +// DEFAULT_SKILLS +// --------------------------------------------------------------------------- + +describe('DEFAULT_SKILLS', () => { + it('equals ["testsprite-verify", "testsprite-onboard"]', () => { + expect(DEFAULT_SKILLS).toEqual(['testsprite-verify', 'testsprite-onboard']); + }); + + it('has exactly two entries', () => { + expect(DEFAULT_SKILLS.length).toBe(2); + }); +}); + +// --------------------------------------------------------------------------- +// pathFor +// --------------------------------------------------------------------------- + +describe('pathFor', () => { + it('claude + testsprite-verify', () => { + expect(pathFor('claude', 'testsprite-verify')).toBe( + '.claude/skills/testsprite-verify/SKILL.md', + ); + }); + + it('antigravity + testsprite-verify', () => { + expect(pathFor('antigravity', 'testsprite-verify')).toBe( + '.agents/skills/testsprite-verify/SKILL.md', + ); + }); + + it('cursor + testsprite-verify', () => { + expect(pathFor('cursor', 'testsprite-verify')).toBe('.cursor/rules/testsprite-verify.mdc'); + }); + + it('cline + testsprite-verify', () => { + expect(pathFor('cline', 'testsprite-verify')).toBe('.clinerules/testsprite-verify.md'); + }); + + it('codex + testsprite-verify', () => { + expect(pathFor('codex', 'testsprite-verify')).toBe('AGENTS.md'); + }); + + it('gemini + testsprite-verify', () => { + expect(pathFor('gemini', 'testsprite-verify')).toBe('GEMINI.md'); + }); + + it('claude + testsprite-onboard', () => { + expect(pathFor('claude', 'testsprite-onboard')).toBe( + '.claude/skills/testsprite-onboard/SKILL.md', + ); + }); + + it('antigravity + testsprite-onboard', () => { + expect(pathFor('antigravity', 'testsprite-onboard')).toBe( + '.agents/skills/testsprite-onboard/SKILL.md', + ); + }); + + it('cursor + testsprite-onboard', () => { + expect(pathFor('cursor', 'testsprite-onboard')).toBe('.cursor/rules/testsprite-onboard.mdc'); + }); + + it('cline + testsprite-onboard', () => { + expect(pathFor('cline', 'testsprite-onboard')).toBe('.clinerules/testsprite-onboard.md'); + }); + + it('codex + testsprite-onboard is AGENTS.md (shared)', () => { + expect(pathFor('codex', 'testsprite-onboard')).toBe('AGENTS.md'); + }); + + it('gemini + testsprite-onboard is GEMINI.md (shared)', () => { + expect(pathFor('gemini', 'testsprite-onboard')).toBe('GEMINI.md'); + }); + + it('TARGETS[t].path === pathFor(t, "testsprite-verify") for every target', () => { + for (const [target] of Object.entries(TARGETS)) { + expect(TARGETS[target as keyof typeof TARGETS].path).toBe( + pathFor(target as Parameters[0], 'testsprite-verify'), + ); + } + }); +}); + +// --------------------------------------------------------------------------- +// loadSkillBodyFor +// --------------------------------------------------------------------------- + +describe('loadSkillBodyFor', () => { + it('stub read returns the provided stub body for testsprite-verify', () => { + const body = loadSkillBodyFor('testsprite-verify', () => STUB_BODY); + expect(body).toBe(STUB_BODY); + }); + + it('stub read returns the provided stub body for testsprite-onboard', () => { + const ONBOARD_STUB = '# TestSprite: onboard a repo with a seed test suite\nStub.'; + const body = loadSkillBodyFor('testsprite-onboard', () => ONBOARD_STUB); + expect(body).toBe(ONBOARD_STUB); + }); + + it('real loadSkillBodyFor("testsprite-verify") starts with the verify H1', () => { + const body = loadSkillBodyFor('testsprite-verify'); + expect(body.trimStart().startsWith('# TestSprite Verification Loop')).toBe(true); + }); + + it('real loadSkillBodyFor("testsprite-onboard") contains the onboard H1', () => { + const body = loadSkillBodyFor('testsprite-onboard'); + expect(body).toContain('# TestSprite: onboard a repo with a seed test suite'); + }); + + it('unknown skill throws', () => { + expect(() => loadSkillBodyFor('testsprite-unknown')).toThrow('unknown skill'); + }); +}); + +// --------------------------------------------------------------------------- +// codexContentFor +// --------------------------------------------------------------------------- + +describe('codexContentFor', () => { + it('testsprite-verify (full) contains "testsprite test run"', () => { + const content = codexContentFor('testsprite-verify'); + expect(content).toContain('testsprite test run'); + }); + + it('testsprite-verify (full) contains "--wait"', () => { + const content = codexContentFor('testsprite-verify'); + expect(content).toContain('--wait'); + }); + + it('testsprite-onboard (line) equals ONBOARD_CODEX_LINE', () => { + const content = codexContentFor('testsprite-onboard'); + expect(content).toBe(ONBOARD_CODEX_LINE); + }); + + it('ONBOARD_CODEX_LINE starts with "**First-time setup:**"', () => { + expect(ONBOARD_CODEX_LINE.startsWith('**First-time setup:**')).toBe(true); + }); + + it('unknown skill throws', () => { + expect(() => codexContentFor('testsprite-unknown')).toThrow('unknown skill'); + }); + + it('testsprite-verify with stub read returns stub value', () => { + const content = codexContentFor('testsprite-verify', () => '# stub codex'); + expect(content).toBe('# stub codex'); + }); +}); + +// --------------------------------------------------------------------------- +// buildCodexAggregate +// --------------------------------------------------------------------------- + +describe('buildCodexAggregate', () => { + it('single verify is byte-identical to loadCodexSkillBody().trimEnd()', () => { + const aggregate = buildCodexAggregate(['testsprite-verify']); + expect(aggregate).toBe(loadCodexSkillBody().trimEnd()); + }); + + it('DEFAULT_SKILLS aggregate contains the verify H1', () => { + const aggregate = buildCodexAggregate(DEFAULT_SKILLS); + expect(aggregate).toContain('# TestSprite Verification Loop'); + }); + + it('DEFAULT_SKILLS aggregate contains the onboard line', () => { + const aggregate = buildCodexAggregate(DEFAULT_SKILLS); + expect(aggregate).toContain('**First-time setup:**'); + }); + + it('DEFAULT_SKILLS aggregate byte length is < 32768 (AGENTS.md budget)', () => { + const aggregate = buildCodexAggregate(DEFAULT_SKILLS); + expect(Buffer.byteLength(aggregate, 'utf8')).toBeLessThan(32768); + }); + + it('empty skills list returns empty string', () => { + const aggregate = buildCodexAggregate([]); + expect(aggregate).toBe(''); + }); + + it('DEFAULT_SKILLS aggregate with stub read joins both contributions', () => { + const stubRead = () => '# Verify stub'; + const aggregate = buildCodexAggregate(DEFAULT_SKILLS, stubRead); + // verify contributes the stub read result, onboard contributes its inline line + expect(aggregate).toContain('# Verify stub'); + expect(aggregate).toContain('**First-time setup:**'); + }); +}); + +// --------------------------------------------------------------------------- +// renderForTarget — onboard skill +// --------------------------------------------------------------------------- + +describe('renderForTarget for testsprite-onboard', () => { + it('claude path is .claude/skills/testsprite-onboard/SKILL.md', () => { + const result = renderForTarget('claude', 'testsprite-onboard'); + expect(result.path).toBe('.claude/skills/testsprite-onboard/SKILL.md'); + }); + + it('claude frontmatter contains name: testsprite-onboard', () => { + const result = renderForTarget('claude', 'testsprite-onboard'); + expect(result.content).toContain('name: testsprite-onboard'); + }); + + it('claude body contains onboard H1', () => { + const result = renderForTarget('claude', 'testsprite-onboard'); + expect(result.content).toContain('# TestSprite: onboard a repo with a seed test suite'); + }); + + it('cursor onboard path is .cursor/rules/testsprite-onboard.mdc', () => { + const result = renderForTarget('cursor', 'testsprite-onboard'); + expect(result.path).toBe('.cursor/rules/testsprite-onboard.mdc'); + }); + + it('cursor onboard frontmatter has alwaysApply: false', () => { + const result = renderForTarget('cursor', 'testsprite-onboard'); + expect(result.content).toContain('alwaysApply: false'); + }); + + it('cline onboard path is .clinerules/testsprite-onboard.md', () => { + const result = renderForTarget('cline', 'testsprite-onboard'); + expect(result.path).toBe('.clinerules/testsprite-onboard.md'); + }); + + it('cline onboard has no frontmatter fence', () => { + const result = renderForTarget('cline', 'testsprite-onboard'); + expect(result.content).not.toContain('---'); + }); + + it('codex onboard renders the unwrapped ONBOARD_CODEX_LINE', () => { + const result = renderForTarget('codex', 'testsprite-onboard'); + expect(result.content).toBe(ONBOARD_CODEX_LINE); + expect(result.content).not.toContain('---'); + }); + + it('codex onboard path is AGENTS.md', () => { + const result = renderForTarget('codex', 'testsprite-onboard'); + expect(result.path).toBe('AGENTS.md'); + }); + + it('gemini onboard renders unwrapped content into GEMINI.md', () => { + const result = renderForTarget('gemini', 'testsprite-onboard'); + expect(result.path).toBe('GEMINI.md'); + expect(result.content).toBe(ONBOARD_CODEX_LINE); + }); + + it('unknown skill throws for renderForTarget', () => { + expect(() => renderForTarget('claude', 'testsprite-unknown')).toThrow('unknown skill'); + }); +}); diff --git a/src/lib/agent-targets.ts b/src/lib/agent-targets.ts index fb83ca2..947fd47 100644 --- a/src/lib/agent-targets.ts +++ b/src/lib/agent-targets.ts @@ -1,62 +1,196 @@ import { readFileSync } from 'node:fs'; -export type AgentTarget = 'claude' | 'cursor' | 'cline' | 'antigravity' | 'codex'; +export type AgentTarget = 'claude' | 'cursor' | 'cline' | 'antigravity' | 'codex' | 'gemini'; + +export interface ManagedSectionSpec { + begin: string; + end: string; + /** Optional documented load budget for root instruction files such as AGENTS.md. */ + loadBudgetBytes?: number; + loadBudgetLabel?: string; +} export interface TargetSpec { status: 'ga' | 'experimental'; - /** repo-relative landing path, POSIX separators */ + /** + * Repo-relative landing path for the CANONICAL skill (`testsprite-verify`), + * POSIX separators. Kept for back-compat: `skill-nudge.ts` reads this to detect + * a verify install, and `agent list`/tests reference it. For any skill, derive + * the real path via {@link pathFor} — this field is `pathFor(target, SKILL_NAME)`. + */ path: string; /** - * 'own-file': the CLI owns the whole file (existing 4 targets). + * 'own-file': the CLI owns the whole file (claude/cursor/cline/antigravity). * 'managed-section': the CLI writes only a sentinel-delimited section inside - * a potentially user-authored file (codex target, AGENTS.md). + * a potentially user-authored root instruction file. */ mode: 'own-file' | 'managed-section'; - /** wrap the canonical body in this target's frontmatter/header */ - wrap(body: string): string; + /** + * Wrap a skill body in this target's frontmatter/header. Takes the skill's + * `name`+`description` (own-file targets emit them as frontmatter) and the body. + * No-op for cline (body verbatim) and managed-section targets (plain + * Markdown with no frontmatter). + */ + wrap(name: string, description: string, body: string): string; + /** Sentinel and budget metadata for managed-section targets. */ + managedSection?: ManagedSectionSpec; } +// --------------------------------------------------------------------------- +// Skill registry +// --------------------------------------------------------------------------- + +/** + * How a skill contributes to the codex target's always-on `AGENTS.md` section. + * + * - 'full': inject the skill's trimmed codex body (a `*.codex.md` asset). Used by + * `testsprite-verify` (~6 KiB). + * - 'line': inject a single short line authored inline here. Used by + * `testsprite-onboard` — the full 6-step flow doesn't belong in an always-on, + * 32 KiB-budgeted file, but a one-line signal does. + * - 'none': skill is not represented in AGENTS.md at all (reserved). + */ +export type CodexContribution = + | { kind: 'full'; file: string } + | { kind: 'line'; text: string } + | { kind: 'none' }; + +export interface SkillSpec { + /** Skill name — appears in own-file frontmatter and the landing path. */ + name: string; + /** ≤1536 chars (claude description cap). Byte-identical to its template doc. */ + description: string; + /** Own-file body asset basename under `skills/`, e.g. 'testsprite-verify.skill.md'. */ + bodyFile: string; + /** How this skill contributes to the codex AGENTS.md managed section. */ + codex: CodexContribution; +} + +/** + * `testsprite-onboard` codex contribution — a single always-on line. Kept here + * (not in a `*.codex.md` asset) because it is one line; see {@link CodexContribution}. + */ +export const ONBOARD_CODEX_LINE = + '**First-time setup:** if this repo has no TestSprite tests yet, seed a *broad* first suite across its main user flows — not just one test — each with a concrete, observable assertion, before reporting setup as done.'; + +/** + * The skill registry. Each entry owns its name, description (drift-guarded by a + * byte-identity unit test against a template doc), own-file body asset, and codex + * contribution. `agent install` / `setup` install {@link DEFAULT_SKILLS}; the + * codex target aggregates every installed skill's codex contribution into ONE + * AGENTS.md section. + */ +export const SKILLS: Record = { + 'testsprite-verify': { + name: 'testsprite-verify', + description: + 'TestSprite verification loop — after finishing a feature or fix in a TestSprite-tested repo, use the `testsprite` CLI to run the relevant TestSprite tests against the change and inspect any failure artifacts before reporting the work as done. Use whenever code has changed outside docs/config and is about to be reported complete — by running an existing test that covers the change, or by creating a new TestSprite test (a frontend plan, or a backend Python assertion) and running it to a terminal verdict.', + bodyFile: 'testsprite-verify.skill.md', + codex: { kind: 'full', file: 'testsprite-verify.codex.md' }, + }, + 'testsprite-onboard': { + name: 'testsprite-onboard', + description: + 'Stand up a complete, runnable TestSprite test suite for the current repo at first use — create a project (with a target URL and auth), derive a coherent set of tests from the codebase, batch-create them, and smoke-run a few to a green verdict so the user immediately has something worth running. Use ONLY when a repo has no TestSprite tests yet (a fresh project), right after `testsprite setup`, or when the user asks to "set up / bootstrap / seed tests". This is first-run setup, NOT change verification — once a project already has tests, use the testsprite-verify skill instead.', + bodyFile: 'testsprite-onboard.skill.md', + codex: { kind: 'line', text: ONBOARD_CODEX_LINE }, + }, +}; + +/** + * Skills installed by `setup` and by `agent install` when no `--skill` subset is + * given. Order is significant for the codex aggregate (verify first, then the + * onboard line as a short addendum). + */ +export const DEFAULT_SKILLS = ['testsprite-verify', 'testsprite-onboard'] as const; + +// --------------------------------------------------------------------------- +// Back-compat single-skill exports (= the canonical `testsprite-verify` skill) +// --------------------------------------------------------------------------- + +/** @deprecated The canonical skill name. New code: iterate {@link SKILLS}. */ export const SKILL_NAME = 'testsprite-verify'; /** - * Mirrors skill-template.md frontmatter `description`. ≤1536 chars (claude cap). - * A unit test asserts byte-identity with the template file. + * @deprecated The canonical skill's description. New code: + * `SKILLS['testsprite-verify'].description`. Kept so existing importers and the + * byte-identity unit test keep working. */ -export const SKILL_DESCRIPTION = - 'TestSprite verification loop — after finishing a feature or fix in a TestSprite-tested repo, use the `testsprite` CLI to run the relevant TestSprite tests against the change and inspect any failure artifacts before reporting the work as done. Use whenever code has changed outside docs/config and is about to be reported complete — by running an existing test that covers the change, or by creating a new TestSprite test (a frontend plan, or a backend Python assertion) and running it to a terminal verdict.'; +export const SKILL_DESCRIPTION = SKILLS['testsprite-verify']!.description; + +// --------------------------------------------------------------------------- +// Wrappers +// --------------------------------------------------------------------------- + +function wrapSkill(name: string, description: string, body: string): string { + return `---\nname: ${name}\ndescription: ${description}\n---\n\n${body}\n`; +} -function wrapSkill(body: string): string { - return `---\nname: ${SKILL_NAME}\ndescription: ${SKILL_DESCRIPTION}\n---\n\n${body}\n`; +function wrapMdc(_name: string, description: string, body: string): string { + return `---\ndescription: ${description}\nalwaysApply: false\n---\n\n${body}\n`; } -function wrapMdc(body: string): string { - return `---\ndescription: ${SKILL_DESCRIPTION}\nalwaysApply: false\n---\n\n${body}\n`; +// --------------------------------------------------------------------------- +// Landing paths +// --------------------------------------------------------------------------- + +/** + * Repo-relative landing path for a given skill on a given target (POSIX + * separators). Own-file targets embed the skill name in the path so multiple + * skills coexist; the codex target always lands at the single shared `AGENTS.md` + * (every skill's codex contribution is merged into one managed section there). + */ +export function pathFor(target: AgentTarget, skill: string): string { + switch (target) { + case 'claude': + return `.claude/skills/${skill}/SKILL.md`; + case 'antigravity': + return `.agents/skills/${skill}/SKILL.md`; + case 'cursor': + return `.cursor/rules/${skill}.mdc`; + case 'cline': + return `.clinerules/${skill}.md`; + case 'codex': + return 'AGENTS.md'; + case 'gemini': + return 'GEMINI.md'; + } } +/** Sentinel pair that bounds our managed section in AGENTS.md. */ +export const MANAGED_SECTION_BEGIN = + ''; +export const MANAGED_SECTION_END = ''; + +/** Sentinel pair that bounds our managed section in GEMINI.md. */ +export const GEMINI_MANAGED_SECTION_BEGIN = + ''; +export const GEMINI_MANAGED_SECTION_END = ''; + export const TARGETS: Record = { claude: { status: 'ga', - path: '.claude/skills/testsprite-verify/SKILL.md', + path: pathFor('claude', SKILL_NAME), mode: 'own-file', wrap: wrapSkill, }, antigravity: { status: 'experimental', - path: '.agents/skills/testsprite-verify/SKILL.md', + path: pathFor('antigravity', SKILL_NAME), mode: 'own-file', wrap: wrapSkill, }, cursor: { status: 'experimental', - path: '.cursor/rules/testsprite-verify.mdc', + path: pathFor('cursor', SKILL_NAME), mode: 'own-file', wrap: wrapMdc, }, cline: { status: 'experimental', - path: '.clinerules/testsprite-verify.md', + path: pathFor('cline', SKILL_NAME), mode: 'own-file', - wrap: body => body, + wrap: (_name, _description, body) => body, }, /** * codex target — managed-section mode. @@ -65,7 +199,9 @@ export const TARGETS: Record = { * for the whole file). Unlike own-file targets, we must NOT clobber a user's * existing AGENTS.md: we write only a sentinel-delimited section so other * project instructions coexist. The sentinel pair is the canonical identity - * marker; the content between them is ours to replace. + * marker; the content between them is ours to replace. EVERY installed skill's + * codex contribution is aggregated into this one section (see + * {@link buildCodexAggregate}). * * --force with managed-section: replaces the section unconditionally but * NEVER destroys content outside the sentinels. No whole-file .bak is written @@ -74,58 +210,134 @@ export const TARGETS: Record = { */ codex: { status: 'experimental', - path: 'AGENTS.md', + path: pathFor('codex', SKILL_NAME), mode: 'managed-section', // wrap is a no-op for managed-section — content is authored as plain Markdown // with no frontmatter (AGENTS.md is plain prose, not a skill schema). - wrap: body => body, + wrap: (_name, _description, body) => body, + managedSection: { + begin: MANAGED_SECTION_BEGIN, + end: MANAGED_SECTION_END, + loadBudgetBytes: 32768, + loadBudgetLabel: 'Codex may not load content beyond its 32 KiB budget', + }, + }, + /** + * gemini target — managed-section mode. + * + * Gemini CLI reads GEMINI.md as the project instruction file. Like AGENTS.md, + * it is commonly user-authored project context, so we merge a sentinel-delimited + * section rather than owning the whole file. + */ + gemini: { + status: 'experimental', + path: pathFor('gemini', SKILL_NAME), + mode: 'managed-section', + wrap: (_name, _description, body) => body, + managedSection: { + begin: GEMINI_MANAGED_SECTION_BEGIN, + end: GEMINI_MANAGED_SECTION_END, + }, }, }; -/** Sentinel pair that bounds our managed section in AGENTS.md. */ -export const MANAGED_SECTION_BEGIN = - ''; -export const MANAGED_SECTION_END = ''; - type ReadFn = (url: URL) => string; const defaultRead: ReadFn = (url: URL) => readFileSync(url, 'utf8'); +// --------------------------------------------------------------------------- +// Asset loaders +// --------------------------------------------------------------------------- + /** - * Load the canonical skill body. `../../skills/...` resolves to the repo-root + * Resolve a `skills/` asset. `../../skills/...` resolves to the repo-root * `skills/` directory in BOTH source (vitest: `src/lib/` → `../../skills`) and * the built/published package (`dist/lib/` → `../../skills` = package root). The * directory ships verbatim via package.json `files`, so no build-time copy step - * is needed (unlike the old `src/assets` → `dist/assets` mirror). - * - * Injectable `read` fn keeps unit tests off disk. + * is needed. Injectable `read` keeps unit tests off disk. + */ +function readSkillAsset(file: string, read: ReadFn): string { + return read(new URL(`../../skills/${file}`, import.meta.url)); +} + +/** Load a skill's own-file body by skill name (frontmatter is added by `wrap`). */ +export function loadSkillBodyFor(skill: string, read: ReadFn = defaultRead): string { + const spec = SKILLS[skill]; + if (!spec) throw new Error(`unknown skill: ${skill}`); + return readSkillAsset(spec.bodyFile, read); +} + +/** + * Resolve a skill's codex (AGENTS.md) contribution as a Markdown string. + * 'full' → read the `*.codex.md` asset; 'line' → the inline one-liner; 'none' → ''. + */ +export function codexContentFor(skill: string, read: ReadFn = defaultRead): string { + const spec = SKILLS[skill]; + if (!spec) throw new Error(`unknown skill: ${skill}`); + const c = spec.codex; + if (c.kind === 'full') return readSkillAsset(c.file, read); + if (c.kind === 'line') return c.text; + return ''; +} + +/** + * Compose the codex managed-section BODY (sans sentinels) from several skills: + * each skill's codex contribution, trimmed, joined by a blank line, in the given + * order. A single `['testsprite-verify']` aggregate is byte-identical to the old + * single-skill codex body, so existing AGENTS.md installs round-trip unchanged. + */ +export function buildCodexAggregate(skills: readonly string[], read: ReadFn = defaultRead): string { + return skills + .map(s => codexContentFor(s, read).trimEnd()) + .filter(Boolean) + .join('\n\n'); +} + +/** + * Back-compat: the canonical verify skill body (own-file). Kept so existing + * importers and the `loadSkillBody(read)` unit-test signature keep working. + * @deprecated Use {@link loadSkillBodyFor}. */ export function loadSkillBody(read: ReadFn = defaultRead): string { - return read(new URL('../../skills/testsprite-verify.skill.md', import.meta.url)); + return loadSkillBodyFor(SKILL_NAME, read); } /** - * Load the trimmed codex skill body (plain Markdown, no frontmatter). - * Designed for AGENTS.md managed-section injection. + * Back-compat: the canonical verify skill's trimmed codex body. Kept so existing + * importers and the `loadCodexSkillBody(read)` unit-test signature keep working. + * @deprecated Use {@link codexContentFor}. */ export function loadCodexSkillBody(read: ReadFn = defaultRead): string { - return read(new URL('../../skills/testsprite-verify.codex.md', import.meta.url)); + return codexContentFor(SKILL_NAME, read); } +// --------------------------------------------------------------------------- +// renderForTarget +// --------------------------------------------------------------------------- + /** - * Convenience for piece-2: returns the exact bytes to write for a target. + * The exact bytes to write for one skill on one target. * - * For own-file targets, `body` defaults to the full skill body. - * For the codex managed-section target, the trimmed codex body is used instead — - * pass an explicit `body` to override in tests. + * - own-file targets: `body` defaults to the skill's own-file asset, wrapped in + * the target's frontmatter/header. + * - codex (managed-section): returns the skill's codex contribution unwrapped + * (plain Markdown, no frontmatter). The real install does NOT call this for + * codex — it aggregates all skills via {@link buildCodexAggregate} — but it is + * kept single-skill here for tests and parity. Pass an explicit `body` to override. */ -export function renderForTarget(t: AgentTarget, body?: string): { path: string; content: string } { +export function renderForTarget( + t: AgentTarget, + skill: string, + body?: string, +): { path: string; content: string } { const spec = TARGETS[t]; - const resolvedBody = - body !== undefined - ? body - : spec.mode === 'managed-section' - ? loadCodexSkillBody() - : loadSkillBody(); - return { path: spec.path, content: spec.wrap(resolvedBody) }; + const skillSpec = SKILLS[skill]; + if (!skillSpec) throw new Error(`unknown skill: ${skill}`); + const path = pathFor(t, skill); + if (spec.mode === 'managed-section') { + const resolvedBody = body !== undefined ? body : codexContentFor(skill); + return { path, content: spec.wrap(skillSpec.name, skillSpec.description, resolvedBody) }; + } + const resolvedBody = body !== undefined ? body : loadSkillBodyFor(skill); + return { path, content: spec.wrap(skillSpec.name, skillSpec.description, resolvedBody) }; } diff --git a/src/lib/skill-nudge.test.ts b/src/lib/skill-nudge.test.ts index 34f043b..12b9297 100644 --- a/src/lib/skill-nudge.test.ts +++ b/src/lib/skill-nudge.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest'; -import { MANAGED_SECTION_BEGIN, TARGETS } from './agent-targets.js'; +import { GEMINI_MANAGED_SECTION_BEGIN, MANAGED_SECTION_BEGIN, TARGETS } from './agent-targets.js'; import type { OutputMode } from './output.js'; import { SKILL_NUDGE_COMMANDS, @@ -40,12 +40,24 @@ describe('isVerifySkillInstalled', () => { expect(isVerifySkillInstalled('/proj', { existsSync, readFileSync })).toBe(true); }); + it('true when GEMINI.md exists AND carries the Gemini BEGIN sentinel', () => { + const existsSync = (p: string) => p.endsWith('GEMINI.md'); + const readFileSync = () => `# project\n${GEMINI_MANAGED_SECTION_BEGIN}\n...skill...\n`; + expect(isVerifySkillInstalled('/proj', { existsSync, readFileSync })).toBe(true); + }); + it('false when only a bare AGENTS.md (no sentinel) exists', () => { const existsSync = (p: string) => p.endsWith('AGENTS.md'); const readFileSync = () => '# my project\nNothing TestSprite here.\n'; expect(isVerifySkillInstalled('/proj', { existsSync, readFileSync })).toBe(false); }); + it('false when only a bare GEMINI.md (no sentinel) exists', () => { + const existsSync = (p: string) => p.endsWith('GEMINI.md'); + const readFileSync = () => '# my project\nNothing TestSprite here.\n'; + expect(isVerifySkillInstalled('/proj', { existsSync, readFileSync })).toBe(false); + }); + it('false when an unreadable AGENTS.md is the only candidate', () => { const existsSync = (p: string) => p.endsWith('AGENTS.md'); const readFileSync = () => { diff --git a/src/lib/skill-nudge.ts b/src/lib/skill-nudge.ts index 1afc863..a89c929 100644 --- a/src/lib/skill-nudge.ts +++ b/src/lib/skill-nudge.ts @@ -1,6 +1,6 @@ import { existsSync, readFileSync } from 'node:fs'; import { join } from 'node:path'; -import { MANAGED_SECTION_BEGIN, TARGETS } from './agent-targets.js'; +import { TARGETS } from './agent-targets.js'; import { defaultCredentialsPath, readProfile } from './credentials.js'; import type { OutputMode } from './output.js'; @@ -43,9 +43,9 @@ export interface SkillPresenceDeps { /** * True if the `testsprite-verify` skill is installed for ANY supported agent in * `dir`. own-file targets (claude/cursor/cline/antigravity): the landing file - * exists. managed-section target (codex / AGENTS.md): the file exists AND - * carries our BEGIN sentinel — a user-authored AGENTS.md without the sentinel - * does NOT count as our skill. + * exists. managed-section targets (root instruction files): the file exists + * AND carries that target's BEGIN sentinel — a user-authored instruction file + * without the sentinel does NOT count as our skill. * * The TARGETS table is the single source of truth for landing paths, so this * stays in lockstep with `agent install` without re-listing paths. Best-effort: @@ -59,9 +59,11 @@ export function isVerifySkillInstalled(dir: string, deps: SkillPresenceDeps = {} if (!exists(full)) continue; if (spec.mode === 'managed-section') { try { - if (read(full).includes(MANAGED_SECTION_BEGIN)) return true; + if (spec.managedSection && read(full).includes(spec.managedSection.begin)) { + return true; + } } catch { - // unreadable AGENTS.md → treat this target as absent, keep checking + // unreadable instruction file → treat this target as absent, keep checking } continue; } diff --git a/test/__snapshots__/help.snapshot.test.ts.snap b/test/__snapshots__/help.snapshot.test.ts.snap index b4ea59b..a6057bb 100644 --- a/test/__snapshots__/help.snapshot.test.ts.snap +++ b/test/__snapshots__/help.snapshot.test.ts.snap @@ -4,16 +4,16 @@ exports[`--help snapshots > agent 1`] = ` "Usage: testsprite agent [options] [command] Install TestSprite guidance into coding-agent config (Claude Code, Cursor, -Cline, Antigravity, Codex) +Cline, Antigravity, Codex, Gemini CLI) Options: -h, --help display help for command Commands: - install [options] Write the TestSprite verification-loop skill file into a - project for a coding agent - list List supported agent targets, their status, and landing - paths + install [options] Write the TestSprite agent skills (verification loop + + first-run onboarding) into a project for a coding agent + list List supported agent targets and skills, their status, and + landing paths help [command] display help for command " `; @@ -21,18 +21,20 @@ Commands: exports[`--help snapshots > agent install 1`] = ` "Usage: testsprite agent install [options] -Write the TestSprite verification-loop skill file into a project for a coding -agent +Write the TestSprite agent skills (verification loop + first-run onboarding) +into a project for a coding agent Options: - --target Agent target(s): claude, cursor, cline, antigravity, codex - (comma-separated or repeated) (default: []) - --dir Project root to write into (default: cwd) - --force For own-file targets: overwrite existing file (a .bak backup is - kept). For codex (managed-section): replaces the section - unconditionally; user content outside the section is never - destroyed. - -h, --help display help for command + --target Agent target(s): claude, cursor, cline, antigravity, codex, + gemini (comma-separated or repeated) (default: []) + --skill Skill(s) to install: testsprite-verify, testsprite-onboard + (comma-separated or repeated; default: all) (default: []) + --dir Project root to write into (default: cwd) + --force For own-file targets: overwrite existing file (a .bak backup + is kept). For managed-section targets: replaces the section + unconditionally; user content outside the section is never + destroyed. + -h, --help display help for command Global options (--dry-run, --output, --profile, --endpoint-url, --verbose, --debug): testsprite --help @@ -42,7 +44,7 @@ Global options (--dry-run, --output, --profile, --endpoint-url, --verbose, --deb exports[`--help snapshots > agent list 1`] = ` "Usage: testsprite agent list [options] -List supported agent targets, their status, and landing paths +List supported agent targets and skills, their status, and landing paths Options: -h, --help display help for command @@ -110,7 +112,8 @@ Options: --from-env Read TESTSPRITE_API_KEY from the environment instead of prompting (default: false) --agent Coding-agent target to install: claude, antigravity, - cursor, cline, codex (default: claude) (default: "claude") + cursor, cline, codex, gemini (default: claude) (default: + "claude") --no-agent Skip the agent skill install (configure credentials only) --force Overwrite an existing skill file (a .bak backup is kept) --dir Project root for the skill install (default: current @@ -573,14 +576,14 @@ Options: Commands: setup [options] Set up TestSprite: configure your API key and - install the verification skill for your coding - agent + install the TestSprite agent skills for your + coding agent auth Manage TestSprite credentials project Manage TestSprite projects test Inspect TestSprite tests agent Install TestSprite guidance into coding-agent config (Claude Code, Cursor, Cline, Antigravity, - Codex) + Codex, Gemini CLI) usage|credits Show credit balance and plan/entitlement info (proactive pre-flight before a large test run) help [command] display help for command diff --git a/test/e2e/agent-install.e2e.test.ts b/test/e2e/agent-install.e2e.test.ts index c32c8bc..18b659c 100644 --- a/test/e2e/agent-install.e2e.test.ts +++ b/test/e2e/agent-install.e2e.test.ts @@ -19,6 +19,9 @@ import { MANAGED_SECTION_BEGIN, MANAGED_SECTION_END, TARGETS, + SKILLS, + DEFAULT_SKILLS, + pathFor, renderForTarget, type AgentTarget, } from '../../src/lib/agent-targets.js'; @@ -88,7 +91,7 @@ describe('fresh install (per target)', () => { const allTargets = Object.keys(TARGETS) as AgentTarget[]; for (const target of allTargets) { - it(`installs ${target} → exit 0, file at matrix path, action: written/section-installed`, () => { + it(`installs ${target} → exit 0, all skill files land, action: written/section-installed`, () => { const tmpDir = freshTmpDir(); const result = runCli([ 'agent', @@ -106,23 +109,31 @@ describe('fresh install (per target)', () => { target: string; path: string; action: string; + skills: string[]; }>; expect(Array.isArray(parsed), 'output should be a JSON array').toBe(true); - const entry = parsed.find(r => r.target === target); - expect(entry, `entry for ${target}`).toBeDefined(); - // managed-section targets report 'section-installed'; own-file targets report 'written' - const expectedAction = - TARGETS[target].mode === 'managed-section' ? 'section-installed' : 'written'; - expect(entry!.action, `action for ${target}`).toBe(expectedAction); - expect(entry!.path).toBe(TARGETS[target].path); - - // File must exist at the matrix path - const expectedAbsPath = join(tmpDir, TARGETS[target].path); - expect(existsSync(expectedAbsPath), `file at ${expectedAbsPath}`).toBe(true); - - // Parent dirs must have been created - expect(existsSync(dirname(expectedAbsPath))).toBe(true); + if (TARGETS[target].mode === 'managed-section') { + // codex: ONE result aggregating all skills + const entry = parsed.find(r => r.target === target); + expect(entry, `entry for ${target}`).toBeDefined(); + expect(entry!.action, `action for ${target}`).toBe('section-installed'); + expect(entry!.path).toBe(TARGETS[target].path); + const absPath = join(tmpDir, TARGETS[target].path); + expect(existsSync(absPath), `file at ${absPath}`).toBe(true); + expect(existsSync(dirname(absPath))).toBe(true); + } else { + // own-file: one result per skill, one file per skill + for (const skill of DEFAULT_SKILLS) { + const entry = parsed.find(r => r.target === target && r.path === pathFor(target, skill)); + expect(entry, `entry for ${target}/${skill}`).toBeDefined(); + expect(entry!.action, `action for ${target}/${skill}`).toBe('written'); + + const absPath = join(tmpDir, pathFor(target, skill)); + expect(existsSync(absPath), `file at ${absPath}`).toBe(true); + expect(existsSync(dirname(absPath))).toBe(true); + } + } }); } }); @@ -132,76 +143,105 @@ describe('fresh install (per target)', () => { // --------------------------------------------------------------------------- describe('content integrity', () => { - // own-file targets: full skill body with frontmatter + // own-file targets: both skill files land with correct structure const ownFileTargets = (Object.keys(TARGETS) as AgentTarget[]).filter( t => TARGETS[t].mode === 'own-file', ); for (const target of ownFileTargets) { - it(`${target} file has correct structure and load-bearing strings`, () => { + it(`${target} testsprite-verify file has correct structure and load-bearing strings`, () => { const tmpDir = freshTmpDir(); runCli(['agent', 'install', `--target=${target}`, '--dir', tmpDir, '--output', 'json']); - const filePath = join(tmpDir, TARGETS[target].path); + const filePath = join(tmpDir, pathFor(target, 'testsprite-verify')); const content = readFileSync(filePath, 'utf8'); // (a) Frontmatter check if (target === 'claude' || target === 'antigravity') { - // Must start with --- (SKILL.md frontmatter) expect(content.startsWith('---'), `${target}: should start with ---`).toBe(true); expect(content).toContain('name: testsprite-verify'); } else if (target === 'cursor') { expect(content.startsWith('---'), `cursor: should start with ---`).toBe(true); expect(content).toContain('alwaysApply: false'); } else if (target === 'cline') { - // cline has NO frontmatter fence expect(content.startsWith('---'), `cline: must NOT start with ---`).toBe(false); - // cline body starts with the skill heading expect( content.trimStart().startsWith('#'), `cline: should start with a markdown heading`, ).toBe(true); } - // (b) branding — the renamed H1 must be present. - + // (b) branding — the renamed H1 must be present expect(content).toContain('TestSprite Verification Loop'); - // Match the verification-loop intro line used in the asset expect(content).toContain('The verification loop that flies'); - // (c) Load-bearing command strings — a body trim that drops these must fail CI + // (c) Load-bearing command strings expect(content, `${target}: missing 'testsprite test run'`).toContain('testsprite test run'); expect(content, `${target}: missing '--wait'`).toContain('--wait'); expect(content, `${target}: missing 'test artifact get'`).toContain('test artifact get'); }); + + it(`${target} testsprite-onboard file lands and has correct structure`, () => { + const tmpDir = freshTmpDir(); + runCli(['agent', 'install', `--target=${target}`, '--dir', tmpDir, '--output', 'json']); + + const filePath = join(tmpDir, pathFor(target, 'testsprite-onboard')); + expect(existsSync(filePath), `onboard file must exist at ${filePath}`).toBe(true); + const content = readFileSync(filePath, 'utf8'); + + // Frontmatter check for onboard skill + if (target === 'claude' || target === 'antigravity') { + expect(content.startsWith('---'), `${target}/onboard: should start with ---`).toBe(true); + expect(content).toContain('name: testsprite-onboard'); + } else if (target === 'cursor') { + expect(content.startsWith('---'), `cursor/onboard: should start with ---`).toBe(true); + expect(content).toContain('alwaysApply: false'); + } else if (target === 'cline') { + expect(content.startsWith('---'), `cline/onboard: must NOT start with ---`).toBe(false); + } + + // Load-bearing onboard string: the skill body must reference setup + expect(content).toContain('testsprite'); + }); } - // codex: managed-section target — trimmed body, sentinels wrapping content, no frontmatter - it('codex AGENTS.md contains sentinels and load-bearing command strings', () => { + // codex: ONE managed section containing BOTH verify body and onboard one-liner + it('codex AGENTS.md contains ONE managed section with both verify and onboard content', () => { const tmpDir = freshTmpDir(); runCli(['agent', 'install', '--target=codex', '--dir', tmpDir, '--output', 'json']); const filePath = join(tmpDir, TARGETS.codex.path); const content = readFileSync(filePath, 'utf8'); - // (a) Sentinels must be present - expect(content).toContain(MANAGED_SECTION_BEGIN); - expect(content).toContain(MANAGED_SECTION_END); + // (a) Exactly ONE pair of sentinels + const beginCount = ( + content.match( + new RegExp(MANAGED_SECTION_BEGIN.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g'), + ) ?? [] + ).length; + const endCount = ( + content.match(new RegExp(MANAGED_SECTION_END.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g')) ?? + [] + ).length; + expect(beginCount, 'exactly one BEGIN sentinel').toBe(1); + expect(endCount, 'exactly one END sentinel').toBe(1); + // BEGIN must come before END expect(content.indexOf(MANAGED_SECTION_BEGIN)).toBeLessThan( content.indexOf(MANAGED_SECTION_END), ); - // (b) branding (trimmed asset shares the renamed H1) + // (b) Verify content: branding heading + load-bearing command strings expect(content).toContain('TestSprite Verification Loop'); - - // (c) No frontmatter fence — AGENTS.md is plain prose - expect(content.startsWith('---'), 'codex: must NOT start with ---').toBe(false); - - // (d) Load-bearing command strings expect(content).toContain('testsprite test run'); expect(content).toContain('--wait'); expect(content).toContain('test artifact get'); + + // (c) Onboard content: the one-liner must be inside the managed section + expect(content).toContain('First-time setup'); + + // (d) No frontmatter fence — AGENTS.md is plain prose + expect(content.startsWith('---'), 'codex: must NOT start with ---').toBe(false); }); }); @@ -210,7 +250,7 @@ describe('content integrity', () => { // --------------------------------------------------------------------------- describe('idempotent re-run', () => { - it('second install exits 0 with action: skipped and file is byte-identical', () => { + it('second install exits 0 with all actions: skipped, files byte-identical', () => { const tmpDir = freshTmpDir(); // First install @@ -224,12 +264,15 @@ describe('idempotent re-run', () => { 'json', ]); expect(first.status).toBe(0); - const firstParsed = JSON.parse(first.stdout) as Array<{ action: string }>; - expect(firstParsed[0]!.action).toBe('written'); + const firstParsed = JSON.parse(first.stdout) as Array<{ path: string; action: string }>; + // Both skills written on first install + expect(firstParsed.every(r => r.action === 'written')).toBe(true); - // Capture file content before second run - const filePath = join(tmpDir, TARGETS.claude.path); - const contentBefore = readFileSync(filePath, 'utf8'); + // Capture file contents before second run + const verifyPath = join(tmpDir, pathFor('claude', 'testsprite-verify')); + const onboardPath = join(tmpDir, pathFor('claude', 'testsprite-onboard')); + const verifyBefore = readFileSync(verifyPath, 'utf8'); + const onboardBefore = readFileSync(onboardPath, 'utf8'); // Second install const second = runCli([ @@ -242,12 +285,13 @@ describe('idempotent re-run', () => { 'json', ]); expect(second.status).toBe(0); - const secondParsed = JSON.parse(second.stdout) as Array<{ action: string }>; - expect(secondParsed[0]!.action).toBe('skipped'); + const secondParsed = JSON.parse(second.stdout) as Array<{ path: string; action: string }>; + // Both skills skipped on second install + expect(secondParsed.every(r => r.action === 'skipped')).toBe(true); - // File must be byte-identical - const contentAfter = readFileSync(filePath, 'utf8'); - expect(contentAfter).toBe(contentBefore); + // Files must be byte-identical + expect(readFileSync(verifyPath, 'utf8')).toBe(verifyBefore); + expect(readFileSync(onboardPath, 'utf8')).toBe(onboardBefore); }); }); @@ -256,7 +300,7 @@ describe('idempotent re-run', () => { // --------------------------------------------------------------------------- describe('conflict handling', () => { - it('exits 6 with action: blocked when file differs and no --force', () => { + it('exits 6 with action: blocked when verify file differs and no --force', () => { const tmpDir = freshTmpDir(); // First install @@ -271,11 +315,11 @@ describe('conflict handling', () => { ]); expect(first.status).toBe(0); - // Hand-edit the file - const filePath = join(tmpDir, TARGETS.claude.path); - const originalContent = readFileSync(filePath, 'utf8'); + // Hand-edit the verify file + const verifyFilePath = join(tmpDir, pathFor('claude', 'testsprite-verify')); + const originalContent = readFileSync(verifyFilePath, 'utf8'); const editedContent = originalContent + '\n\n'; - writeFileSync(filePath, editedContent, 'utf8'); + writeFileSync(verifyFilePath, editedContent, 'utf8'); // Re-run without --force const second = runCli([ @@ -289,12 +333,14 @@ describe('conflict handling', () => { ]); expect(second.status).toBe(6); - // action: blocked in JSON - const parsed = JSON.parse(second.stdout) as Array<{ action: string }>; - expect(parsed[0]!.action).toBe('blocked'); + // At least one entry must be blocked (the verify file) + const parsed = JSON.parse(second.stdout) as Array<{ path: string; action: string }>; + const blockedEntry = parsed.find(r => r.path === pathFor('claude', 'testsprite-verify')); + expect(blockedEntry, 'verify entry should be blocked').toBeDefined(); + expect(blockedEntry!.action).toBe('blocked'); // File must be unchanged (not overwritten) - expect(readFileSync(filePath, 'utf8')).toBe(editedContent); + expect(readFileSync(verifyFilePath, 'utf8')).toBe(editedContent); // Stderr must contain --force hint expect(second.stderr).toContain('--force'); @@ -306,16 +352,16 @@ describe('conflict handling', () => { // --------------------------------------------------------------------------- describe('force overwrite with backup', () => { - it('--force exits 0 with action: updated, file = canonical, .bak holds edited bytes', () => { + it('--force exits 0 with action: updated for edited file, .bak holds edited bytes', () => { const tmpDir = freshTmpDir(); // First install runCli(['agent', 'install', '--target=claude', '--dir', tmpDir, '--output', 'json']); - // Hand-edit the file - const filePath = join(tmpDir, TARGETS.claude.path); - const editedContent = readFileSync(filePath, 'utf8') + '\n\n'; - writeFileSync(filePath, editedContent, 'utf8'); + // Hand-edit the verify file + const verifyFilePath = join(tmpDir, pathFor('claude', 'testsprite-verify')); + const editedContent = readFileSync(verifyFilePath, 'utf8') + '\n\n'; + writeFileSync(verifyFilePath, editedContent, 'utf8'); // Re-run with --force const forced = runCli([ @@ -330,15 +376,17 @@ describe('force overwrite with backup', () => { ]); expect(forced.status).toBe(0); - const parsed = JSON.parse(forced.stdout) as Array<{ action: string }>; - expect(parsed[0]!.action).toBe('updated'); + const parsed = JSON.parse(forced.stdout) as Array<{ path: string; action: string }>; + const verifyEntry = parsed.find(r => r.path === pathFor('claude', 'testsprite-verify')); + expect(verifyEntry, 'verify entry must be present').toBeDefined(); + expect(verifyEntry!.action).toBe('updated'); - // File must now equal canonical content - const { content: canonical } = renderForTarget('claude'); - expect(readFileSync(filePath, 'utf8')).toBe(canonical); + // Verify file must now equal canonical content + const { content: canonicalVerify } = renderForTarget('claude', 'testsprite-verify'); + expect(readFileSync(verifyFilePath, 'utf8')).toBe(canonicalVerify); // .bak must hold the edited bytes - const bakPath = filePath + '.bak'; + const bakPath = verifyFilePath + '.bak'; expect(existsSync(bakPath), '.bak file must exist').toBe(true); expect(readFileSync(bakPath, 'utf8')).toBe(editedContent); }); @@ -349,7 +397,7 @@ describe('force overwrite with backup', () => { // --------------------------------------------------------------------------- describe('dry-run', () => { - it('--dry-run exits 0, prints path + byte count to stderr, creates no file', () => { + it('--dry-run exits 0, prints both skill paths to stderr, creates no files', () => { const tmpDir = freshTmpDir(); const result = runCli([ @@ -364,15 +412,16 @@ describe('dry-run', () => { ]); expect(result.status).toBe(0); - // Stderr shows the intended path (absolute resolved path) and byte count - // The CLI emits "[dry-run] would write ( bytes)" - expect(result.stderr).toContain(TARGETS.claude.path); - // "would write" banner + // Stderr shows both skill paths and "would write" banner expect(result.stderr).toContain('would write'); + expect(result.stderr).toContain(pathFor('claude', 'testsprite-verify')); + expect(result.stderr).toContain(pathFor('claude', 'testsprite-onboard')); - // No file created on disk - const filePath = join(tmpDir, TARGETS.claude.path); - expect(existsSync(filePath), 'file must NOT be created in dry-run').toBe(false); + // No files created on disk for either skill + for (const skill of DEFAULT_SKILLS) { + const filePath = join(tmpDir, pathFor('claude', skill)); + expect(existsSync(filePath), `file must NOT be created in dry-run: ${skill}`).toBe(false); + } }); }); @@ -381,7 +430,7 @@ describe('dry-run', () => { // --------------------------------------------------------------------------- describe('multi-target install', () => { - it('--target=claude,cursor,cline,antigravity,codex writes all five targets, exit 0', () => { + it('--target=claude,cursor,cline,antigravity,codex writes all targets + skills, exit 0', () => { const tmpDir = freshTmpDir(); const result = runCli([ @@ -403,15 +452,24 @@ describe('multi-target install', () => { const allTargets: AgentTarget[] = ['claude', 'cursor', 'cline', 'antigravity', 'codex']; for (const target of allTargets) { - const entry = parsed.find(r => r.target === target); - expect(entry, `entry for ${target}`).toBeDefined(); - - const expectedAction = - TARGETS[target].mode === 'managed-section' ? 'section-installed' : 'written'; - expect(entry!.action, `action for ${target}`).toBe(expectedAction); - - const absPath = join(tmpDir, TARGETS[target].path); - expect(existsSync(absPath), `file at ${absPath}`).toBe(true); + if (TARGETS[target].mode === 'managed-section') { + // codex: one result aggregating all skills + const entry = parsed.find(r => r.target === target); + expect(entry, `entry for ${target}`).toBeDefined(); + expect(entry!.action, `action for ${target}`).toBe('section-installed'); + const absPath = join(tmpDir, TARGETS[target].path); + expect(existsSync(absPath), `file at ${absPath}`).toBe(true); + } else { + // own-file: one result per skill + for (const skill of DEFAULT_SKILLS) { + const skillPath = pathFor(target, skill); + const entry = parsed.find(r => r.target === target && r.path === skillPath); + expect(entry, `entry for ${target}/${skill}`).toBeDefined(); + expect(entry!.action, `action for ${target}/${skill}`).toBe('written'); + const absPath = join(tmpDir, skillPath); + expect(existsSync(absPath), `file at ${absPath}`).toBe(true); + } + } } }); }); @@ -631,12 +689,148 @@ describe('managed-section (codex target)', () => { }); // --------------------------------------------------------------------------- -// 10. Matrix-coverage guard — hardcoded list forces a conscious update when +// 10. --skill flag: install a single named skill +// --------------------------------------------------------------------------- + +describe('--skill flag', () => { + it('--skill testsprite-onboard installs only the onboard file, not verify', () => { + const tmpDir = freshTmpDir(); + + const result = runCli([ + 'agent', + 'install', + '--target=claude', + '--skill', + 'testsprite-onboard', + '--dir', + tmpDir, + '--output', + 'json', + ]); + expect(result.status).toBe(0); + + const parsed = JSON.parse(result.stdout) as Array<{ path: string; action: string }>; + // Only one result — the onboard skill + expect(parsed.length).toBe(1); + expect(parsed[0]!.path).toBe(pathFor('claude', 'testsprite-onboard')); + expect(parsed[0]!.action).toBe('written'); + + // Onboard file must exist + const onboardPath = join(tmpDir, pathFor('claude', 'testsprite-onboard')); + expect(existsSync(onboardPath), 'onboard file must exist').toBe(true); + + // Verify file must NOT exist + const verifyPath = join(tmpDir, pathFor('claude', 'testsprite-verify')); + expect(existsSync(verifyPath), 'verify file must NOT exist').toBe(false); + }); + + it('unknown --skill bogus exits 5 with documented error message', () => { + const tmpDir = freshTmpDir(); + + const result = runCli([ + 'agent', + 'install', + '--target=claude', + '--skill', + 'bogus', + '--dir', + tmpDir, + '--output', + 'json', + ]); + expect(result.status).toBe(5); + + // The error message must name the unknown skill and list supported skills + expect(result.stderr).toContain('bogus'); + expect(result.stderr).toContain('testsprite-verify'); + expect(result.stderr).toContain('testsprite-onboard'); + + // Nothing written to disk + for (const skill of Object.keys(SKILLS)) { + const absPath = join(tmpDir, pathFor('claude', skill)); + expect(existsSync(absPath), `unexpected file at ${absPath}`).toBe(false); + } + }); +}); + +// --------------------------------------------------------------------------- +// 11. agent list — includes SKILL column with both skill names +// --------------------------------------------------------------------------- + +describe('agent list', () => { + it('output includes TARGET, SKILL column header and both default skill names', () => { + const result = runCli(['agent', 'list']); + expect(result.status).toBe(0); + + // Header must include TARGET and SKILL columns + expect(result.stdout).toContain('TARGET'); + expect(result.stdout).toContain('SKILL'); + + // Both default skills must appear in the output + for (const skill of DEFAULT_SKILLS) { + expect(result.stdout, `${skill} should appear in agent list`).toContain(skill); + } + + // All targets must appear + for (const target of Object.keys(TARGETS)) { + expect(result.stdout, `${target} should appear in agent list`).toContain(target); + } + }); + + it('--output json returns an array with one entry per (target × skill)', () => { + const result = runCli(['agent', 'list', '--output', 'json']); + expect(result.status).toBe(0); + + const parsed = JSON.parse(result.stdout) as Array<{ + target: string; + skill: string; + status: string; + mode: string; + path: string; + }>; + expect(Array.isArray(parsed)).toBe(true); + + // Expected: 5 targets × 2 skills = 10 rows + const expectedCount = Object.keys(TARGETS).length * DEFAULT_SKILLS.length; + expect(parsed.length).toBe(expectedCount); + + // Every row must have a non-empty skill field from DEFAULT_SKILLS + for (const row of parsed) { + expect(DEFAULT_SKILLS as readonly string[]).toContain(row.skill); + } + + // Claude verify row must have the verify path + const claudeVerify = parsed.find(r => r.target === 'claude' && r.skill === 'testsprite-verify'); + expect(claudeVerify).toBeDefined(); + expect(claudeVerify!.path).toBe(pathFor('claude', 'testsprite-verify')); + + // Claude onboard row must have the onboard path + const claudeOnboard = parsed.find( + r => r.target === 'claude' && r.skill === 'testsprite-onboard', + ); + expect(claudeOnboard).toBeDefined(); + expect(claudeOnboard!.path).toBe(pathFor('claude', 'testsprite-onboard')); + }); +}); + +// --------------------------------------------------------------------------- +// 12. Matrix-coverage guard — hardcoded list forces a conscious update when // a target is added or removed from TARGETS. // --------------------------------------------------------------------------- describe('matrix coverage guard', () => { it('TARGETS matches the documented, e2e-covered set (update this list when adding a target)', () => { - expect(Object.keys(TARGETS)).toEqual(['claude', 'antigravity', 'cursor', 'cline', 'codex']); + expect(Object.keys(TARGETS)).toEqual([ + 'claude', + 'antigravity', + 'cursor', + 'cline', + 'codex', + 'gemini', + ]); + }); + + it('SKILLS matches the documented, e2e-covered set (update this list when adding a skill)', () => { + expect(Object.keys(SKILLS)).toEqual(['testsprite-verify', 'testsprite-onboard']); }); }); diff --git a/test/e2e/setup.e2e.test.ts b/test/e2e/setup.e2e.test.ts index 961d2dc..357a2aa 100644 --- a/test/e2e/setup.e2e.test.ts +++ b/test/e2e/setup.e2e.test.ts @@ -15,7 +15,7 @@ import { dirname, join, resolve } from 'node:path'; import { spawnSync } from 'node:child_process'; import { fileURLToPath } from 'node:url'; import { afterEach, beforeAll, describe, expect, it } from 'vitest'; -import { TARGETS } from '../../src/lib/agent-targets.js'; +import { TARGETS, pathFor } from '../../src/lib/agent-targets.js'; const __dirname = dirname(fileURLToPath(import.meta.url)); const REPO_ROOT = resolve(__dirname, '../..'); @@ -111,11 +111,11 @@ describe('setup --dry-run --no-agent', () => { }); // --------------------------------------------------------------------------- -// 3. setup --dry-run with default claude agent: shows would-write for claude +// 3. setup --dry-run with default claude agent: shows would-write for both skills // --------------------------------------------------------------------------- describe('setup --dry-run (with agent)', () => { - it('exits 0, shows would-write preview for claude skill, no file created', () => { + it('exits 0, shows would-write preview for both claude skill files, no files created', () => { const tmpDir = freshTmpDir(); const credsTmpDir = freshTmpDir(); @@ -135,10 +135,16 @@ describe('setup --dry-run (with agent)', () => { expect(result.status).toBe(0); expect(result.stderr).toContain('[dry-run]'); - expect(result.stderr).toContain(TARGETS.claude.path); - const skillPath = join(tmpDir, TARGETS.claude.path); - expect(existsSync(skillPath)).toBe(false); + // Both skill paths must appear in the dry-run preview + const verifyPath = pathFor('claude', 'testsprite-verify'); + const onboardPath = pathFor('claude', 'testsprite-onboard'); + expect(result.stderr, 'verify path in dry-run preview').toContain(verifyPath); + expect(result.stderr, 'onboard path in dry-run preview').toContain(onboardPath); + + // No files written under dry-run + expect(existsSync(join(tmpDir, verifyPath))).toBe(false); + expect(existsSync(join(tmpDir, onboardPath))).toBe(false); }); }); @@ -216,7 +222,14 @@ describe('deprecated `init` alias', () => { describe('matrix coverage guard', () => { it('TARGETS matches the documented set (update this list when adding a target)', () => { - expect(Object.keys(TARGETS)).toEqual(['claude', 'antigravity', 'cursor', 'cline', 'codex']); + expect(Object.keys(TARGETS)).toEqual([ + 'claude', + 'antigravity', + 'cursor', + 'cline', + 'codex', + 'gemini', + ]); }); });