From 55097deadb32cf13c70af246bc26da2e9a95a804 Mon Sep 17 00:00:00 2001 From: Oz Date: Mon, 3 Aug 2026 17:13:21 +0000 Subject: [PATCH 1/4] docs(skills): tighten source verification and durable UI guidance Co-Authored-By: Oz --- .agents/skills/draft_docs/SKILL.md | 13 ++++++++++++- .agents/templates/feature-doc.md | 16 +++++++++++----- .agents/templates/guide-page.md | 8 +++++--- 3 files changed, 28 insertions(+), 9 deletions(-) diff --git a/.agents/skills/draft_docs/SKILL.md b/.agents/skills/draft_docs/SKILL.md index 41fccc5a..4b76a0b4 100644 --- a/.agents/skills/draft_docs/SKILL.md +++ b/.agents/skills/draft_docs/SKILL.md @@ -81,6 +81,8 @@ To find these repos, search for directories named `warp-internal` and `warp-serv Use source code to verify technical behavior, understand feature implementation, and find accurate terminology. +When the draft names UI labels, Settings paths, CLI flags, default permissions, plan eligibility, or platform support, treat source (or a live build) as required verification—not optional color. If you cannot verify a claim, omit it or mark it for human review instead of guessing. + ### 6.5. Critical formatting rules These rules are frequently violated by agents. Apply them carefully during drafting: @@ -93,6 +95,11 @@ These rules are frequently violated by agents. Apply them carefully during draft - **Tables or parallel bullets for comparison and reference data** — When you present two or more parallel items (key types, plan tiers, environments) or structured reference data (API endpoints, parameters), use a Markdown table or tightly parallel bullets instead of one dense paragraph. ✅ a table with one row per API endpoint, or parallel `**Personal API keys**` / `**Agent API keys**` bullet groups ❌ a single paragraph mixing both key types and their billing rules - **Bold for UI elements** — Use `**Save**` not `` `Save` `` after action verbs like "click" - **Bold per-segment for Settings paths** — Use `**Settings** > **AI** > **Knowledge**` not `` `Settings > AI > Knowledge` `` +- **Verify labels, flags, and defaults against source** — Before documenting a button name, Settings path, CLI flag, permission default, or eligibility rule, confirm it in `warp-internal` / `warp-server` (or the live UI). ✅ `warp --auto-approve` after checking `TuiArgs` ❌ inventing `--fast-forward` from memory or an old PR description +- **Document durable behavior, not ephemeral chrome** — Prefer workflows, shortcuts, and outcomes that stay true when styling shifts. Drop glyph colors, pixel-level layout narration, and other pure presentation detail unless the reader must recognize them to succeed. ✅ "Press `Ctrl+C` once to stop the in-progress response." ❌ a full inventory of pending/running/failed glyph colors +- **State availability honestly** — If a capability is preview-only, platform-limited, interactive-only, or not yet in cloud agents, say so next to the claim. Never describe limited-preview behavior as generally available. ✅ "Linux post-processing adds smart cut; macOS applies a uniform speedup." ❌ listing smart cut as a property of every recording +- **Cover team-wide and admin effects** — For integrations and team features, state who can install, whether every teammate gets access immediately, and any per-user auth or admin steps on the external system. ✅ "A Jira admin must install the app; each teammate links their own account for run attribution." ❌ setup steps that only describe the installer's happy path +- **Related pages are required on new pages** — End new feature, integration, and guide pages with a Related pages (or Next steps) section. Link siblings and the natural next workflow so the page does not dead-end. ✅ Integrations overview + Agent Management Panel + a sibling Slack/Linear page ❌ no cross-links after Troubleshooting ### 7. Draft the doc Create the documentation using the appropriate template from `.agents/templates/`. Follow the structure for the identified content type and all rules in `AGENTS.md`. Each template includes visible bracketed instructions explaining what to put in each section. @@ -120,10 +127,14 @@ Before presenting the draft, verify against the quality checklist in `AGENTS.md` - [ ] Terminology matches the glossary (`.agents/references/terminology.md`) - [ ] Headers use sentence case (with proper feature name capitalization) - [ ] Lists use bold term + dash + explanation format -- [ ] Cross-references to related features are included +- [ ] Cross-references to related features are included (new pages include a Related pages or Next steps section) - [ ] Instructions include expected outcomes - [ ] Procedures are scannable: dense sections are split into numbered steps, short bullets, or concise subsections - [ ] UI surfaces and product terms use canonical names from `.agents/references/terminology.md` +- [ ] UI labels, CLI flags, permission defaults, and eligibility claims were verified against source or the live product +- [ ] The draft emphasizes durable behavior over ephemeral UI chrome (glyphs, pure styling, layout minutiae) +- [ ] Preview-only, platform-limited, or interactive-only capabilities are labeled as such +- [ ] Integrations and team features state admin requirements and who gets access after install - [ ] Product names with a corresponding entry in `src/data/vars.ts` use the variable syntax (`{VARS.KEY}` in prose, `{{TOKEN}}` in frontmatter) — not hardcoded strings - [ ] If AEO-driven, the draft follows the AEO brief, uses source vocabulary naturally, and avoids duplicative or junk-drawer coverage - [ ] Images have descriptive alt text diff --git a/.agents/templates/feature-doc.md b/.agents/templates/feature-doc.md index 4f3d24ca..8ca62f3d 100644 --- a/.agents/templates/feature-doc.md +++ b/.agents/templates/feature-doc.md @@ -34,7 +34,9 @@ Focus on what each capability means for the user.] Explain "what" and "why" before "how." Define new terms when they first appear. IMPORTANT: Do NOT include step-by-step procedures in this section. -Keep the conceptual and procedural sections clearly separated.] +Keep the conceptual and procedural sections clearly separated. +State platform, plan, preview, or interactive-only limits next to the behavior they constrain. +Do not invent internal tool names or implementation details the reader cannot act on.] ## [Usage/configuration section — sentence case. Rename to match the feature, e.g., "Creating environments", "Configuring integrations"] @@ -42,12 +44,15 @@ Keep the conceptual and procedural sections clearly separated.] Apply all procedural rules from AGENTS.md: - Motivate steps before giving instructions - Include expected outcomes after key steps -- Group related actions when they share the same UI context] +- Group related actions when they share the same UI context +- Verify every UI label, Settings path, and CLI flag against source or the live product before publishing +- Prefer durable actions and outcomes over ephemeral chrome (glyph colors, pure layout narration)] ### Prerequisites [Bulleted list with inline context for each prerequisite. -Include: what the thing is, where to get it, link to full reference.] +Include: what the thing is, where to get it, link to full reference. +For integrations and team features, include admin requirements, who gains access after install, and any per-user auth steps.] ### [Task name — sentence case. e.g., "Create an environment with the CLI"] @@ -62,8 +67,9 @@ Keep sections clearly delineated by type.] ## Related pages -[Cross-references to related features, next steps, deeper references. -Use descriptive link text.] +[Required on new feature and integration pages so the page does not dead-end. +Cross-reference related features, sibling integrations, next steps, and deeper references. +Use descriptive link text. Include at least one sibling or overview link and one next-step workflow link.] * [Related feature](path/to/page.md) * [Deeper guide](path/to/page.md) diff --git a/.agents/templates/guide-page.md b/.agents/templates/guide-page.md index 31c93e13..ae6887da 100644 --- a/.agents/templates/guide-page.md +++ b/.agents/templates/guide-page.md @@ -13,16 +13,18 @@ import VideoEmbed from '@components/VideoEmbed.astro'; {/* VARS: Also add the line below if this guide references product names from src/data/vars.ts. Then use {VARS.KEY} in prose. Use {{TOKEN}} in frontmatter YAML values above. See AGENTS.md → Content variables. */} {/* import { VARS } from '@data/vars'; */} - + ## Prerequisites -[List what the reader needs before starting. Include inline context: what each prerequisite is, where to get it, and a link to more info. Orient the reader by naming the application before any menu paths.] +[List what the reader needs before starting. Include inline context: what each prerequisite is, where to get it, and a link to more info. Orient the reader by naming the application before any menu paths. +If the workflow touches a team tool (Jira, Slack, GitHub, and so on), say who can install it, whether teammates get access immediately, and any per-user auth steps.] * **[Prerequisite 1]** — [What it is and where to get it]. See [link to docs] for details. * **[Prerequisite 2]** — [Brief context]. -[Use numbered H2 headings for each step (e.g., "## 1. Install Claude Code"). Do not add a "## Steps" wrapper heading — jump straight into the numbered steps after Prerequisites. Motivate each step: explain WHY before HOW, especially for setup steps. End each numbered sub-step with a period. Use ALL_CAPS for placeholder values in commands (e.g., YOUR_API_KEY). Do not use em dashes in procedural or instructional text. If there's an open-source repo for an example, link it. When referencing a Settings path or menu for the first time, orient the reader: "in the Warp app, go to **Settings** > ...".] +[Use numbered H2 headings for each step (e.g., "## 1. Install Claude Code"). Do not add a "## Steps" wrapper heading — jump straight into the numbered steps after Prerequisites. Motivate each step: explain WHY before HOW, especially for setup steps. End each numbered sub-step with a period. Use ALL_CAPS for placeholder values in commands (e.g., YOUR_API_KEY). Do not use em dashes in procedural or instructional text. If there's an open-source repo for an example, link it. When referencing a Settings path or menu for the first time, orient the reader: "in the Warp app, go to **Settings** > ...". +Keep prompts and UI detail durable: teach the workflow and decisions, not glyph colors or layout minutiae that drift. Verify flag names and menu labels against source before publishing.] ## [Action-oriented step title] From d837a538d7908205b76263a85f993c484fac039f Mon Sep 17 00:00:00 2001 From: oz-by-warp Date: Mon, 3 Aug 2026 18:33:58 +0000 Subject: [PATCH 2/4] docs: fix stale Docs Agent environment reference in weekly-404-monitor skill Co-Authored-By: Oz --- .agents/skills/weekly-404-monitor/SKILL.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/.agents/skills/weekly-404-monitor/SKILL.md b/.agents/skills/weekly-404-monitor/SKILL.md index 13a7f2cb..12a23cdc 100644 --- a/.agents/skills/weekly-404-monitor/SKILL.md +++ b/.agents/skills/weekly-404-monitor/SKILL.md @@ -236,17 +236,23 @@ Check: Vercel project env vars include `PUBLIC_RUDDERSTACK_WRITE_KEY` and `PUBLI This skill is designed for an Oz scheduled agent with a weekly cron trigger: every Monday at 9am PT (`0 17 * * 1` in UTC). +The live schedule (`Weekly docs 404 monitor`, schedule ID `xAqd8EBlaJ3EihLlfF3fOs`) runs in the **buzz** Oz environment (`qDewDp082oqhaq7ZJd6LJI`) — not the `Docs Agent` environment (`K5KStCm5aYvhfBJb8cHol6`) referenced by other docs automations (e.g. `validate_ui_refs`, `release_updates`). Point secret checks and schedule registration at the `buzz` environment for this skill. + To deploy (one-time setup): 1. Push this skill to `main` in the docs repo. -2. Verify the **Docs Agent** Oz environment (`K5KStCm5aYvhfBJb8cHol6`) has these secrets set: - - `METABASE_API_KEY` — Metabase API key for BigQuery - - `BUZZ_SLACK_TOKEN` — Slack bot token (already provisioned; used by other doc agents in this environment) +2. Verify the **buzz** Oz environment (`qDewDp082oqhaq7ZJd6LJI`) has: + - `warpdotdev/docs` in its configured repos, so `/home/user/warpdotdev/docs` exists at run start (confirmed present as of 2026-08-03). + - These secrets set (both are Team-scoped and already provisioned): + - `METABASE_API_KEY` — Metabase API key for BigQuery + - `BUZZ_SLACK_TOKEN` — Slack bot token (used by other doc agents in this environment) 3. Register the schedule via the Oz CLI: ```sh oz-dev schedule create \ --name "weekly-404-monitor" \ --cron "0 17 * * 1" \ - --environment K5KStCm5aYvhfBJb8cHol6 \ + --environment qDewDp082oqhaq7ZJd6LJI \ --prompt "You are running in the warpdotdev/docs repo. Read and follow the instructions in .agents/skills/weekly-404-monitor/SKILL.md." ``` This will make the schedule visible in oz.warp.dev under Schedules and ensure runs open PRs under the @oz-by-warp bot account. + +**2026-08-03 incident note**: An earlier run this week failed with "METABASE_API_KEY is not set" while pointing at the nonexistent `K5KStCm5aYvhfBJb8cHol6` environment referenced by this section (that ID is not visible/resolvable from the Oz account operating this schedule). The live schedule already runs against `qDewDp082oqhaq7ZJd6LJI` (`buzz`), which has both secrets and now has the `docs` repo checked out. This section was corrected to match; see `.agents/logs/weekly_404_monitor_runs.md` for the run history. From 6731cf3fe17f80fe97b4b5fb2e775b4cf84633d3 Mon Sep 17 00:00:00 2001 From: oz-by-warp Date: Mon, 3 Aug 2026 18:46:32 +0000 Subject: [PATCH 3/4] Revert "docs: fix stale Docs Agent environment reference in weekly-404-monitor skill" This reverts commit d837a538d7908205b76263a85f993c484fac039f. --- .agents/skills/weekly-404-monitor/SKILL.md | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/.agents/skills/weekly-404-monitor/SKILL.md b/.agents/skills/weekly-404-monitor/SKILL.md index 12a23cdc..13a7f2cb 100644 --- a/.agents/skills/weekly-404-monitor/SKILL.md +++ b/.agents/skills/weekly-404-monitor/SKILL.md @@ -236,23 +236,17 @@ Check: Vercel project env vars include `PUBLIC_RUDDERSTACK_WRITE_KEY` and `PUBLI This skill is designed for an Oz scheduled agent with a weekly cron trigger: every Monday at 9am PT (`0 17 * * 1` in UTC). -The live schedule (`Weekly docs 404 monitor`, schedule ID `xAqd8EBlaJ3EihLlfF3fOs`) runs in the **buzz** Oz environment (`qDewDp082oqhaq7ZJd6LJI`) — not the `Docs Agent` environment (`K5KStCm5aYvhfBJb8cHol6`) referenced by other docs automations (e.g. `validate_ui_refs`, `release_updates`). Point secret checks and schedule registration at the `buzz` environment for this skill. - To deploy (one-time setup): 1. Push this skill to `main` in the docs repo. -2. Verify the **buzz** Oz environment (`qDewDp082oqhaq7ZJd6LJI`) has: - - `warpdotdev/docs` in its configured repos, so `/home/user/warpdotdev/docs` exists at run start (confirmed present as of 2026-08-03). - - These secrets set (both are Team-scoped and already provisioned): - - `METABASE_API_KEY` — Metabase API key for BigQuery - - `BUZZ_SLACK_TOKEN` — Slack bot token (used by other doc agents in this environment) +2. Verify the **Docs Agent** Oz environment (`K5KStCm5aYvhfBJb8cHol6`) has these secrets set: + - `METABASE_API_KEY` — Metabase API key for BigQuery + - `BUZZ_SLACK_TOKEN` — Slack bot token (already provisioned; used by other doc agents in this environment) 3. Register the schedule via the Oz CLI: ```sh oz-dev schedule create \ --name "weekly-404-monitor" \ --cron "0 17 * * 1" \ - --environment qDewDp082oqhaq7ZJd6LJI \ + --environment K5KStCm5aYvhfBJb8cHol6 \ --prompt "You are running in the warpdotdev/docs repo. Read and follow the instructions in .agents/skills/weekly-404-monitor/SKILL.md." ``` This will make the schedule visible in oz.warp.dev under Schedules and ensure runs open PRs under the @oz-by-warp bot account. - -**2026-08-03 incident note**: An earlier run this week failed with "METABASE_API_KEY is not set" while pointing at the nonexistent `K5KStCm5aYvhfBJb8cHol6` environment referenced by this section (that ID is not visible/resolvable from the Oz account operating this schedule). The live schedule already runs against `qDewDp082oqhaq7ZJd6LJI` (`buzz`), which has both secrets and now has the `docs` repo checked out. This section was corrected to match; see `.agents/logs/weekly_404_monitor_runs.md` for the run history. From f6319bfc7e8fe8fd48c0fc6dfc2fdd05c248832f Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 4 Aug 2026 10:48:04 -0600 Subject: [PATCH 4/4] docs(skills): require unverified claims to be reported, not just marked Resolves the open question on source verification: keep the escape hatch for drafts written without warp-internal/warp-server access, but make it non-skippable. Unverified UI labels, flags, defaults, and eligibility claims must be marked inline and surfaced in an Unverified claims section of the PR description (or in the response for local sessions). Co-Authored-By: Warp Agent --- .agents/skills/create_pr/SKILL.md | 19 +++++++++++++++++++ .agents/skills/draft_docs/SKILL.md | 20 ++++++++++++++++++-- .agents/templates/feature-doc.md | 2 +- 3 files changed, 38 insertions(+), 3 deletions(-) diff --git a/.agents/skills/create_pr/SKILL.md b/.agents/skills/create_pr/SKILL.md index 158c078d..07a819db 100644 --- a/.agents/skills/create_pr/SKILL.md +++ b/.agents/skills/create_pr/SKILL.md @@ -133,6 +133,25 @@ This PR updates the Terminal and Agent modes documentation for the Oz launch. - Updated navigation entry title ``` +### Unverified claims (required on drafting PRs) + +Any PR that adds or updates page content must state which UI labels, Settings paths, CLI flags, permission defaults, plan eligibility, and platform-support claims could not be verified against `warp-internal`, `warp-server`, or a live build. See step 9.5 of the `draft_docs` skill. + +Include the section even when nothing is outstanding: + +```markdown +## Unverified claims +None — all UI labels, flags, defaults, and eligibility claims were verified against source. +``` + +When claims are outstanding, give the reviewer one bullet per claim with what would confirm it: + +```markdown +## Unverified claims +- `--auto-approve` flag name — `cloud-agents.mdx`, "Run an agent" — taken from the PRD; confirm against `TuiArgs` in `warp-internal`. +- **Settings** > **Agents** > **Permissions** path — `permissions.mdx`, "Defaults" — source repos were not available in this environment. +``` + ### Additional context (optional) - Link to related issues or discussions - Screenshots for visual changes diff --git a/.agents/skills/draft_docs/SKILL.md b/.agents/skills/draft_docs/SKILL.md index 4b76a0b4..b3c5fd6c 100644 --- a/.agents/skills/draft_docs/SKILL.md +++ b/.agents/skills/draft_docs/SKILL.md @@ -81,7 +81,14 @@ To find these repos, search for directories named `warp-internal` and `warp-serv Use source code to verify technical behavior, understand feature implementation, and find accurate terminology. -When the draft names UI labels, Settings paths, CLI flags, default permissions, plan eligibility, or platform support, treat source (or a live build) as required verification—not optional color. If you cannot verify a claim, omit it or mark it for human review instead of guessing. +When the draft names UI labels, Settings paths, CLI flags, default permissions, plan eligibility, or platform support, treat source (or a live build) as required verification—not optional color. A PRD or spec is not verification: labels and flag names routinely change between spec and ship. + +If you cannot verify a claim (for example, the source repos are not available in this environment), do not guess and do not silently drop it. Choose one of these, and record the claim either way: + +1. **Omit the claim** - Write around it when the page still works without it. Describe the action without naming the exact flag, or link to the reference page that will carry the detail. +2. **Include it with an inline marker** - Keep the spec's wording and flag it in an MDX comment next to the claim: `{/* VERIFY: flag name from PRD, unconfirmed against warp-internal */}`. + +Keep a running list of every unverified claim as you draft. Reporting that list is required — see step 9.5. ### 6.5. Critical formatting rules @@ -131,7 +138,7 @@ Before presenting the draft, verify against the quality checklist in `AGENTS.md` - [ ] Instructions include expected outcomes - [ ] Procedures are scannable: dense sections are split into numbered steps, short bullets, or concise subsections - [ ] UI surfaces and product terms use canonical names from `.agents/references/terminology.md` -- [ ] UI labels, CLI flags, permission defaults, and eligibility claims were verified against source or the live product +- [ ] UI labels, CLI flags, permission defaults, and eligibility claims were verified against source or the live product — anything unverified is marked inline and reported per step 9.5 - [ ] The draft emphasizes durable behavior over ephemeral UI chrome (glyphs, pure styling, layout minutiae) - [ ] Preview-only, platform-limited, or interactive-only capabilities are labeled as such - [ ] Integrations and team features state admin requirements and who gets access after install @@ -139,6 +146,15 @@ Before presenting the draft, verify against the quality checklist in `AGENTS.md` - [ ] If AEO-driven, the draft follows the AEO brief, uses source vocabulary naturally, and avoids duplicative or junk-drawer coverage - [ ] Images have descriptive alt text +### 9.5. Report unverified claims + +Inline markers alone are skippable: a reviewer who skims the rendered page or the diff will miss them. Surface the full list where the human cannot miss it. + +- **Agent-authored PRs** - Add an `## Unverified claims` section to the PR description. Include one bullet per claim with the claim itself, the file and section where it appears, and what would confirm it (for example, "check `TuiArgs` in `warp-internal`"). Include the section even when the list is empty, with the single line `None — all UI labels, flags, defaults, and eligibility claims were verified against source.` Never drop the section. +- **Local or interactive sessions** - List the same claims in your response to the user, before they review the draft. + +A reviewer must be able to see every unconfirmed claim without opening the diff. + ### 10. Update navigation and redirects If this is a new page, remind the user to: - Add it to the relevant section in `src/sidebar.ts`. diff --git a/.agents/templates/feature-doc.md b/.agents/templates/feature-doc.md index 8ca62f3d..5051337d 100644 --- a/.agents/templates/feature-doc.md +++ b/.agents/templates/feature-doc.md @@ -45,7 +45,7 @@ Apply all procedural rules from AGENTS.md: - Motivate steps before giving instructions - Include expected outcomes after key steps - Group related actions when they share the same UI context -- Verify every UI label, Settings path, and CLI flag against source or the live product before publishing +- Verify every UI label, Settings path, and CLI flag against source or the live product before publishing. If you cannot verify one, omit it or mark it with an inline `{/* VERIFY: ... */}` comment and report it per step 9.5 of the draft_docs skill - Prefer durable actions and outcomes over ephemeral chrome (glyph colors, pure layout narration)] ### Prerequisites