Skip to content

docs: add an AI policy and agent instructions - #5377

Open
marcoscaceres wants to merge 28 commits into
mainfrom
docs/ai-policy
Open

docs: add an AI policy and agent instructions#5377
marcoscaceres wants to merge 28 commits into
mainfrom
docs/ai-policy

Conversation

@marcoscaceres

@marcoscaceres marcoscaceres commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Adds an AI policy, and the two files GitHub documents for telling a coding agent how this project works.

The policy asks for disclosure, one concern per pull request, an honest accounting of what an issue asked for, and that a test offered as proof of a fix can actually fail without it. It also says what the maintainers hold themselves to when we generate work ourselves, which is stricter than what we ask of anyone else, and it asks nothing of contributors beyond one model and their own judgment. CONTRIBUTING.md points at it, since that is the file an agent reads first.

.github/copilot-instructions.md carries the things an agent cannot infer: that karma loads bundles from builds/ and there is one per profile, so a source change does nothing until it is rebuilt, and that BROWSERS must be set or the suite hangs after printing START: with nothing explaining why. .github/workflows/copilot-setup-steps.yml pre-installs dependencies and keeps puppeteer's browser inside the workspace, since the default cache location is not guaranteed to survive into the agent's session.

Reviewed by Gemini and Copilot as well as by hand. Between them they found a contradiction between two sections of the policy, a labeling rule that required triage access most contributors do not have, a missing privacy rule, and the hang above, which was the worst of them.

Written with AI: Claude drafted the policy and the two configuration files, Gemini and Copilot reviewed them, and I read and ran everything before pushing. Per AI_POLICY.md.

Copilot AI lite review requested due to automatic review settings August 4, 2026 06:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds contributor-facing guidance and GitHub Copilot cloud-agent configuration so AI-assisted contributions follow project expectations and Copilot sessions start with a working, reproducible ReSpec toolchain.

Changes:

  • Add an AI_POLICY.md and link to it from README.md and CONTRIBUTING.md.
  • Add Copilot agent instructions in .github/copilot-instructions.md describing build/test and project conventions.
  • Add a Copilot setup workflow to preinstall dependencies/builds and validate puppeteer availability, plus ignore the local puppeteer cache directory.

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
README.md Adds a pointer to the new AI policy for prospective AI-assisted contributors.
CONTRIBUTING.md Updates contributor guidance to include AI policy and adds an illustrative image.
AI_POLICY.md Introduces the project’s AI contribution policy and maintainer standards.
.gitignore Ignores the repo-local puppeteer cache directory used by Copilot setup.
.github/workflows/copilot-setup-steps.yml Adds Copilot setup steps to install deps, build bundles, and validate puppeteer browser availability.
.github/copilot-instructions.md Documents how Copilot should build/test and follow house style in this repo.
Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:49

  • The puppeteer browser check should run with the same PUPPETEER_CACHE_DIR as the install step; otherwise it may validate a different cache location than the one the agent will rely on (and if Copilot ignores job-level env, this step won't see it either). Set the cache dir explicitly on this step.
      # Fails the setup loudly if puppeteer has no usable browser, rather than
      # leaving the agent to discover it mid-task.
      - name: Check puppeteer has a browser
        run: |
          node -e "

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/copilot-setup-steps.yml
Comment thread .github/copilot-instructions.md
Copilot AI review requested due to automatic review settings August 4, 2026 07:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

AI_POLICY.md:32

  • This sentence is missing a verb (“If AI generated logic…”), which makes it ungrammatical and harder to understand. Consider rephrasing to include the missing verb and (optionally) hyphenating “AI-generated”.
If AI generated logic in your contribution, say so in the pull request
description. Generated logic means an agent, a function, an algorithm, a test, a
feature: something you would otherwise have had to work out.

Copilot AI review requested due to automatic review settings August 4, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

AI_POLICY.md:6

  • The image alt text is extremely detailed and includes non-essential scene description. For accessibility, alt text should focus on what matters to a reader (the "AI is welcome here" message) and stay concise.
<img src="assets/ai-is-welcome-here.png" alt="A cheerful pixel-art robot smiling straight at the camera while a forest burns orange behind it. The caption reads &quot;AI is welcome here.&quot;" width="320">

CONTRIBUTING.md:1

  • The image alt text is very long and includes incidental background detail. For accessibility, consider using concise alt text that conveys the purpose/message of the image.
<img src="assets/ai-is-welcome-here.png" alt="A cheerful pixel-art robot smiling straight at the camera while a forest burns orange behind it. The caption reads &quot;AI is welcome here.&quot;" width="220" align="right">

Comment thread .github/workflows/copilot-setup-steps.yml
Copilot AI review requested due to automatic review settings August 4, 2026 11:09
@marcoscaceres

Copy link
Copy Markdown
Contributor Author

Also acted on the two suppressed comments, since both were right.

The puppeteer check step now sets PUPPETEER_CACHE_DIR explicitly rather than inheriting it. Same root cause as the thread above: job-level env is not honored, so the check would have validated a location nothing had populated.

And AI_POLICY.md now reads "If AI generated any of the logic in your contribution", which fixes the ambiguity.

Two things running it afterwards found that the review could not have predicted. Putting the browser inside the workspace means eslint walks it, producing 12 errors from Chrome's own bundled scripts, including 'chrome' is not defined and eval can be harmful. .gitignore was not enough on its own, since eslint does not consult it, so .cache/** is now in eslint's ignores alongside builds/** and js/**.

The other is that the workspace cache is about 340 MB and is not shared between checkouts. That is the right trade for an ephemeral agent environment, but wrong for a human with several worktrees, so the instructions now say to skip the export if you already have a browser in puppeteer's default cache.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:64

  • puppeteer is ESM-only in v25+, so require('puppeteer') in the node -e check will throw and make the setup fail even when the browser is installed. Use a dynamic import('puppeteer') instead.
          Promise.resolve(require('puppeteer').executablePath())

Copilot AI review requested due to automatic review settings August 4, 2026 11:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

AI_POLICY.md:78

  • The commit-message guidance here says a Co-Authored-By trailer naming the model is welcome, but .github/copilot-instructions.md later says "Never add Co-Authored-By for a model." Having both rules in-repo is contradictory and will confuse contributors/agents about the expected house style. Consider aligning the policy with the Copilot instructions (or vice-versa) so there is a single rule.
**The commit message** is one imperative subject line, lowercase after any
prefix, no trailing period. A body only when the diff genuinely cannot be
understood without one, and then a sentence or two. No bullet lists, no
"Summary", no recap of what the diff already shows. A `Co-Authored-By` trailer
naming the model that wrote it is welcome, and consistent with what already
appears in this project's history.

.github/workflows/copilot-setup-steps.yml:70

  • The catch handler assumes the rejection reason has a .message property. If the promise rejects with a non-Error (string, null, etc.), this will throw inside the catch, making the failure output less reliable. Coerce the error to a string (or narrow to Error) before logging so this step always fails loudly with a useful message.
          import('puppeteer')
            .then(p => p.default.executablePath())
            .then(p => { fs.accessSync(p); console.log('browser ok:', p); })
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 05:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (1)

AI_POLICY.md:78

  • AI_POLICY.md says a Co-Authored-By trailer naming the model is welcome, but .github/copilot-instructions.md explicitly says to never add Co-Authored-By for a model. These docs should be consistent so contributors (and agents) don’t get conflicting guidance.
"Summary", no recap of what the diff already shows. A `Co-Authored-By` trailer
naming the model that wrote it is welcome, and consistent with what already
appears in this project's history.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (3)

CLAUDE.md:33

  • The Key modules table row includes an inline example ` | var | ` containing pipe characters. In GitHub Flavored Markdown tables, | inside a cell can be parsed as a column separator, which can break the table rendering. Use escaping or raw HTML to render the pipes literally inside the cell.
| `src/core/inlines.js`               | All inline shorthand syntax: `[= =]`, `{{ }}`, `[[SPEC]]`, `[[[...]]]`, ` | var | `   |

.github/workflows/copilot-setup-steps.yml:69

  • The puppeteer probe logs e.message, but the caught value is not guaranteed to be an Error. If it’s a string or another value, e.message will be undefined and the workflow will emit an unhelpful error. Normalize the message before logging so failures are always actionable.
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

.github/copilot-instructions.md:96

  • This guidance on commit messages contradicts AI_POLICY.md: the policy says a Co-Authored-By trailer naming the model is welcome, but these agent instructions say to never add it. Since this file points readers to AI_POLICY.md as the source of house style, these should be consistent to avoid confusing contributors/agents.
- Commit messages are one imperative subject line, lowercase after any prefix, no
  trailing period, and no body unless the diff cannot be understood without one.
  Never add `Co-Authored-By` for a model.

Comment thread .github/copilot-instructions.md Outdated
Comment thread .github/copilot-instructions.md Outdated
Comment thread .github/copilot-instructions.md Outdated
Comment thread .github/copilot-instructions.md Outdated
Comment thread .github/copilot-instructions.md
Comment thread AI_POLICY.md
Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 07:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:69

  • The puppeteer probe logs e.message without ensuring the rejection value is an Error. If a non-Error is thrown/rejected, this will print undefined (or itself throw if e is nullish), making the setup failure harder to diagnose. Coerce to a string (or narrow with instanceof Error) before logging.
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

Comment thread CLAUDE.md Outdated
Comment thread CONTRIBUTING.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Copilot <noreply@github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 23:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 10 changed files in this pull request and generated no new comments.

@marcoscaceres
marcoscaceres marked this pull request as ready for review August 19, 2026 23:17
Comment thread .github/copilot-instructions.md Outdated
Co-authored-by: Marcos Cáceres <marcos@marcosc.com>
Comment thread ai-pr-proof-requirement.md Outdated
@sidvishnoi
sidvishnoi self-requested a review August 25, 2026 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Contains AI-generated logic; see AI_POLICY.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants