From ba9ec3e945b6e1ea267f0bf4ab6815073ede4168 Mon Sep 17 00:00:00 2001 From: Mike Pennisi Date: Wed, 20 May 2026 15:50:34 -0400 Subject: [PATCH 1/3] RFC 239: Policy on LLM assistance in contributions --- rfcs/llm_policy.md | 85 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 rfcs/llm_policy.md diff --git a/rfcs/llm_policy.md b/rfcs/llm_policy.md new file mode 100644 index 0000000..879b843 --- /dev/null +++ b/rfcs/llm_policy.md @@ -0,0 +1,85 @@ +# RFC 239: Policy on LLM assistance in contributions + +## Summary + +Introduce guidelines for acceptable use of large-language models when +contributing to web-platform-tests. + +## Background + +[#202 Set policy for LLM-generated +tests](https://github.com/web-platform-tests/rfcs/issues/202) includes evidence +for public interest in a formal policy for LLM usage in authoring contributions +to WPT. + +The Chrome team is exploring applications of LLMs for detecting coverage gaps +and for filling those gaps with generated code. ([Project +repository](https://github.com/GoogleChromeLabs/wpt-gen), [April 2026 +presentation](https://www.youtube.com/watch?v=9r0PBbJFLoM)) + +A few examples of policies on LLM use in FOSS contributions: + +- permissive + - [ghostty/AI_POLICY.md at main · ghostty-org/ghostty](https://github.com/ghostty-org/ghostty/blob/main/AI_POLICY.md) + - [Policy about LLM generated code from PRs · Issue #28335 · opencv/opencv](https://github.com/opencv/opencv/issues/28335) + - [CONTRIBUTING.md: Guidelines relevant to AI-assisted contributions by gasche · Pull Request #14052 · ocaml/ocaml](https://github.com/ocaml/ocaml/pull/14052) + - [LLVM AI Tool Use Policy — LLVM 23.0.0git documentation](https://llvm.org/docs/AIToolPolicy.html) +- prohibitive + - [Code of Conduct ⚡ Zig Programming Language](https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy) + - [Getting Started - The Servo Book](https://book.servo.org/contributing/getting-started.html#ai-contributions) + +## Details + +Proposed text: + +> ### For Individual Contributors +> +> #### Disclosure +> +> Contributions that contain substantial amounts of tool-generated content must +> be labeled as such. +> +> #### Attribution +> +> Commits generated entirely by an LLM must be attributed to the LLM in the +> "Author" field. +> +> #### Understanding +> +> Every pull request must be initiated by one human. That person must author +> the pull request description, understand every change proposed, and be +> prepared to engage in technical discussion regarding those changes. +> +> ### For Trusted External Review +> +> Some external projects conduct review which the WPT maintainers recognize as +> authoritative. From rendering engines like Gecko to dedicated test suites +> like WASM, patches merged in these projects are incorporated into WPT without +> further review. The policy outlined by this document does not apply to these +> contributions; the external projects are trusted to determine their own +> mechanisms for quality assurance. + +## Risks + +### Discouraging volunteers + +All but the most permissive policy is effectively another hurdle to +contributing to the project. Friction in the contribution process could deter +people who might otherwise volunteer their time to help improve the project. + +In some sense, adding friction is the goal of this policy. New technology has +removed barriers which previously restricted unqualified individuals from +participation. Rather than introducing more restrictions on good-faith actors, +an ideal policy will buttress eroded structural barriers with more intentional +social ones. + +### Encouraging low-value contributions + +All but the most restrictive policy could be interpreted as an invitation to +take shortcuts which undermine the quality of contributions. + +However, it will not be possible to strictly enforce any policy. It inevitably +falls on contributors to follow rules and for administrators to police +transgressions. Respect in public works projects is never guaranteed; policies +exist only to make expectations clear (this is the same dynamic that guides the +design and enforcement of codes of conduct). From 3f4f8e0787d52a3bb4f4436b415c0e9146cd334b Mon Sep 17 00:00:00 2001 From: Mike Pennisi Date: Tue, 2 Jun 2026 16:06:48 -0400 Subject: [PATCH 2/3] Incorporate review feedback --- rfcs/llm_policy.md | 42 ++++++++++++++++++++++++++++-------------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/rfcs/llm_policy.md b/rfcs/llm_policy.md index 879b843..c4d7581 100644 --- a/rfcs/llm_policy.md +++ b/rfcs/llm_policy.md @@ -24,25 +24,41 @@ A few examples of policies on LLM use in FOSS contributions: - [Policy about LLM generated code from PRs · Issue #28335 · opencv/opencv](https://github.com/opencv/opencv/issues/28335) - [CONTRIBUTING.md: Guidelines relevant to AI-assisted contributions by gasche · Pull Request #14052 · ocaml/ocaml](https://github.com/ocaml/ocaml/pull/14052) - [LLVM AI Tool Use Policy — LLVM 23.0.0git documentation](https://llvm.org/docs/AIToolPolicy.html) + - [Chromium AI Coding Policy](https://chromium.googlesource.com/chromium/src/+/4f44016dfbd4fcd890694c00d7f9ec6dcefe4955/agents/ai_policy.md) + - [Firefox AI Coding Policy](https://github.com/mozilla-firefox/firefox/blob/1f7030c8de8f2b349c7d91d7b5a3253c109a1cc1/docs/contributing/ai-coding.md) - prohibitive - [Code of Conduct ⚡ Zig Programming Language](https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy) - [Getting Started - The Servo Book](https://book.servo.org/contributing/getting-started.html#ai-contributions) ## Details -Proposed text: +The following text describes the policy in full and will be maintained in a +dedicated document within WPT's `docs/writing-tests/` directory (which will be +referenced both from the project's `README.md` file and the +`docs/writing-tests/index.md` file): -> ### For Individual Contributors +> ### Guidelines for acceptable LLM use +> +> The use of large language models (LLMs) as tools to help author commits to +> this repository is allowed with the stipulations described below. +> Contributors who repeatedly fail to adhere to these guidelines may be banned +> from contributing to this project. > > #### Disclosure > -> Contributions that contain substantial amounts of tool-generated content must -> be labeled as such. +> If LLMs are used as a significant input to a commit, authors are encouraged +> to include details about how they were used as part of the commit message in +> order to help review and future understanding of the code. +> +> Human-authored code discourse (e.g. issue descriptions, pull request +> descriptions, and responses to discussion threads) should not include +> LLM-generated content in the main text; any such content must be clearly +> labelled and placed inside a `
` element. > > #### Attribution > -> Commits generated entirely by an LLM must be attributed to the LLM in the -> "Author" field. +> All commits must be attributed to the human who is taking responsibility for +> them, regardless of LLM use. > > #### Understanding > @@ -50,14 +66,9 @@ Proposed text: > the pull request description, understand every change proposed, and be > prepared to engage in technical discussion regarding those changes. > -> ### For Trusted External Review +> #### Discussion > -> Some external projects conduct review which the WPT maintainers recognize as -> authoritative. From rendering engines like Gecko to dedicated test suites -> like WASM, patches merged in these projects are incorporated into WPT without -> further review. The policy outlined by this document does not apply to these -> contributions; the external projects are trusted to determine their own -> mechanisms for quality assurance. +> Discussion ## Risks @@ -76,7 +87,10 @@ social ones. ### Encouraging low-value contributions All but the most restrictive policy could be interpreted as an invitation to -take shortcuts which undermine the quality of contributions. +take shortcuts which undermine the quality of contributions. Any permissive +policy might be taken as encouragement to rely on fallible tools (LLMs are +particularly susceptible to certain kinds of test-writing errors, such as +over-fitting and fabrication). However, it will not be possible to strictly enforce any policy. It inevitably falls on contributors to follow rules and for administrators to police From 8645bc2868a931595e470e23dbb1a10520ac2ee4 Mon Sep 17 00:00:00 2001 From: Mike Pennisi Date: Tue, 9 Jun 2026 14:02:07 -0400 Subject: [PATCH 3/3] Remove errant section --- rfcs/llm_policy.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/rfcs/llm_policy.md b/rfcs/llm_policy.md index c4d7581..f7fd64b 100644 --- a/rfcs/llm_policy.md +++ b/rfcs/llm_policy.md @@ -65,10 +65,6 @@ referenced both from the project's `README.md` file and the > Every pull request must be initiated by one human. That person must author > the pull request description, understand every change proposed, and be > prepared to engage in technical discussion regarding those changes. -> -> #### Discussion -> -> Discussion ## Risks