Skip to content

fix(ci): make issue triage labeling deterministic - #1206

Open
phernandez wants to merge 4 commits into
mainfrom
codex/fix-claude-issue-triage
Open

fix(ci): make issue triage labeling deterministic#1206
phernandez wants to merge 4 commits into
mainfrom
codex/fix-claude-issue-triage

Conversation

@phernandez

@phernandez phernandez commented Aug 6, 2026

Copy link
Copy Markdown
Member

Why

The Claude issue-triage workflow advertises priority, component, complexity, and status labels that do not exist in this repository. Its only label tool accepted arbitrary candidate labels, silently ignored unknown names, and immediately added every candidate that happened to exist. Because the tool was add-only, label discovery mutated the live issue and Claude could not undo mistakes.

This has recurred since the helper was introduced on July 16: at least 12 triage comments explicitly report accidental label application. On issue #1205, an enhancement tracker ended up labeled as a bug, documentation request, duplicate, good first issue, question, and Cloud issue as well.

What Changed

  • Replace the generic --add-label interface with one semantic classification:
    --type bug|enhancement|documentation|question --component cloud|none.
  • Reject unsupported values, duplicate options, and candidate-probing arguments before any GitHub mutation.
  • Remove only obsolete triage-owned labels and add the requested classification without replacing the issue's full label set.
  • Abort before mutation if the current labels cannot be read.
  • Add the desired classification before removing obsolete labels, so an additive API failure leaves the prior classification intact.
  • Update the workflow prompt to call the helper exactly once, treat priority and complexity as prose, classify MCP as a component rather than a fifth type, and avoid duplicate/status labels that the bot does not own.
  • Require a concise triage comment without tool logs, taxonomy discussion, or task checklists.
  • Add regression coverage for targeted mutations, read failures, conflicting labels, invalid probes, required arguments, and workflow YAML/prompt syntax.

Implementation Details

The helper reads the issue's current labels only to identify obsolete values in the automated triage ownership set (bug, enhancement, documentation, question, and cloud). It first uses GitHub's additive labels endpoint to apply the requested type and optional Cloud component, then deletes obsolete owned labels individually. Labels owned by templates, maintainers, or other automation are never included in a replacement payload, so a label added concurrently after the read is preserved. If the additive request fails, no destructive mutation has occurred.

Repeated valid calls converge on one type and one Cloud/non-Cloud component choice. Unsupported attempts and failed reads stop before the first write. Duplicate detection remains read-only; Claude links the original issue in prose rather than labeling or closing the new issue.

Testing

Automated

  • uv run pytest -p pytest_mock --no-cov -q tests/test_claude_issue_triage.py: 11 passed
  • just fast-check: Ruff checks and formatting passed; ty passed
  • bash -n scripts/edit-issue-labels.sh: passed
  • git diff --check: passed

The focused suite parses the edited workflow as YAML and asserts the single semantic helper contract, the four-type/MCP-component taxonomy, exact additive POST/targeted DELETE ordering, preservation of non-triage labels, no mutation after a failed read, and no deletion after a failed additive request.

Manual

  • Inspected the live repository label taxonomy and recent Claude triage comments.
  • Confirmed at least 12 self-reported accidental-label incidents since July 16.
  • No live issue labels were mutated while testing this fix.

Risks / Follow-ups

  • The workflow itself cannot be exercised end to end without opening a real issue; the first issue opened after merge is the live acceptance check.
  • GitHub does not provide an atomic operation for replacing only a label subset. If a targeted delete fails after the desired labels are added, the helper exits nonzero and may temporarily leave an extra bot-owned label; a rerun converges without risking the requested classification or unrelated labels.
  • This PR prevents new label accumulation but intentionally does not clean labels on previously affected issues.

Signed-off-by: phernandez <paul@basicmachines.co>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31ca315e07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/edit-issue-labels.sh Outdated
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76f0f02db2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/edit-issue-labels.sh
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee06e942f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/edit-issue-labels.sh Outdated
Comment thread .github/workflows/claude-issue-triage.yml
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 1c15d00736

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@phernandez

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 1c15d00736

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant