-
-
Notifications
You must be signed in to change notification settings - Fork 276
feat: maintainer toolset β scheduled triage sweep + daily Discord scorecard #950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tombeckenham
wants to merge
4
commits into
main
Choose a base branch
from
maintainer-toolset
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
9cabe22
feat: maintainer toolset β scheduled triage sweep + daily Discord scoβ¦
tombeckenham 0ff9f17
refactor: make maintainer areas optional, default to least-loaded rotβ¦
tombeckenham 96f3fe0
ci: apply automated fixes
autofix-ci[bot] 5169c28
fix(maintainer): escape Discord markdown in untrusted item titles
tombeckenham File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "repo": "TanStack/ai", | ||
| "maintainers": [ | ||
| { | ||
| "github": "tombeckenham", | ||
| "discord": null, | ||
| "maxOpenAssignments": 10 | ||
| }, | ||
| { | ||
| "github": "AlemTuzlak", | ||
| "discord": null, | ||
| "maxOpenAssignments": 15 | ||
| }, | ||
| { | ||
| "github": "jherr", | ||
| "discord": null, | ||
| "maxOpenAssignments": 8 | ||
| } | ||
| ], | ||
| "sla": { | ||
| "firstResponseHours": 24, | ||
| "followUpResponseHours": 48, | ||
| "staleAuthorDays": 14 | ||
| }, | ||
| "spam": { | ||
| "maxAccountAgeDays": 30, | ||
| "maxChangedLines": 30 | ||
| }, | ||
| "botAllowlist": ["renovate", "dependabot", "github-actions"], | ||
| "maxCommentsPerRun": 25 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: Maintainer Scorecard | ||
|
|
||
| # Daily maintainer to-do digest posted to Discord: SLA breaches, ready-to-merge | ||
| # PRs, per-maintainer queues, new/stale items, and response-time stats. | ||
| # Requires the DISCORD_MAINTAINER_WEBHOOK repo secret; without it the digest | ||
| # only lands in the workflow step summary. | ||
|
|
||
| on: | ||
| schedule: | ||
| # 21:00 UTC β start of the Australian workday | ||
| - cron: '0 21 * * *' | ||
| workflow_dispatch: | ||
| inputs: | ||
| dry_run: | ||
| description: 'Print the digest instead of posting to Discord' | ||
| type: boolean | ||
| default: true | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }} | ||
| cancel-in-progress: false | ||
|
|
||
| permissions: {} | ||
|
|
||
| jobs: | ||
| scorecard: | ||
| name: Scorecard | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| permissions: | ||
| contents: read | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Setup Tools | ||
| uses: TanStack/config/.github/setup@190f659075ff0845850e330883eb26d7ffd0671f # main | ||
| - name: Build and post scorecard | ||
| run: pnpm maintainer:scorecard | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_MAINTAINER_WEBHOOK }} | ||
| DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run && '1' || '' }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| name: Maintainer Sweep | ||
|
|
||
| # Scheduled triage sweep: assigns open PRs/issues to maintainers, posts a | ||
| # one-time ack + pre-review checklist, and reconciles waiting-on labels. | ||
| # API-only β PR code is never checked out or executed (no pull_request_target). | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '17 */3 * * *' | ||
| workflow_dispatch: | ||
| inputs: | ||
| dry_run: | ||
| description: 'Print planned actions without executing them' | ||
| type: boolean | ||
| default: true | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }} | ||
| cancel-in-progress: false | ||
|
|
||
| permissions: {} | ||
|
|
||
| jobs: | ||
| sweep: | ||
| name: Sweep | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| permissions: | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Setup Tools | ||
| uses: TanStack/config/.github/setup@190f659075ff0845850e330883eb26d7ffd0671f # main | ||
| - name: Run sweep | ||
| run: pnpm maintainer:sweep | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run && '1' || '' }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Maintainer toolset | ||
|
|
||
| Automation that keeps on top of open PRs, issues, and discussions: | ||
|
|
||
| - **Sweep** (`.github/workflows/maintainer-sweep.yml`, every 3h): assigns each | ||
| open PR/issue to a maintainer, posts a one-time ack comment with a | ||
| deterministic pre-review checklist, asks for a reproduction on bug reports | ||
| that lack one, and reconciles `waiting-on: *` / `ready-to-merge` / | ||
| `needs-repro` / `has-pr` labels. | ||
| - **Scorecard** (`.github/workflows/maintainer-scorecard.yml`, daily): posts a | ||
| to-do digest to Discord β SLA breaches, ready-to-merge PRs, per-maintainer | ||
| queues (including which items got fresh contributor replies), new/flagged/ | ||
| stale items, unanswered discussions, and response-time stats. | ||
|
|
||
| Both are **stateless and API-only**: every metric (first-response time, 7-day | ||
| deltas, SLA clocks) is recomputed from GitHub timelines each run, idempotency | ||
| comes from HTML markers in bot comments plus visible assignment/label state, | ||
| and PR code is never checked out or executed (no `pull_request_target`). | ||
|
|
||
| ## Configuration β `.github/maintainers.json` | ||
|
|
||
| | Field | Meaning | | ||
| | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `maintainers[].github` | GitHub login; the roster defines who counts as "a maintainer responded". | | ||
| | `maintainers[].discord` | Discord user id (snowflake) for digest @-mentions; `null` β plain bold name. | | ||
| | `maintainers[].areas` | Optional file globs (`**`, `*`, `?`) giving this maintainer routing priority for matching PRs (issues match package names in title/body). Omitted/empty (the default) β assignment is pure least-loaded rotation. | | ||
| | `maintainers[].maxOpenAssignments` | Routing skips a maintainer at this many open assignments. | | ||
| | `sla.firstResponseHours` | Deadline for the first human response on a new item (default 24h). | | ||
| | `sla.followUpResponseHours` | Deadline for answering a follow-up message (default 48h). | | ||
| | `sla.staleAuthorDays` | Author silence before an item is a nudge/close candidate (default 14d). | | ||
| | `spam.*` | Drive-by/bounty heuristic: account younger than `maxAccountAgeDays` **and** diff β€ `maxChangedLines` **and** no linked issue β flagged for human judgment, never auto-acked. | | ||
| | `botAllowlist` | Logins always treated as bots (excluded from human metrics and acks). | | ||
| | `maxCommentsPerRun` | Safety cap on comments per sweep; overflow defers to the next run. | | ||
|
|
||
| ## Secrets | ||
|
|
||
| - `DISCORD_MAINTAINER_WEBHOOK` β Discord webhook URL for the daily digest | ||
| (Server Settings β Integrations β Webhooks). Without it, the digest lands in | ||
| the workflow step summary only. | ||
| - The sweep uses the default `GITHUB_TOKEN` β no extra setup. | ||
|
|
||
| ## Running locally | ||
|
|
||
| ```bash | ||
| # Dry-run: prints the mutation plan / digest, changes nothing | ||
| pnpm maintainer:sweep --dry-run | ||
| pnpm maintainer:scorecard --dry-run | ||
|
|
||
| # Tests | ||
| pnpm test:maintainer | ||
| ``` | ||
|
|
||
| Both entries use `GITHUB_TOKEN` if set, else fall back to `gh auth token`. | ||
|
|
||
| ## Layout | ||
|
|
||
| Pure logic (`classify.ts` waiting-on state machine + SLA clocks, `route.ts` | ||
| assignment, `metrics.ts` scorecard, `glob.ts`) is fixture-tested and does no | ||
| I/O. `collect.ts` (GraphQL, read-only), `actions.ts` (REST mutations), | ||
| `discord.ts` (webhook) are the I/O seams; `sweep.ts` / `scorecard.ts` are the | ||
| entry points. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| /** | ||
| * Mutation planning + execution for the sweep. The sweep builds a plan of | ||
| * `Mutation` objects first (printable in dry-run mode), then executes them | ||
| * through the REST API. | ||
| */ | ||
|
|
||
| import type { GitHubClient } from './github' | ||
|
|
||
| export const MANAGED_LABELS: Array<{ | ||
| name: string | ||
| color: string | ||
| description: string | ||
| }> = [ | ||
| { | ||
| name: 'waiting-on: maintainer', | ||
| color: 'd93f0b', | ||
| description: 'The ball is in the maintainersβ court', | ||
| }, | ||
| { | ||
| name: 'waiting-on: author', | ||
| color: 'fbca04', | ||
| description: 'Waiting for the author to respond or update', | ||
| }, | ||
| { | ||
| name: 'ready-to-merge', | ||
| color: '0e8a16', | ||
| description: 'Approved, CI green, no conflicts', | ||
| }, | ||
| { | ||
| name: 'needs-repro', | ||
| color: 'e99695', | ||
| description: 'Needs a minimal reproduction before it can be worked on', | ||
| }, | ||
| { | ||
| name: 'has-pr', | ||
| color: 'bfd4f2', | ||
| description: 'An open PR references this issue', | ||
| }, | ||
| ] | ||
|
|
||
| const MANAGED_LABEL_NAMES = new Set(MANAGED_LABELS.map((l) => l.name)) | ||
|
|
||
| export type Mutation = | ||
| | { kind: 'assign'; number: number; assignee: string } | ||
| | { kind: 'comment'; number: number; body: string; note: string } | ||
| | { kind: 'add-labels'; number: number; labels: Array<string> } | ||
| | { kind: 'remove-label'; number: number; label: string } | ||
|
|
||
| export function describeMutation(m: Mutation): string { | ||
| switch (m.kind) { | ||
| case 'assign': | ||
| return `assign #${m.number} β @${m.assignee}` | ||
| case 'comment': | ||
| return `comment on #${m.number} (${m.note})` | ||
| case 'add-labels': | ||
| return `label #${m.number} + [${m.labels.join(', ')}]` | ||
| case 'remove-label': | ||
| return `label #${m.number} β [${m.label}]` | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Diff an item's current labels against the desired managed labels; labels | ||
| * outside MANAGED_LABELS are never touched. | ||
| */ | ||
| export function planLabelChanges( | ||
| number: number, | ||
| currentLabels: Array<string>, | ||
| desiredManaged: Array<string>, | ||
| ): Array<Mutation> { | ||
| const current = new Set( | ||
| currentLabels.filter((l) => MANAGED_LABEL_NAMES.has(l)), | ||
| ) | ||
| const desired = new Set(desiredManaged) | ||
| const mutations: Array<Mutation> = [] | ||
| const toAdd = [...desired].filter((l) => !current.has(l)) | ||
| if (toAdd.length > 0) { | ||
| mutations.push({ kind: 'add-labels', number, labels: toAdd }) | ||
| } | ||
| for (const label of current) { | ||
| if (!desired.has(label)) { | ||
| mutations.push({ kind: 'remove-label', number, label }) | ||
| } | ||
| } | ||
| return mutations | ||
| } | ||
|
|
||
| export async function ensureManagedLabels( | ||
| client: GitHubClient, | ||
| repo: string, | ||
| ): Promise<void> { | ||
| for (const label of MANAGED_LABELS) { | ||
| try { | ||
| await client.rest('POST', `/repos/${repo}/labels`, label) | ||
| } catch (error) { | ||
| // 422 = already exists; anything else should surface. | ||
| if (!(error instanceof Error) || !error.message.includes('422')) { | ||
| throw error | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| export async function executeMutations( | ||
| client: GitHubClient, | ||
| repo: string, | ||
| mutations: Array<Mutation>, | ||
| ): Promise<void> { | ||
| for (const m of mutations) { | ||
| switch (m.kind) { | ||
| case 'assign': | ||
| await client.rest( | ||
| 'POST', | ||
| `/repos/${repo}/issues/${m.number}/assignees`, | ||
| { | ||
| assignees: [m.assignee], | ||
| }, | ||
| ) | ||
| break | ||
| case 'comment': | ||
| await client.rest( | ||
| 'POST', | ||
| `/repos/${repo}/issues/${m.number}/comments`, | ||
| { | ||
| body: m.body, | ||
| }, | ||
| ) | ||
| break | ||
| case 'add-labels': | ||
| await client.rest('POST', `/repos/${repo}/issues/${m.number}/labels`, { | ||
| labels: m.labels, | ||
| }) | ||
| break | ||
| case 'remove-label': | ||
| await client.rest( | ||
| 'DELETE', | ||
| `/repos/${repo}/issues/${m.number}/labels/${encodeURIComponent(m.label)}`, | ||
| ) | ||
| break | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
π©Ί Stability & Availability | π Major | β‘ Quick win
π§© Analysis chain
π Script executed:
Repository: TanStack/ai
Length of output: 244
π Script executed:
Repository: TanStack/ai
Length of output: 50367
π Web query:
GitHub Actions workflow permissions write includes read discussions issues pull-requests repository permissions docsπ‘ Result:
In GitHub Actions, the
permissionskey allows you to define the access level of theGITHUB_TOKENfor a specific job or an entire workflow [1][2]. This is a security best practice that enables the principle of least privilege, ensuring the token has only the minimum access required to perform its tasks [1][3]. When you explicitly define thepermissionskey in your workflow file, all unspecified permissions are automatically set tonone, except for themetadatascope, which always retainsreadaccess [2]. The available permission scopes include, but are not limited to, the following [4]: contents: Manages repository code access. Thewritelevel allows you to push commits, create tags, and manage releases. issues: Manages issue tracking. Thewritelevel allows you to create, edit, close, or delete issues and add comments. pull-requests: Manages pull request operations. Thewritelevel permits actions such as creating or merging pull requests, as well as modifying PR labels, assignees, and descriptions. discussions: Manages GitHub Discussions. Thewritelevel allows you to create, edit, or delete discussions and comments. actions: Manages workflow execution and control. checks: Manages status checks and build results. deployments: Manages deployment status and infrastructure. packages: Manages GitHub Packages. pages: Manages GitHub Pages settings and publishing. When grantingwriteaccess, the token is granted the ability to modify or delete existing data within that specific scope, whereasreadaccess only permits viewing that data [2]. For example,contents: writeenables the token to push changes to the repository, which is necessary for tasks like automated version bumping or generating documentation [5][2]. For security, it is recommended to audit and limit these permissions, as broad default settings (if configured at the repository or organization level) may grant more access than your workflow requires [3]. You can manage these settings at the job or workflow level in your YAML configuration [2].Citations:
π Script executed:
Repository: TanStack/ai
Length of output: 30056
Grant the repo read scopes snapshot collection needs
collectSnapshot()reads open PRs, issues, and discussions.maintainer-scorecard.ymlonly grantscontents: read, so addissues: read,pull-requests: read, anddiscussions: readthere;maintainer-sweep.ymlalready hasissues: writeandpull-requests: write, so it only needsdiscussions: read.π Affects 2 files
.github/workflows/maintainer-scorecard.yml#L30-L31(this comment).github/workflows/maintainer-sweep.yml#L28-L31π€ Prompt for AI Agents