Skip to content

Stale auto-discovered PRs: re-test, ping the author, close only after an unanswered ping - #10014

Open
probonopd wants to merge 3 commits into
masterfrom
claude/auto-close-discovered
Open

probonopd wants to merge 3 commits into
masterfrom
claude/auto-close-discovered

Conversation

@probonopd

@probonopd probonopd commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Reworked after discussion: instead of re-test-then-close, a three-step ladder that pings the application's author before anything is closed. 3,504 of the 4,692 open pull requests are auto-discovered ones whose test failed; nothing re-tests, reminds or closes them, and only about 5 % ever got a human reply (the first failing test result mentions the author once, months ago).

New .github/workflows/stale-discovered.yml (manual only, dry run by default; limit 20, days 90, close_days 60) with code/stale-discovered.sh. One step per pull request and run, each on a fresh result:

  1. Re-test: an open auto-discovered PR with an error-* label, no comment by anyone but github-actions, and a last test result older than 90 days is tested once more (closed and reopened with SCREENSHOT_UPLOAD_TOKEN, like /retest: current workflow files, newest release); label retested-stale.
  2. Ping: once the re-test's result is in and it still fails, one friendly comment that @mentions the GitHub account the AppImage comes from (upstream-owner.sh; never a maintainer or a bot), points at the result above, asks for /retest when a fixed release is out, and says that the PR is closed in 60 days otherwise; label pinged-author.
  3. Close: 60 days after the ping with still no human comment, closed with a short comment (proposed again on a new release; an own PR is welcome); label auto-closed, the other two removed.

A PR with a comment by a human, at any step, is never touched. One that passes the re-test keeps screenshot-ok and shows up in the health report as waiting for a maintainer. manual-check-needed ones are left alone.

code/discover-apps.sh: a PR labeled auto-closed no longer counts as "proposed before" (never again); its repository counts as checked without success on the closing day, so it is skipped for 90 days but checked again as soon as it is pushed to (the recent mode's rule), and a fixed AppImage comes back by itself.

Suggested use: dry run, look at the list, then real runs with a limit of 20 to 50 twice a week. Each real run re-tests up to limit PRs (that many Test runs) and mentions up to limit people, which is why the limit is per run.

Checked: bash -n, shellcheck, actionlint; a read-only dry run against the real repository (see the comments below for its output; today nearly all results are younger than 90 days because everything was re-tested in late September, so the ladder starts in late December unless days is lowered). AGENTS.md describes it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DZvaN9tm5taf5avUvVtmyq


Generated by Claude Code

…very re-proposes them on a new release

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZvaN9tm5taf5avUvVtmyq
@probonopd

Copy link
Copy Markdown
Member Author

Result of the read-only dry run against the real repository (limit 8, 90 days):

3443 open auto-discovered pull requests with a failed test
Re-tested (would): 0; closed (would): 0 (limit 8, results older than 90 days).
Left alone: 176 with a comment by a human, 3267 with a result newer than 90 days, 0 re-tested and waiting for the result.

So right now it would do nothing: nearly all of these PRs were re-tested in late September (e.g. the oldest, #4210, has a result from 2026-09-27), and 176 have a human comment and are never touched. With the 90-day default the first closings would come in late December. Running it earlier means lowering days (e.g. 45), which is a judgement call: the PRs then get one more re-test anyway before anything is closed, so a lower value mostly shortens the wait, not the fairness.


Generated by Claude Code

… an unanswered ping

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZvaN9tm5taf5avUvVtmyq
@probonopd probonopd changed the title Close stale auto-discovered pull requests after a last re-test Stale auto-discovered PRs: re-test, ping the author, close only after an unanswered ping Oct 11, 2026
…e API quota runs low

Walking the comments of all 3400 failed auto-discovered pull requests cost
one request each, more than the 1000 an hour the workflow token has. The
list already carries updated_at: a pull request with any activity newer than
--days cannot have an older result, so its comments are not fetched. Every
request goes through api(), which counts them and stops the run (exit 4)
when fewer than RESERVE requests are left; the summary prints the count.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZvaN9tm5taf5avUvVtmyq

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant