ci: skip heavy test suites on bot version-bump PRs (PER-9560)#2284
Open
AkashBrowserStack wants to merge 4 commits into
Open
ci: skip heavy test suites on bot version-bump PRs (PER-9560)#2284AkashBrowserStack wants to merge 4 commits into
AkashBrowserStack wants to merge 4 commits into
Conversation
Automated version-bump PRs (opened by version-bump.yml from `release/<version>` branches) only touch lerna.json + package.json and have no code impact, yet they block ~1h on the full Linux + Windows test matrices. Skip those suites for these PRs. Skip only when BOTH hold: the head branch is `release/*` AND the PR diff is confined to the version files version-bump.yml is allowed to commit (lerna.json + packages/**/package.json), computed by a `changes` job via dorny/paths-filter (predicate-quantifier: every). So a `release/*` PR that touches source still runs the full suite — the skip can't be abused to land untested code behind a green-looking "skipped" check. - Gate `build`, `test`, `regression` in test.yml and `build` + `test` in windows.yml on `!(startsWith(github.head_ref,'release/') && needs.changes.outputs.version_only=='true')`. - Branch pattern is `release/*` (slash) — the convention set by version-bump.yml — not `release-*` (hyphen) as the ticket guessed. - Done as a job-level `if`, not an `on:` branch filter: a skipped job posts a "skipped" check that satisfies required status checks, whereas an `on:`-level skip leaves required checks "pending" and would block the release PR from merging. - lint, typecheck, and Semgrep (all ~1m) keep running on release PRs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rishigupta1599
approved these changes
Jun 15, 2026
AkashBrowserStack
changed the title
…PER-9560) Refines the version-bump test skip from the earlier version-only diff approach to a check on the PR author. Skipping now requires the head branch to be `release/*` AND the PR to be opened by `github-actions[bot]`. The bot-identity check is native to GitHub — no extra `changes` job that could fail and silently skip tests — and fully closes the "a human names a branch `release/*` to dodge CI" hole; only the release bot's PRs skip. - Replace the per-job `if` with `!(startsWith(github.head_ref,'release/') && github.event.pull_request.user.login=='github-actions[bot]')` on build/test/regression (test.yml) and build/test (windows.yml). - Drop the `changes` job and the `dorny/paths-filter` dependency. - Keep a least-privilege `permissions: contents: read` block on both workflows (no longer need `pull-requests: read`). - Remove the explanatory comments per review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Author + branch alone is not a safe skip gate: github.event.pull_request .user.login is fixed at PR creation, so commits pushed to a release/* branch *after* the bot opens the PR keep the same author and branch and would skip CI on untested source. Re-add the `changes` job (dorny/paths-filter, SHA-pinned to v3.0.3) and require version_only on every heavy job, so the skip now needs all three: branch is release/* AND author is github-actions[bot] AND the diff is confined to lerna.json + packages/**/package.json. paths-filter re-evaluates the whole PR diff on each push, so any non-version file flips version_only to false and tests run again. - Restore `permissions: pull-requests: read` (paths-filter reads PR files). - build needs [changes]; test/regression need [build, changes]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ER-9560) The dorny/paths-filter gate never matched: predicate-quantifier 'every' means a file must match *every* pattern, so no file could be both lerna.json AND packages/**/package.json -> version_only was always false -> the skip never fired. Replace it with an explicit check that lists the PR's changed files via `gh api .../pulls/N/files` and confirms every one is lerna.json or a top-level packages/<pkg>/package.json. Fail-safe: any API error or a non-version file -> version_only=false -> full CI runs. Also drops the third-party action (and its SHA pin / Node-20 deprecation). Verified end-to-end on a fork of percy/cli: - bot version-bump PR (release/*, version-only diff) -> Build/Test/ Regression (Linux) and Build/Test (Windows) reported "skipped". - normal PR (non-version file) -> heavy jobs ran. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Automated version-bump PRs — opened by
version-bump.yml— only touchlerna.json+packages/**/package.jsonand have no code impact, yet they currently block ~1h waiting on the full Linux (test.yml) and Windows (windows.yml) test matrices (PER-9560).This skips those heavy suites for the release bot's version-bump PRs, while keeping them fully in force everywhere else.
The skip fires only when ALL THREE hold (evaluated per push):
release/*(the convention set byversion-bump.yml— a slash, not therelease-*hyphen in the ticket),github-actions[bot](github.event.pull_request.user.login), andchangesjob lists the PR's changed files (gh api .../pulls/N/files) and confirms every one islerna.jsonor a top-levelpackages/<pkg>/package.json.The diff check is the load-bearing one and is re-evaluated on every push, so if a source commit lands on a
release/*branch after the bot opens the PR,version_onlybecomesfalseand the full suite runs — author/branch are fixed at PR-creation time and can't mask a later source change. The check is fail-safe: any API error (or a non-version file) →version_only=false→ full CI runs.What's skipped vs kept
release/*PR, version-only difftest.yml→build,test(17-pkg matrix),regressionwindows.yml→build,test(17-pkg matrix)lint,typecheck,SemgrepWhy a job-level
if, not anon:branch filterThe test jobs are required status checks. A skipped job posts a "skipped" conclusion that satisfies branch protection, whereas an
on:-level skip leaves required checks stuck "pending" and would block the release PR from ever merging.Also adds a least-privilege
permissions:block (contents: read+pull-requests: read, the latter for listing PR files).Verified end-to-end (on a fork of percy/cli)
github-actions[bot],release/1.32.0-beta.10, version-only diff) →Build/Test/Regression(Linux) andBuild/Test(Windows) reportedskipped;changescomputedversion_only=true; lint/typecheck/Semgrep ran.changescomputedversion_only=falseand the heavyBuild/Testjobs ran.Test plan
release/*branch →version_onlyflips false → full matrices run.release/*branch → runs (author isn't the bot).masterpush /workflow_dispatch→ unchanged; everything runs.🤖 Generated with Claude Code