Repository navigation
Merge sync PRs to prerelease right away and retry failed backports - #2293
Merged
Merged
Conversation
Since the v1.10 release merge, every auto-backport to prerelease has failed, and the fixes reached prerelease only through manual backports weeks later. Most failures were an ordering race rather than real divergence: sync PRs stay open for minutes to hours, and a second PR to main touching the same file is cherry-picked onto a prerelease that does not have the first change yet. #2141 conflicted on a freeze file because the sync PR of #2138 merged 19 seconds after it. Failed backports are never retried, so each one then breaks every later PR on the same files (#2227 after #2141, #2258 and #2257 after #2235). The workflow now merges a clean sync PR immediately and publishes prerelease itself, since a merge made with GITHUB_TOKEN does not fire the push event publish.yml listens to. A failed backport labels the source PR backport-failed, so the missing changes are listable instead of buried in bot comments. After each sync merge, and after any manual merge into prerelease, the workflow comments /sync-prerelease on the oldest labeled PR, so a backport that only waited on an earlier change applies on its own. No concurrency group: GitHub cancels a pending run when another queues, which would drop backports silently when several PRs merge within minutes. The cost of merging without review is that a main-only PR now needs the no-sync-prerelease label before it merges, where it used to be enough to leave the sync PR unmerged.
A sync PR can also fail to merge when prerelease moves between the cherry-pick and the merge. That backport now gets the backport-failed label too, instead of being missed by the list and the retry. The open sync PR is left for a manual merge. Publishing no longer depends on the label cleanup succeeding after the merge, since nothing else publishes a GITHUB_TOKEN merge.
cderv
force-pushed
the
ci/backport-auto-merge
branch
from
October 16, 2026 13:27
7efe69b to
6875e06
Compare
1 of 3 tasks
gh pr list returns at most 30 results by default, newest first, so with more labeled PRs the sort only saw the newest 30 and the oldest backport, often the one the others wait on, would never be retried.
Contributor
|
Successfully created backport PR for |
Merged
3 of 5 tasks
github-actions Bot
added a commit
that referenced
this pull request
Oct 16, 2026
…2293) (#2295) * Merge sync PRs to prerelease right away and retry failed backports Since the v1.10 release merge, every auto-backport to prerelease has failed, and the fixes reached prerelease only through manual backports weeks later. Most failures were an ordering race rather than real divergence: sync PRs stay open for minutes to hours, and a second PR to main touching the same file is cherry-picked onto a prerelease that does not have the first change yet. #2141 conflicted on a freeze file because the sync PR of #2138 merged 19 seconds after it. Failed backports are never retried, so each one then breaks every later PR on the same files (#2227 after #2141, #2258 and #2257 after #2235). The workflow now merges a clean sync PR immediately and publishes prerelease itself, since a merge made with GITHUB_TOKEN does not fire the push event publish.yml listens to. A failed backport labels the source PR backport-failed, so the missing changes are listable instead of buried in bot comments. After each sync merge, and after any manual merge into prerelease, the workflow comments /sync-prerelease on the oldest labeled PR, so a backport that only waited on an earlier change applies on its own. No concurrency group: GitHub cancels a pending run when another queues, which would drop backports silently when several PRs merge within minutes. The cost of merging without review is that a main-only PR now needs the no-sync-prerelease label before it merges, where it used to be enough to leave the sync PR unmerged. * Label sync PRs that fail to merge A sync PR can also fail to merge when prerelease moves between the cherry-pick and the merge. That backport now gets the backport-failed label too, instead of being missed by the list and the retry. The open sync PR is left for a manual merge. Publishing no longer depends on the label cleanup succeeding after the merge, since nothing else publishes a GITHUB_TOKEN merge. * Retry the oldest failed backport even past 30 labeled PRs gh pr list returns at most 30 results by default, newest first, so with more labeled PRs the sort only saw the newest 30 and the oldest backport, often the one the others wait on, would never be retried. (cherry picked from commit 35c61f4) Co-authored-by: Christophe Dervieux <christophe.dervieux@gmail.com>
1 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since the v1.10 release merge, the automatic backport to
prereleasefailed for every PR merged tomain(#2141, #2227, #2235, #2257, #2258, #2222). The changes reachedprereleaseonly through manual backports later (#2271, #2290), #2141 about two months after it merged.Root Cause
Most failures were not real divergence between the branches but an ordering race. A sync PR stays open on
prereleaseuntil someone merges it, from minutes to many hours. When a second PR tomaintouches the same file in the meantime, it is cherry-picked onto aprereleasethat does not have the first change yet, and conflicts._freeze/docs/authoring/tables/because the sync PR of Add alt text to figures on authoring pages #2138 ([Sync to prerelease] Add alt text to figures on authoring pages #2142) merged 19 seconds after it.styles.cssbecause the sync PRs of Move "Choose your tool" out of the tools-tabset tablist #2233 and Stop screen-reader-only code blocks from being scrollable regions #2234 merged 20 minutes later.A failed backport is never retried, so it then breaks every later PR on the same files: #2227 failed on #2141, #2258 and #2257 on #2235. Only #2222 was a real conflict, as #2272 had already changed the same page on
prerelease.Fix
When the cherry-pick applies cleanly, the workflow now merges the sync PR immediately and publishes
prereleaseby callingpublish.yml, as a merge made withGITHUB_TOKENdoes not fire the push event. It publishes the branch tip, not the merge commit, so two syncs landing close together still deploy the latest content.When the cherry-pick fails, the source PR gets the new
backport-failedlabel, sogh pr list --label backport-failed --state mergedlists what is missing fromprerelease. A sync PR that fails to merge, becauseprereleasemoved between the cherry-pick and the merge, gets the label too and is left open for a manual merge.After each sync merge, and after any PR merged by hand into
prerelease, the workflow comments/sync-prereleaseon the oldestbackport-failedPR. A backport that only waited on an earlier change then applies on its own, and the chain moves to the next one. It stops at the first PR that fails again, which needs a manual backport and the label removed afterwards.No concurrency group is used: GitHub cancels a pending run when another one queues, which would drop a backport silently when several PRs merge within a few minutes.
As sync PRs are no longer reviewed before merging, a PR that must not reach
prerelease(like #1983 for v1.9 only) now needs theno-sync-prereleaselabel before it is merged intomain. Leaving the sync PR unmerged is no longer possible.Direct pushes to
mainare still not synced. That is left for another PR.Test Plan
zizmorandactionlinton the workflow: no new findings in the CI auditghqueries used by the workflow against this repo, read-only (label lookup, oldest labeled PR, empty result)backport-failedlabelmaingets its sync PR merged andprereleasepublished