Skip to content

ci: run workflows on the release branch and restore reproducible image builds - #1028

Open
kvinwang wants to merge 3 commits into
release/v0.5.xfrom
ci/enable-release-branch-checks
Open

ci: run workflows on the release branch and restore reproducible image builds#1028
kvinwang wants to merge 3 commits into
release/v0.5.xfrom
ci/enable-release-branch-checks

Conversation

@kvinwang

@kvinwang kvinwang commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Problem

Two separate defects on release/v0.5.x, one of which hides the other.

1. The branch has no CI at all

release/v0.5.x was cut from v0.5.11 and carries that eras workflow files, whose branch filters are [ master, next, dev-* ] (and [ master, main ] for spdx-check.yml). None match release/v0.5.x, so creating the branch triggered zero workflow runs.

This cannot be fixed from next. For both push and pull_request, GitHub runs the workflow files as they exist on the branch being built, so adding release/** to the filters on next (already done) only helps release branches cut from next later. The fix has to land here. master no longer exists at all, so these filters currently name two branches that cannot match and one that is gone.

2. The gateway and KMS builder images are not reproducible

Once CI does run, docker-build-check fails on Verify builder pinned packages for both gateway and kms:

-openssl=3.5.5-1~deb13u1          -musl:amd64=1.2.5-3          -protobuf-compiler=3.21.12-11
+openssl=3.5.6-1~deb13u2          +musl:amd64=1.2.5-3.1~deb13u1 +protobuf-compiler=3.21.12-11+deb13u1

This is not benign upstream churn — it means rebuilding the same commit today produces a different image than the pinned list records, which defeats the point of pinning on a project whose security argument rests on measured, verifiable images.

The cause is build/shared/pin-packages.sh. It hardcodes bookworm and only overwrites /etc/apt/sources.list. gateway/dstack-app/builder and kms/dstack-app/builder both build FROM rust:1.92.0, which is trixie, and trixie ships its sources via deb822 at /etc/apt/sources.list.d/debian.sources. That file is left untouched, so apt keeps resolving from live deb.debian.org and the snapshot pin is bypassed entirely. verifier/builder uses rust:1.92.0-bookworm explicitly, which is why it is unaffected and passes.

Fix

Three commits:

  • ci: run workflows on the release branch — point the six filtered workflows at release/** (a glob rather than a literal, so the file stays correct if this line is ever branched further).
  • style: fix trailing whitespace in spdx-check workflow — trailing whitespace and a missing EOF newline that have been in that file since v0.5.11. prek only inspects changed files, so touching the file surfaced them.
  • fix(ci): lock snapshot for Debian 13 (trixie) base images — backport of 6a345f75e (chore(ci): bump builder pinned packages for Debian upstream drift #688) plus its follow-up 6a87118b9 from next, squashed into one commit since they are two halves of the same fix (the follow-up adds the # shellcheck source=/dev/null directive that SC1091 requires for the . /etc/os-release). Detects VERSION_CODENAME from /etc/os-release, wipes both the legacy and deb822 source layouts, and rewrites sources.list against the snapshot for the detected suite.

Confirmed: no pinned-package list needed regenerating. gateway and kms both went green on this branch with the lists untouched, which is the proof the section below predicted.

No pinned-package list is regenerated, deliberately. The lists on this branch still hold the original snapshot-correct values (openssl=3.5.5-1~deb13u1, musl:amd64=1.2.5-3, protobuf-compiler=3.21.12-11) — nobody ever bumped them here to paper over the drift, unlike on next where 6a345f75e had to revert exactly such a bump. With the snapshot pin working again, apt should resolve back to the versions already recorded. This was the falsifiable prediction, and it held: gateway 8m27s and kms 12m3s both pass with the lists unchanged. verifier (11m18s) also passes, as expected — it builds FROM rust:1.92.0-bookworm and was never affected.

The script hunk was applied from 6a345f75e unchanged — this branchs copy of pin-packages.sh is byte-identical to the pre-fix version, so there was nothing to adapt.

Verification

  • Filters: all six workflow files parse; resolved triggers are push/pull_request on [release/**] uniformly. No master / next / dev-* branch filter remains under .github/workflows/.
  • bash -n build/shared/pin-packages.sh passes, and the resulting file is byte-identical to next's dstack/build/shared/pin-packages.sh.
  • Reproducibility restored: gateway and kms moved fail -> pass on the pinned-package verification step with no change to the pinned lists themselves.
  • Before the pin fix, this PR already established that the v0.5.11-era workflows still work on current runners: rust-checks (9m14s), sdk-tests (7m20s), reuse-lint (12s), prek (29s) and the mkosi build job all pass. Only gateway and kms failed, and only on the pinned-package step.
  • Tag-triggered release workflows (gateway-release.yml, kms-release.yml, rust-sdk-release.yml, simulator-release.yml, verifier-release.yml) carry no branch filter and are untouched.

Follow-up

Once green, release/** can take a required_status_checks ruleset with rust-checks, sdk-tests, reuse-lint, prek. That is deliberately not part of this PR — adding the rule before CI dispatches on the branch would leave every backport PR pending forever, the same trap #1025 had to pre-empt for the branch rename.

release/v0.3.x must be excluded from that ruleset regardless: it carries only rust.yml, so the other three contexts can never report there.

Copilot AI lite review requested due to automatic review settings August 7, 2026 14:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates CI workflow branch filters on the release/v0.5.x line so that GitHub Actions runs on release branches (which currently receive no CI due to legacy filters targeting master, next, and dev-*).

Changes:

  • Updated push and pull_request branch filters in six workflows to run only on release/**.
  • Removed stale branch filter references (master, next, dev-*, and main in spdx-check.yml) from workflow triggers.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
.github/workflows/vmm-ui.yml Switches CI triggers to release/** so VMM UI build runs on release branches.
.github/workflows/spdx-check.yml Switches reuse-lint triggers to release/** so SPDX/REUSE checks run on release branches.
.github/workflows/sdk.yaml Switches SDK tests triggers to release/** so SDK CI runs on release branches.
.github/workflows/rust.yml Switches Rust checks triggers to release/** so core Rust CI runs on release branches.
.github/workflows/prek-check.yml Switches Prek checks triggers to release/** so Prek CI runs on release branches.
.github/workflows/docker-build-check.yml Switches Docker build check triggers to release/** so container build validation runs on release branches.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kvinwang kvinwang changed the title ci: run workflows on the release branch ci: run workflows on the release branch and restore reproducible image builds Aug 7, 2026
@kvinwang
kvinwang force-pushed the ci/enable-release-branch-checks branch from 6b339dc to 8737b47 Compare August 7, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants