From ee7258fe329fb5c2f02060561cc608efc3625de8 Mon Sep 17 00:00:00 2001 From: Dion McMurtrie Date: Tue, 18 Aug 2026 13:41:05 +1000 Subject: [PATCH] Publish an immutable build tag alongside the mutable one The image build publishes exactly one tag - `latest` on groups-api-fix, the branch name elsewhere - and both are mutable. So the digest we deploy has no durable name of its own, which caused two distinct problems on 2026-08-18: * Build 15968 was the image deployed to production. Its digest carried no tag at all until one was imported by hand, leaving the manifest production is pinned to eligible for garbage collection. * Build 15969 rebuilt the same commit (a19843d9), produced a different digest, and took ownership of `latest`. From that point the registry could not answer which image production was running. Add a `build-` tag that is never reused, and verify it landed rather than trusting the exit status - the manual repair used for 15968 returned success without creating the tag. This does not change what the chart pins. `values.yaml` still carries the digest, because a tag is a mutable pointer someone can move. The build tag exists so the manifest is permanent and findable, not so it can be pinned. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018aYBXcyiozxQQGhzzhS7KG --- azure-pipelines.yml | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 87458225..33884060 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -54,3 +54,45 @@ stages: publishJUnitResults: true testResultsFiles: '**/surefire-reports/TEST-*.xml' goals: 'clean install dockerfile:build dockerfile:push' + - script: | + set -euo pipefail + IMAGE="$(acr_repo)/release-validation-framework" + BUILD_TAG="build-$(Build.BuildId)" + + # `$(imageTag)` above is mutable by design: the next build of this + # branch takes it over. That is fine for convenience, but it means + # the digest we actually deploy has no durable name, and an + # untagged manifest is a garbage-collection candidate. Publish a + # second tag that is never reused, so every build stays findable + # and any prior build is a rollback target by construction. + # + # Named after the BUILD, not the commit: builds 15968 and 15969 + # both built a19843d9 and produced DIFFERENT digests, so a + # commit-SHA tag would be silently overwritten by a rebuild. + # + # Not named after $(Build.BuildNumber) either - tags beginning + # `rvf-` (which is the build-number prefix) were observed on + # 2026-08-18 failing to be created in this registry while the + # command reported success. + docker tag "$IMAGE:$(imageTag)" "$IMAGE:$BUILD_TAG" + docker push "$IMAGE:$BUILD_TAG" + + # Read the tag back. A publish that reports success without + # creating anything is the precise failure this step exists to + # engineer around, so do not trust the exit status. RepoDigests is + # populated only once the registry has accepted the manifest. + DIGEST=$(docker image inspect "$IMAGE:$BUILD_TAG" \ + --format '{{index .RepoDigests 0}}') + if [ -z "$DIGEST" ]; then + echo "##vso[task.logissue type=error]$IMAGE:$BUILD_TAG was not published" + exit 1 + fi + + echo "##[section]Immutable image published" + echo " tag: $IMAGE:$BUILD_TAG" + echo " digest: $DIGEST" + echo "" + echo " To deploy this build, pin it by DIGEST in the chart's values.yaml:" + echo " imageName: $DIGEST" + echo " (a tag is a mutable pointer; the digest is the guarantee)" + displayName: Publish immutable build tag