feat(ci): promote Trigger.dev tasks in lockstep with the ECS traffic cutover by TheodoreSpeaks · Pull Request #5725 · simstudioai/sim

TheodoreSpeaks · 2026-07-17T01:33:53Z

Summary

Move Trigger.dev staging + production deploys into CI (off the git-integration auto-deploy), deploying with --skip-promotion so new task runs keep executing the old version until we promote.
Promote the deployed version at the exact ECS traffic cutover: a new promote-trigger-* job polls CodeDeploy (AllowTraffic) and then runs trigger promote, so tasks and app cut over in lockstep instead of tasks flipping ~4–50 min early (the prod bake queue).
Correlate this push to its ECS deploy via the app image digest (build-amd64 publishes it as an artifact) → CodePipeline execution → CodeDeploy deployment id → AllowTraffic. ECR deploy tags are floating, so the digest is the only durable key.
Gate promotion on promote-images.promoted: when the stale-run guard skips the image retag, tasks correctly stay on the old version. If the app deploy fails or never cuts over, promote never fires and the job fails visibly.
New .github/scripts/wait-for-ecs-cutover.sh does the poll; verified end-to-end against real staging deploys (digest → d-… → AllowTraffic).

Prerequisites before this runs on staging/prod

Add CodePipeline + CodeDeploy read perms to the OIDC deploy roles (STAGING_AWS_ROLE_TO_ASSUME, AWS_ROLE_TO_ASSUME), which are managed outside the repo: codepipeline:ListPipelineExecutions/GetPipelineExecution/ListActionExecutions, codedeploy:GetDeployment/ListDeploymentTargets/GetDeploymentTarget.
Set the TRIGGER_ACCESS_TOKEN repo secret (a Trigger.dev PAT; spans all environments). Then disable the Trigger.dev git integration for staging/prod.

Type of Change

New feature (CI/CD)

Testing

Ran wait-for-ecs-cutover.sh against real completed staging deployments: correctly resolved digest → pipeline execution → CodeDeploy deployment → AllowTraffic (exit 0), and timed out cleanly on an unmatched digest (exit 1).
actionlint clean (only pre-existing custom-runner-label warnings); bun run lint and bun run check:api-validation:strict pass; YAML validated.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…cutover

vercel · 2026-07-17T01:33:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 19, 2026 9:00pm

cursor · 2026-07-17T01:34:04Z

PR Summary

Medium Risk
Changes production deploy sequencing for app and background tasks; mis-correlation or missing AWS read IAM could block promotion or delay cutover, but app deploy path is unchanged and stale-run guards limit accidental early promote.

Overview
Moves main/staging Trigger.dev deploys into CI and delays task promotion until the ECS app has finished its blue/green traffic shift, so background tasks and the app cut over together instead of tasks flipping minutes earlier.

A new deploy-trigger job uploads task versions with --skip-promotion (parallel to image build, not blocking app deploy). build-amd64 publishes the app image digest as an artifact; promote-images now exports promoted and retag_epoch when deploy tags move. promote-trigger runs only when promotion actually happened, polls via .github/scripts/wait-for-ecs-cutover.sh (digest + retag time → CodePipeline → CodeDeploy AllowTraffic on all targets), then runs trigger promote for the version from deploy-trigger. If ECS never cuts over or the stale-run guard skips retagging, promotion is skipped or the job fails visibly.

^{Reviewed by Cursor Bugbot for commit f7135b0. Bugbot is set up for automated code reviews on this repo. Configure here.}

greptile-apps · 2026-07-17T01:38:07Z

Greptile Summary

This PR coordinates Trigger.dev task promotion with the ECS traffic cutover. The main changes are:

Deploy Trigger.dev versions from CI without immediately promoting them.
Publish the app image digest and correlate it with the matching CodePipeline execution.
Wait for successful AllowTraffic events across all ECS deployment targets.
Promote the Trigger.dev version only after the app traffic shift.

Confidence Score: 5/5

The latest fixes look safe to merge.

The timestamp filter prevents old same-digest deployments from satisfying the normal cutover gate.
The cutover loop now requires successful AllowTraffic status for every returned ECS target.
No additional blocking issue met the scope for this follow-up review.

Important Files Changed

Filename	Overview
.github/scripts/wait-for-ecs-cutover.sh	Adds polling that resolves the matching CodeDeploy deployment and waits for every ECS target to complete its traffic shift.
.github/workflows/ci.yml	Adds staged Trigger.dev deployment and promotion jobs tied to the ECS image promotion workflow.

_{Reviews (3): Last reviewed commit: "fix(ci): widen promote-trigger job timeo..." | Re-trigger Greptile}

…erify all ECS targets

TheodoreSpeaks · 2026-07-19T19:19:10Z

@greptile review

TheodoreSpeaks · 2026-07-19T19:19:12Z

@cursor review

cursor · 2026-07-19T21:02:03Z

+        uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
+        with:
+          role-to-assume: ${{ github.ref == 'refs/heads/main' && secrets.AWS_ROLE_TO_ASSUME || secrets.STAGING_AWS_ROLE_TO_ASSUME }}
+          aws-region: ${{ github.ref == 'refs/heads/main' && secrets.AWS_REGION || secrets.STAGING_AWS_REGION }}


AWS creds expire before poll

High Severity

promote-trigger assumes the deploy role with the default ~60-minute session and never sets role-duration-seconds, while wait-for-ecs-cutover.sh is built to poll for up to 70 minutes (the prod bake-queue case). After the session expires, AWS calls fail; those errors are swallowed, so the script keeps retrying until timeout and never promotes. The long-wait path this change was written for leaves the app cut over while Trigger.dev stays on the old version.

Additional Locations (1)

.github/scripts/wait-for-ecs-cutover.sh#L26-L27

^{Reviewed by Cursor Bugbot for commit f7135b0. Configure here.}

TheodoreSpeaks · 2026-07-19T21:03:25Z

@greptile review

TheodoreSpeaks · 2026-07-19T21:03:26Z

@cursor review

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f7135b0. Configure here.}

cursor · 2026-07-19T21:06:47Z

+    log "No matching post-retag pipeline execution yet; retry in ${POLL_INTERVAL}s (remaining $(remaining)s)"
+    sleep "$POLL_INTERVAL"
+  fi
+done


Same digest blocks task promotion

Medium Severity

Phase A requires a CodePipeline execution for the app digest with startTime at/after retag_epoch. That rejects the historical same-digest run this filter was added for, but retagging latest/staging to a digest it already points at is often a no-op for ECR (no new EventBridge/pipeline). The poll then never finds a post-retag execution and times out, so the skip-promoted Trigger.dev version from deploy-trigger is never promoted—especially on re-runs after promote-images already moved the tag, or any push that reuses the app image digest (Trigger tasks live in background/ and are not in the final app image).

Additional Locations (1)

.github/workflows/ci.yml#L514-L522

^{Reviewed by Cursor Bugbot for commit f7135b0. Configure here.}

feat(ci): promote Trigger.dev tasks in lockstep with the ECS traffic …

30e1270

…cutover

cursor Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread .github/workflows/ci.yml

Comment thread .github/scripts/wait-for-ecs-cutover.sh

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread .github/scripts/wait-for-ecs-cutover.sh Outdated

Comment thread .github/scripts/wait-for-ecs-cutover.sh Outdated

vercel Bot deployed to Preview July 17, 2026 01:38 View deployment

fix(ci): harden Trigger.dev cutover gate — reject stale executions, v…

b207a52

…erify all ECS targets

cursor Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread .github/workflows/ci.yml

Comment thread .github/workflows/ci.yml Outdated

vercel Bot deployed to Preview July 19, 2026 19:22 View deployment

fix(ci): widen promote-trigger job timeout above the poll budget

f7135b0

vercel Bot deployed to Preview July 19, 2026 21:00 View deployment

cursor Bot reviewed Jul 19, 2026

View reviewed changes

Uh oh!

Conversation

TheodoreSpeaks commented Jul 17, 2026

Summary

Prerequisites before this runs on staging/prod

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

TheodoreSpeaks commented Jul 19, 2026

Uh oh!

TheodoreSpeaks commented Jul 19, 2026

Uh oh!

Uh oh!

Uh oh!

cursor Bot Jul 19, 2026

Choose a reason for hiding this comment

AWS creds expire before poll

Uh oh!

TheodoreSpeaks commented Jul 19, 2026

Uh oh!

TheodoreSpeaks commented Jul 19, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 19, 2026

Choose a reason for hiding this comment

Same digest blocks task promotion

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 17, 2026 •

edited

Loading

cursor Bot commented Jul 17, 2026 •

edited

Loading

greptile-apps Bot commented Jul 17, 2026 •

edited

Loading