Skip to content

Relax timing assertions in WaitForCompletion to fix flaky TerminateScheduledOrchestration#318

Open
usemam wants to merge 2 commits into
microsoft:mainfrom
usemam:fix/terminate-scheduled-flake
Open

Relax timing assertions in WaitForCompletion to fix flaky TerminateScheduledOrchestration#318
usemam wants to merge 2 commits into
microsoft:mainfrom
usemam:fix/terminate-scheduled-flake

Conversation

@usemam

@usemam usemam commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Relaxes LastUpdatedTime > CreatedTime and CompletedTime > CreatedTime in TestInstance.WaitForCompletion to >=, fixing a recurring CI flake in TerminateScheduledOrchestration.

Why

TerminateScheduledOrchestration terminates a Pending (scheduled) orchestration immediately after creation. In the SQL flow:

  • CreateInstance sets CreatedTime via the column default DEFAULT SYSUTCDATETIME().
  • TerminateInstance takes the Pending → Terminated short-circuit branch (logic.sql:511-521) and sets both LastUpdatedTime and CompletedTime to @now = SYSUTCDATETIME().

When the two SYSUTCDATETIME() reads land within a single OS tick — which happens on CI's SQL Server 2019 Linux container, where the clock resolution is coarser than on Windows/macOS — both timestamps end up identical, and the strict > assertions fail. Same failure was observed on run 24819180387
(https://github.com/usemam/durabletask-mssql/actions/runs/24819180387) (2026-04-23) and again on 27509421841 (https://github.com/usemam/durabletask-mssql/actions/runs/27509421841) (2026-06-14), both with the same ~22 ms fast-fail signature.

LastUpdatedTime == CreatedTime is a legitimate state for an orchestration terminated before it ever ran, so >= is the semantically correct check.

Test plan

  • TerminateScheduledOrchestration passes locally (20/20 runs)
  • Full DurableTask.SqlServer.Tests suite passes on CI - see build log
  • No regressions in other tests that flow through WaitForCompletion

@usemam usemam marked this pull request as ready for review June 14, 2026 23:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant