Skip to content

test(js): stop asserting exact timer tick counts in flaky JS timer tests#4767

Merged
dbrattli merged 2 commits into
mainfrom
fix/flaky-timer-tests
Jul 19, 2026
Merged

test(js): stop asserting exact timer tick counts in flaky JS timer tests#4767
dbrattli merged 2 commits into
mainfrom
fix/flaky-timer-tests

Conversation

@dbrattli

Copy link
Copy Markdown
Collaborator

Timer.Elapsed.Subscribe works failed on the windows-latest JavaScript leg of an unrelated PR (run) with Expected: 10 - Actual: 5. It is a wall-clock race, and so is its sibling Timer with AutoReset = true works.

The race

let t = new Timers.Timer(50.)
let disp = t.Elapsed.Subscribe(fun ev -> res := !res + 5)
t.Start()
do! Async.Sleep 125
disp.Dispose()
do! Async.Sleep 50
equal 10 !res          // exactly two 50ms ticks

The test asserts the handler ran exactly twice inside a 125ms sleep. The second tick is due at 100ms, so there are 25ms of slack. On a loaded CI runner — where Windows timer resolution is ~15.6ms and scheduling is coarse — that tick slips past the window, one tick arrives, and res is 5. The same job also reported the Main.js file itself hitting its 20s timeout, which is the signature of a runner that was simply slow at that moment.

Nothing about the count of 2 is part of the contract being tested; it is an artifact of the sleep being 2.5× the interval.

The fix

Assert the semantics each test is named for:

  • the handler fires repeatedly (more than one tick — this is what AutoReset = true means, and what a live subscription means), and
  • it stops firing once the timer is stopped / the subscription is disposed.

The counter is read after Stop()/Dispose(), so a tick cannot land between the read and the unsubscribe and make the second assertion flaky in turn.

This is the same tolerance the neighbouring Assigning an event to a variable works (#1863) already uses — 200ms sleep, at least two ticks (acc > 2 |> equal true) — which has been stable, so the new budget is a proven one in this file rather than a guess.

Timer with AutoReset = false works is deliberately left alone: it asserts exactly one tick, which is the invariant it exists to check (AutoReset off ⇒ fires once), not a timing artifact.

Testing

Full JavaScript suite: 3049 passing, 0 failed. The two rewritten tests take ~300ms each, up from ~175ms — negligible against the 20s per-file budget.

🤖 Generated with Claude Code

@dbrattli dbrattli changed the title chore(tests): stop asserting exact timer tick counts in JS timer tests chore(js): stop asserting exact timer tick counts in JS timer tests Jul 13, 2026
@dbrattli dbrattli changed the title chore(js): stop asserting exact timer tick counts in JS timer tests test(js): stop asserting exact timer tick counts in JS timer tests Jul 13, 2026
`Timer.Elapsed.Subscribe works` and `Timer with AutoReset = true works` both
started a 50ms timer, slept 125ms, and asserted the handler had run exactly
twice. How many ticks land inside a fixed sleep is a wall-clock race: the second
tick is due at 100ms, leaving 25ms of slack, and on a loaded CI runner — where
timer resolution is coarse — it slips past the window and only one arrives.
`Timer.Elapsed.Subscribe works` failed this way on the windows-latest JS leg.

Assert the semantics the tests are named for instead: the handler fires
repeatedly, and stops firing once the timer is stopped or the subscription is
disposed. The counter is read after Stop/Dispose so a tick cannot land between
the read and the unsubscribe. This is the same tolerance the neighbouring
`Assigning an event to a variable works` already uses (200ms sleep, at least two
ticks), which has been stable.

`Timer with AutoReset = false works` is left alone: exactly one tick is the
invariant it exists to check, not an artifact of timing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli
dbrattli force-pushed the fix/flaky-timer-tests branch from 7588072 to fa6a406 Compare July 13, 2026 07:28
@dbrattli dbrattli changed the title test(js): stop asserting exact timer tick counts in JS timer tests test(js): stop asserting exact timer tick counts in flaky JS timer tests Jul 13, 2026
`equal` is `equal expected actual`, so passing the observed counter first
made failure messages print the two sides reversed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dbrattli
dbrattli merged commit 0b5e78e into main Jul 19, 2026
32 checks passed
@dbrattli
dbrattli deleted the fix/flaky-timer-tests branch July 19, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant