Skip to content

fix(cli): size the vitest timeout to what the suite actually does - #88

Merged
thecodedrift merged 1 commit into
mainfrom
chore/vitest-timeout
Aug 7, 2026
Merged

fix(cli): size the vitest timeout to what the suite actually does#88
thecodedrift merged 1 commit into
mainfrom
chore/vitest-timeout

Conversation

@thecodedrift

Copy link
Copy Markdown
Member

The CLI test suite fails a rotating handful of tests on every run, always with Test timed out in 5000ms and never an assertion failure. It is not flaky code. The timeout is sized below the work.

Much of this suite is integration-shaped rather than unit-shaped. Tests in check, runtime-check, init, and onboard spawn the real built CLI through execFile("node", [binPath, ...]), and several lay out a fixture tree and run git init first. One cold CLI spawn measures ~0.8s on its own, so a test doing four or five of them sits at 3-4s before any load at all. Vitest runs test files in parallel workers that compete for CPU, so whichever tests happen to land together are the ones that tip over, which is why the failing set changes run to run.

This raises testTimeout and hookTimeout to 20s. That swallows the contention without hiding a genuine hang.

Evidence it predates every open branch

Checked out origin/main at d7abf6a with no local changes and no merge, ran the suite, and got 8 failures, all timeouts. So this is the suite's own sizing rather than anything a current branch introduced.

With the change, packages/cli passes 427/427 across three consecutive runs.

Note for reviewers

packages/cli has no vitest.config.ts; vitest reads vite.config.ts, which had no test block, so the 5s default applied. The new block is test-only configuration and does not affect the published bundle.

Worth knowing separately: running vitest without building first fails ~119 tests, because the integration tests spawn dist/index.js. turbo handles this via test.dependsOn: ["build"], so pnpm test at the root is fine; a bare pnpm --filter @taskless/cli test in a fresh checkout is not.

Much of this suite is integration-shaped: tests in check, runtime-check,
init, and onboard spawn the real built CLI with execFile("node",
[binPath, ...]), and several lay out a fixture tree and run `git init`
first. One cold CLI spawn measures ~0.8s by itself, so a test doing four
or five of them sits at 3-4s before any load.

Vitest's default 5s testTimeout left no margin for that. Because vitest
runs test files in parallel workers competing for CPU, whichever tests
landed together tipped over, so a different set failed on each run,
always with "Test timed out in 5000ms" and never an assertion failure.
That reads as a flaky product when it is a timeout sized below the work.

Confirmed to predate any current branch: a clean checkout of main with no
local changes failed 8 tests this way. With the timeout at 20s the suite
passes 427/427 across three consecutive runs.

20s swallows the contention without hiding a genuine hang.
Copilot AI lite review requested due to automatic review settings August 7, 2026 00:08
@thecodedrift thecodedrift added the skip-changeset PR intentionally ships no release note (bypasses the changeset requirement) label Aug 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts Vitest configuration for @taskless/cli so the integration-heavy CLI test suite has realistic timeouts and stops failing due to the default 5s limit when workers contend for CPU.

Changes:

  • Adds a test block to packages/cli/vite.config.ts to set testTimeout and hookTimeout to 20s.
  • Documents the rationale and observed behavior behind the timeout increase (integration-style tests spawning the built CLI, fixture setup, git init, parallelism effects).

@thecodedrift
thecodedrift merged commit 9c1150e into main Aug 7, 2026
10 checks passed
@thecodedrift
thecodedrift deleted the chore/vitest-timeout branch August 7, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changeset PR intentionally ships no release note (bypasses the changeset requirement)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants