Skip to content

test(flakeref): bump process-compose to v1.110.0 and refresh devbox.lock files#2868

Open
mikeland73 wants to merge 8 commits into
mainfrom
mikeland73/fix-flaky-cicd-tests
Open

test(flakeref): bump process-compose to v1.110.0 and refresh devbox.lock files#2868
mikeland73 wants to merge 8 commits into
mainfrom
mikeland73/fix-flaky-cicd-tests

Conversation

@mikeland73

@mikeland73 mikeland73 commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Bumps the process-compose version used by the add_platforms_flakeref testscript from v1.87.0 to v1.110.0.
  • Refreshes the devbox.lock files across the repo root and examples.

The flakeref test (testscripts/add/add_platforms_flakeref.test.txt) exercises devbox add with a GitHub flake ref. Because it pinned an old version, CI built process-compose from source and ran its (occasionally flaky) upstream test suite — a recurring source of flaky CI failures across unrelated PRs.

Changes

  • devbox add github:F1bonacc1/process-compose/v1.110.0 (was v1.87.0)
  • Version assertion updated to 1.110.0
  • Expected devbox.json key updated to the new ref
  • Updated devbox.lock at the repo root and across 23 example projects

🤖 Generated with Claude Code

Update the add-platforms flakeref testscript from process-compose
v1.87.0 to v1.110.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 15, 2026 04:03
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mikeland73 mikeland73 changed the title test(flakeref): bump process-compose to v1.110.0 test(flakeref): bump process-compose to v1.110.0 and refresh devbox.lock files Jun 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to reduce CI flakiness in the add_platforms_flakeref testscript by updating the pinned process-compose GitHub flake ref from v1.87.0 to v1.110.0, avoiding source builds of an older version during CI runs. However, the diff also includes a broad refresh of many devbox.lock files across examples and the repository root, which is not described in the PR metadata and substantially expands the scope.

Changes:

  • Bump testscripts/add/add_platforms_flakeref.test.txt to add github:F1bonacc1/process-compose/v1.110.0 and assert process-compose version output 1.110.0.
  • Update the expected devbox.json package key in the testscript to the new flake ref.
  • Refresh many devbox.lock files (examples and root), updating nixpkgs pins and various package resolutions.

Reviewed changes

Copilot reviewed 1 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
testscripts/add/add_platforms_flakeref.test.txt Bumps process-compose flake ref and updates the version assertion/expected JSON key.
devbox.lock Updates root lockfile nixpkgs pin and tool resolutions (scope-expanding change).
examples/stacks/rails/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/stacks/lepp-stack/devbox.lock Refreshes example lockfile package resolutions (scope-expanding change).
examples/stacks/lapp-stack/devbox.lock Refreshes example lockfile package resolutions (scope-expanding change).
examples/stacks/jekyll/devbox.lock Refreshes example lockfile package resolutions (scope-expanding change).
examples/stacks/drupal/devbox.lock Refreshes example lockfile package resolutions (scope-expanding change).
examples/stacks/django/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/servers/nginx/devbox.lock Refreshes example lockfile package resolutions (scope-expanding change).
examples/plugins/builtin/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/python/poetry/poetry-pyproject-subdir/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/python/poetry/poetry-demo/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/python/pipenv/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/python/pip/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/php/ds-extension/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/nodejs/nodejs-yarn/devbox.lock Refreshes example lockfile nixpkgs pin and nodejs resolution (scope-expanding change).
examples/development/nodejs/nodejs-pnpm/devbox.lock Refreshes example lockfile nixpkgs pin and nodejs resolution (scope-expanding change).
examples/development/haskell/devbox.lock Refreshes example lockfile package resolution entries (scope-expanding change).
examples/development/go/hello-world/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/elixir/elixir_hello/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/development/bun/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/databases/mysql/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/databases/mariadb/devbox.lock Refreshes example lockfile metadata/resolution (scope-expanding change).
examples/data_science/pytorch/basic-example/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).
examples/data_science/jupyter/devbox.lock Refreshes example lockfile nixpkgs pin/resolution (scope-expanding change).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread testscripts/add/add_platforms_flakeref.test.txt Outdated
mikeland73 and others added 5 commits June 14, 2026 21:11
stdout patterns are regexes; escape the dots so the version check is
exact. Addresses PR review feedback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nix.conf

Nix resolves `github:` flake refs (e.g. the latest commit of
nixpkgs-unstable) via the GitHub API. Anonymous requests are capped at
60/hr and were quickly exhausted by the parallel test suite, surfacing as
`unable to download '.../commits/nixpkgs-unstable': HTTP error 403` and
cascading test failures.

The token was only provided via NIX_CONFIG and the user-level
~/.config/nix/nix.conf. Nix only honors `access-tokens` from those
sources when the invoking user is trusted, and the daemon does not read
user config at all. On Linux the system-wide /etc/nix/nix.conf never
received the token, so GitHub calls went out unauthenticated.

- test job: write access-tokens to /etc/nix/nix.conf on every platform
  (always trusted, read by client and daemon) and restart the daemon
  (systemctl on Linux, launchctl on macOS).
- test-nix-versions job: pass access-tokens through the nix-installer
  extra-conf, which lands in /etc/nix/nix.conf.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
process-compose is a slow Go build pinned to a mutable upstream tag that
needed repeated bumping, making the test flaky. Switch to GNU hello pinned
to a fixed nixpkgs commit: it is fetched prebuilt from the binary cache and
the commit never changes, while still exercising the github: flake-ref add
path with --exclude-platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `test` job defined a 3-value `nix-version` matrix, but the value was
never wired into any step after the job switched to installing Nix via
devbox-install-action (which can't pin a Nix version). The three legs ran
identical duplicates, tripling the slowest/flakiest job for no extra
coverage. Remove the matrix and document how to re-add it once the install
action supports specifying a Nix version. Cross-version coverage remains in
the separate test-nix-versions job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y-cicd-tests

Brings in #2869: remove the dead nix-version matrix from the cli-tests
test job (the install action no longer pins a Nix version, so the legs
were identical duplicates).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The jekyll example's run_test fails with `cannot load such file -- csv
(LoadError)` because `bundler@latest` from nixpkgs now runs jekyll under
Ruby 3.4, which moved csv (and base64, logger) from default gems to
bundled gems. Bundled gems must be declared in the Gemfile or `require`
fails at startup. The run_test regenerates Gemfile.lock via `bundle
update`, so only the Gemfile needs the entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants