Conversation
Extracts the Justfile from 98dcde7 (zarr-developers#4096). Environment and CI migration changes are adapted separately. Assisted-by: Codex:GPT-6
Extracts the Justfile changes from 693f694 (zarr-developers#4096). Assisted-by: Codex:GPT-6
Keep Hatch environment definitions while moving task commands into Justfile. Route root CI, Read the Docs, and contributor workflows through the same recipes, preserve quoted arguments, and ship recipes and helpers in the sdist. Assisted-by: Codex:GPT-6
Expose package recipes from the root while retaining each package’s command definitions and working directory. Assisted-by: Codex:GPT-6
… path Review follow-ups to the Justfile centralization: - `just gpu` read HATCH_ENV, so an exported CPU test environment silently redirected `pytest -m gpu` into an env built without the gpu feature. It now reads GPU_HATCH_ENV, which nothing else sets. - Pin `uvx prek` to 0.5.3 and restore hook-environment caching in the lint workflow, so lint no longer floats on whatever prek PyPI serves that day. - Restore the mypy hook's `uv run --frozen mypy` entry: routing it through `just typecheck` made just a prerequisite for committing in every existing clone, and expanded to the identical command. - Run the changelog filename check with `uv run --no-project python` instead of building the `dev` hatch env (test + remote-tests + docs + mypy) for a script that imports only sys and pathlib. - Extend `just just-check` over packages/*/justfile, which the root delegation recipes depend on and the formatting gate did not cover. - Set up Python in the codspeed job before pip-installing into it, matching the other workflows; that runner image was never given one. - Revert the release and nightly-wheel jobs to `hatch build`, so the publishing path does not fetch rust-just to run a pure alias. - Renumber the changelog fragment to this pull request and credit zarr-developers#4096 in the body, so the rendered note does not link to an unrelated upstream PR. Assisted-by: ClaudeCode:claude-opus-5
…one just process `just hooks-install` ran `uvx prek install`, and prek writes a hook shim that hard-codes the binary it was installed from, falling back to `prek` on PATH. Under uvx that path is an entry in uv's archive cache, so the first `uv cache prune` (or a bump of the pinned version) broke every commit with `prek: not found`, and nothing put `prek` on PATH for the fallback. Install it as a pinned `uv tool` instead: a changed pin upgrades in place, a repeated run is a no-op, and the shim points at a path that stays. `setup` re-invoked `just list-env`, which dropped a `just hatch_env=... setup` override on the floor because the child process re-read HATCH_ENV. Run it as a subsequent dependency so both steps see the same value. Also retire the last two comments naming the removed hatch scripts. Assisted-by: ClaudeCode:claude-fable-5-1
towncrier renders the fragment name as a link into zarr-developers/zarr-python, so the fork's PR number would have pointed the released note at an unrelated upstream pull request. zarr-developers#4372 is the PR that merges this work. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Documentation build overview
No files changed. |
Documentation build overview
No files changed. |
Merging this PR will not alter performance
Comparing Footnotes
|
Adding `actions/setup-python` to this job so that `pip install rust-just` had an interpreter also changed which interpreter hatch built `test.py3.12-minimal` on: the runner's own Python was replaced by /opt/hostedtoolcache/Python/3.12.5/arm64. CodSpeed measured the result as a 12.97% degradation across 26 benchmarks, spread -12.76% to -13.99% -- a range of 1.2 points over slice indexing, sharded Morton indexing, three array shapes and two store types. Nothing in this branch touches src/, tests/benchmarks/ or packages/; that uniformity is an interpreter swap, not a regression, and CodSpeed flagged it as "different runtime environments detected". Install just with extractions/setup-just instead, matching the three package workflows. No interpreter is set up, so hatch resolves the same Python the baseline used, and `just-version` is pinned to satisfy zizmor's unpinned-tools audit. The benchmarks need a re-run to produce a comparison against a matching environment; the numbers on the previous run should not be acknowledged as a regression. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Conflict in .github/workflows/docs.yml: zarr-developers#4370 bumped astral-sh/setup-uv on the line above the `uv sync --group docs` step this branch replaces with the two `uv tool install` lines. Took the bumped pin and kept the branch's install steps. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
perf regression reported in the benchmark was a false alarm due to using a different (slower) runner for the benchmark job |
Follow-ups from an adversarial audit of this branch. Supply chain. `rust-just` is a third-party repackaging of just on PyPI (Repository: github.com/gnpaone/rust-just) with no attestation on any of its 17 wheels, and this branch had introduced it in ten places including the self-hosted GPU runner and Read the Docs. All 23 just installs across the repo now come from casey/just's own release binaries via extractions/setup-crate@7577c1bd (v2.0.1), a node action whose SHA pins every line of executed code -- rather than extractions/setup-just, a composite wrapper that astral-sh/python-build-standalone#771 moved off for that reason. This also retires the 12 pre-existing setup-just call sites, so the repo has one mechanism instead of four. setup-crate does not verify checksums (extractions/setup-just#20 is open); Read the Docs, where no action is available, fetches the release tarball and verifies it against the SHA256SUMS casey/just publishes. Lockfile. docs.yml had stopped resolving from uv.lock: `uv sync --group docs` became `hatch run docs:`, and hatch never reads the lockfile, so the entire docs toolchain floated -- including numcodecs[msgpack], which carries no version constraint. Dependabot's uv ecosystem exists to keep that lock fresh and no job exercised it any more. The eight docs and changelog recipes now run `uv run --frozen --group docs`, so single-version tooling comes from the lock (as mypy already did) while hatch keeps the test environments, which exist per interpreter and per dependency set and cannot live in one lockfile. The hatch docs environment is removed as unused, and neither docs.yml nor Read the Docs installs hatch at all now. Argument forwarding. The three package justfiles splatted `{{ args }}` unquoted, so `just zarr-metadata test -k 'a and b'` reached pytest as `-k a` plus two stray paths, and command substitution in an argument executed. They now use `set positional-arguments` and `"$@"` like the root, which is what the contributing guide already claimed. Also, regressions from my own earlier commits on this branch: `just lint` had dropped `--show-diff-on-failure --color=always`, which j178/prek-action passed by default, so a hook that rewrote a file failed with no diff; a four-line comment above `hooks-install` displaced its `just --list` description, since just reads only the last comment line; the prek cache key did not cover prek_version and had a restore-keys fallback that could reuse a store built by a different prek; `just just-check` ran before `just lint`, so a cosmetic formatting nit hid every real lint result; and the `packages/*/justfile` glob lacked nullglob, which breaks in the sdist, where /Justfile ships but /packages deliberately does not. Docs: the install instructions used `pip install`, which fails on an externally-managed interpreter; `just just-check` was an undocumented required check; two sentences still told contributors to activate an environment that no longer exists; and the changelog fragment did not mention that the Hatch script tables were removed. Adds a .gitattributes rule so a Windows checkout does not get a CRLF Justfile that `just --fmt --check` rejects on every line. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
maxrjones
left a comment
There was a problem hiding this comment.
I've been using just for other projects and quite like it. I support these changes, especially if it makes the development more ergonomic for other devs.
ilan-gold
left a comment
There was a problem hiding this comment.
I like this quite a bit more :) just is great, thanks for pushing us on this @maxrjones.
Why not make the hatch env an argument to the just alongside the option of an env variable?
For CI, the variable makes sense, but locally, I would think setting an env variable on the same lien as running just coverage or something is odd
Adds a justfile with package lifecycle verbs. Pretty much #4096 minus the removal of hatch.
Would appreciate feedback from other devs, since this (hopefully) simplifies our dev environment.
🤖 AI text below 🤖
Development commands currently live in Hatch scripts, workflow YAML, and contributor instructions. This draft makes the root
Justfilethe command interface:just test,just coverage,just benchmark,just docs-check,just lint,just build, and changelog/lock helpers define the operations used locally and by root CI.Hatch continues to manage Python environments and dependencies.
HATCH_ENV=test.py3.13-minimal just coverage, for example, selects an environment while the recipe defines the command. The Hatch script tables are removed; contributor instructions, root workflows, Read the Docs, and the type-check hook now call just. Root recipesjust zarr-metadata,just zarr-indexing, andjust zarr-http-serverdelegate to the existing package justfiles, preserving their working directories and command definitions.Attribution
This extracts the Justfile portions of Max Jones's commits
98dcde719834bcd6fc010ffc4b07be5ae02fc2e4and693f694e50b9ae4cd7c6b0d93238b9ee9f8f6865from #4096, preserving attribution in two commits, then adapts them to current main without taking the Hatch-removal changes. Hatch stays.Design notes worth review
prek_version := "0.5.3"in the Justfile, souvx prek@…cannot float onto a new release mid-week. The lint workflow caches prek hook environments (~/.cache/prek) to replace the caching the prek action used to provide.just gpureadsGPU_HATCH_ENV, notHATCH_ENV. A recipe that defaulted throughenv("HATCH_ENV", "gputest.py3.12")would silently honour a CPU test environment exported in the caller's shell and runpytest -m gpuagainst an env built without thegpufeature. The separate name makes that unrepresentable.uv run --frozen mypyentry rather than routing throughjust typecheck, so just does not become a prerequisite for committing in an existing clone.just hooks-installinstalls prek as a persistentuv tool, not viauvx. prek writes a hook shim into.git/hooksthat hard-codes the binary path it was installed from; underuvxthat path is a uv cache archive entry, which disappears on the nextuv cache prune.hatch builddirectly.just buildis a pure alias, and it is not worth adding a package fetch to the path that publishes artifacts.just just-checkcovers all four justfiles, root pluspackages/*/justfile.Validation
CI on the fork (d-v-b#339) is green on this branch, including the jobs these changes touch:
uvx prek@0.5.3 run --all-files, all hooks pass; prek cache saved 50.9 MB for subsequent runsos=windows-latest, macOS,min_depsLocally: quoted pytest filters survive the recipes (
just test -k 'array and resize'deselects correctly); package delegation runs with the package directory as its working directory;just --fmt --checkpasses on all four justfiles and fails when one is unformatted; XML/JUnit and HTML coverage reports generate; the sdist includesJustfileandci/.A note on the CodSpeed run
An earlier revision added
actions/setup-pythontocodspeed.yml, so that the job'spip install rust-justwould have an interpreter. That also changed which interpreter Hatch builttest.py3.12-minimalon, replacing the runner's own Python with/opt/hostedtoolcache/Python/3.12.5/arm64. CodSpeed reported it as a 12.97% degradation across 26 benchmarks, spread only from -12.76% to -13.99% across slice indexing, sharded Morton indexing, three array shapes and two store types — and flagged "different runtime environments detected". Nothing in this branch touchessrc/,tests/benchmarks/orpackages/, so a 1.2-point spread over unrelated benchmarks is an interpreter swap, not a regression.That step is gone. just is now installed with
extractions/setup-just, matching the three package workflows, so the benchmark job sets up no interpreter of its own and Hatch resolves the same Python the baseline used. The benchmarks need a re-run (re-apply thebenchmarklabel — the workflow triggers onlabeled, not on push) before the comparison means anything; the numbers on the earlier run should not be acknowledged as a regression.Still not exercised:
gpu_testneeds the self-hosted GPU runner, so theGPU_HATCH_ENVchange is unverified in CI, and the Read the Docs build is only reasoned about, not run.Filed as a draft for review of the command interface and the CI transition.
🤖 Generated with Claude Code