Conversation
* fix: complete imagecodecs codec package hints Assisted-by: Codex:GPT-6 * docs: associate codec changelog with PR 338 Assisted-by: Codex:GPT-6 * docs: associate codec changelog with upstream PR 4351 Assisted-by: Codex:GPT-6
…r-developers#4345) * docs(indexing): ground design and integration claims in current behavior Assisted-by: Codex:GPT-6 * docs(indexing): correct reader lazy-array and cache contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate wire boundaries and clarify format contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate selector bounds and shared dependencies Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts. Assisted-by: Codex:GPT-6 * docs(indexing): reconcile reader contracts and record audit fixes Assisted-by: Codex:GPT-6 * docs(indexing): reconcile audit with current partition implementation Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing. Assisted-by: Codex:GPT-6 * docs(indexing): clarify planning coverage and benchmark measurement boundaries Assisted-by: Codex:GPT-6 * fix(indexing): group signed chunk coordinates without collisions Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates. Assisted-by: Codex:GPT-6 * docs(indexing): state remaining planner limits precisely Assisted-by: Codex:GPT-6 * docs(indexing): number audit changelog entries for PR 4345 Assisted-by: Codex:GPT-6 * docs(indexing): describe current contracts in docstrings Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values. Assisted-by: Codex:GPT-6
* docs(indexing): ground design and integration claims in current behavior Assisted-by: Codex:GPT-6 * docs(indexing): correct reader lazy-array and cache contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate wire boundaries and clarify format contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate selector bounds and shared dependencies Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts. Assisted-by: Codex:GPT-6 * docs(indexing): reconcile reader contracts and record audit fixes Assisted-by: Codex:GPT-6 * docs(indexing): reconcile audit with current partition implementation Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing. Assisted-by: Codex:GPT-6 * docs(indexing): clarify planning coverage and benchmark measurement boundaries Assisted-by: Codex:GPT-6 * fix(indexing): group signed chunk coordinates without collisions Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates. Assisted-by: Codex:GPT-6 * docs(indexing): state remaining planner limits precisely Assisted-by: Codex:GPT-6 * test(indexing): broaden planner property coverage Assisted-by: Codex:GPT-6 * test(indexing): generate mixed affine planner dependencies Assisted-by: Codex:GPT-6 * docs(indexing): number audit changelog entries for PR 4345 Assisted-by: Codex:GPT-6 * docs(indexing): describe current contracts in docstrings Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values. Assisted-by: Codex:GPT-6
…developers#4347) * docs(indexing): ground design and integration claims in current behavior Assisted-by: Codex:GPT-6 * docs(indexing): correct reader lazy-array and cache contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate wire boundaries and clarify format contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate selector bounds and shared dependencies Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts. Assisted-by: Codex:GPT-6 * docs(indexing): reconcile reader contracts and record audit fixes Assisted-by: Codex:GPT-6 * docs(indexing): reconcile audit with current partition implementation Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing. Assisted-by: Codex:GPT-6 * docs(indexing): clarify planning coverage and benchmark measurement boundaries Assisted-by: Codex:GPT-6 * fix(indexing): group signed chunk coordinates without collisions Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates. Assisted-by: Codex:GPT-6 * docs(indexing): state remaining planner limits precisely Assisted-by: Codex:GPT-6 * fix(indexing): reject unsupported wire index array bounds Assisted-by: Codex:GPT-6 * docs(indexing): clarify wire bounds rejection contract Assisted-by: Codex:GPT-6 * docs(indexing): number audit changelog entries for PR 4345 Assisted-by: Codex:GPT-6 * docs(indexing): describe current contracts in docstrings Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values. Assisted-by: Codex:GPT-6 * fix(indexing): validate raw index values against wire bounds Accept valid finite and one-sided bounds with eager validation shared by both JSON loaders. Validate before affine adjustment and simplification; immutable validated maps need no retained constraint. Assisted-by: Codex:GPT-6
…4348) * docs(indexing): ground design and integration claims in current behavior Assisted-by: Codex:GPT-6 * docs(indexing): correct reader lazy-array and cache contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate wire boundaries and clarify format contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate selector bounds and shared dependencies Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts. Assisted-by: Codex:GPT-6 * docs(indexing): reconcile reader contracts and record audit fixes Assisted-by: Codex:GPT-6 * docs(indexing): reconcile audit with current partition implementation Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing. Assisted-by: Codex:GPT-6 * docs(indexing): clarify planning coverage and benchmark measurement boundaries Assisted-by: Codex:GPT-6 * fix(indexing): group signed chunk coordinates without collisions Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates. Assisted-by: Codex:GPT-6 * docs(indexing): state remaining planner limits precisely Assisted-by: Codex:GPT-6 * fix(indexing): define explicit source token contract Assisted-by: Codex:GPT-6 * fix(indexing): reject mmap-backed token buffers Assisted-by: Codex:GPT-6 * docs(indexing): number audit changelog entries for PR 4345 Assisted-by: Codex:GPT-6 * docs(indexing): describe current contracts in docstrings Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values. Assisted-by: Codex:GPT-6 * fix(indexing): delegate source tokenization to Dask Assisted-by: Codex:GPT-6
Extracts the Justfile from 98dcde7 (zarr-developers#4096). Environment and CI migration changes are adapted separately. Assisted-by: Codex:GPT-6
Extracts the Justfile changes from 693f694 (zarr-developers#4096). Assisted-by: Codex:GPT-6
Keep Hatch environment definitions while moving task commands into Justfile. Route root CI, Read the Docs, and contributor workflows through the same recipes, preserve quoted arguments, and ship recipes and helpers in the sdist. Assisted-by: Codex:GPT-6
Expose package recipes from the root while retaining each package’s command definitions and working directory. Assisted-by: Codex:GPT-6
…arr-developers#4358) Zarr v2 retried DirectoryStore's rename because Windows intermittently refuses to replace a destination (zarr-developers#597, fixed by zarr-developers#698). Atomic writes arrived in v3's LocalStore in zarr-developers#3412 without that retry, so the failure is back: _atomic_write's tmp_path.replace(path) raises PermissionError: [WinError 5] Access is denied: '...zarr.<hex>.partial' -> '...zarr.json' and aborts the write. Reported in zarr-developers#3522. _move_with_retry wraps the final move, retrying only the two Windows codes that mean the destination could not be superseded right now. It needs no platform test: off Windows an OSError carries no winerror, so the first attempt either succeeds or raises. The exclusive path is routed through it too but is unaffected by construction -- the FileExistsError it relies on to report an existing node is ERROR_ALREADY_EXISTS (183), which is not in the retried set, so it still propagates on the first attempt. Measured on Windows 11, 4,000 group-attr rewrites (each a replace onto an existing zarr.json): 155-171 raised before, 0 after, for 3.68 s -> 4.08 s of wall clock on a workload that is nothing but replace-onto-existing. In a narrower stdlib-only loop of 20,000 replaces, 475 of 498 recoveries needed only the second attempt and the worst needed the fourth.
…ers#4358 (zarr-developers#4359) Follow-up cleanup to the retry landed in zarr-developers#4358, no behaviour change: - drop the leading 0.0 sentinel, the bare `last_error` annotation, and the possibly-unbound re-raise from `_move_with_retry`; the final attempt now runs after the loop and propagates its own traceback - correct the comment on ERROR_ACCESS_DENIED: Windows also reports it for destinations that will never clear (directory, read-only, ACL), so those surface the same error after the bounded delay - align the changelog with the code: the exclusive path is routed through the retry, only FileExistsError is excluded; rename the fragment to the PR number and describe zarr-developers#3522 as mitigated, since a second process holding the destination open past the budget still fails - tests record `time.sleep` instead of sleeping (~1.2 s per run before, now instant) and assert the actual delay schedule; the closure test double becomes a small callable class, removing four type-ignores - add a regression test for `_atomic_write` onto an existing directory Assisted-by: ClaudeCode:claude-fable-5-1 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* docs(blog): skeleton for the 3.4.0 release post Headings and front matter in the 3.3.0 house style, with the proposed highlights, evidence, PR references and snippet ideas as markdown comments for the prose to be written against. Marked draft: true so it is excluded from the built site until it is ready. Assisted-by: ClaudeCode:claude-fable-5-1 * docs(blog): 3.4.0 post prose, copy edits, references, and contributor handles Prose for the stack, roadmap, rectilinear, codec-error and msgspec sections; typo and grammar fixes; PR/issue links throughout; a runnable example of the new unknown-codec error; contributors listed by GitHub handle with first-time contributors marked. The 3.3.0 post gains the same contributor section for its own release range. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(blog): drop sections about fixes for 3.3.0 regressions * docs(blog): link the previous post from the contributor note * AI disclaimer * Update 3.4.0-release.md * Update docs/blog/posts/3.4.0-release.md Co-authored-by: Ilan Gold <ilanbassgold@gmail.com> * prose * docs: add link to registry docs * docs: fix the 3.3.0 release-notes link that pointed at a nonexistent PR The load/open docstring entry cited zarr-developers#3984, which does not exist upstream as a PR or an issue, so the weekly link check (zarr-developers#4357) reports a 404. Point at the commit that landed the change instead. Assisted-by: ClaudeCode:claude-fable-5-1 --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Ilan Gold <ilanbassgold@gmail.com>
Built with `towncrier build --version 3.4.0`, consuming the 33 fragments under changes/. Two touch-ups to the generated section: the codec error-message example fence gets a `text` language (markdownlint MD040), and the three fragments that had been named after issues rather than the pull requests that merged them (3285 -> zarr-developers#4063; 4174 and 4272 -> zarr-developers#4218) now link to those pull requests. Assisted-by: ClaudeCode:claude-fable-5-1
…zarr-developers#4361) GitHub's "Generate release notes" lists every pull request merged since the previous tag, which in this repository includes the zarr-metadata, zarr-indexing and zarr-http-server work that ships with those packages' own releases. Label pull requests confined to one packages/ directory with that package's name, and exclude those labels in .github/release.yml; dependabot updates get their own section. Assisted-by: ClaudeCode:claude-fable-5-1
…developers#4349) * docs(indexing): ground design and integration claims in current behavior Assisted-by: Codex:GPT-6 * docs(indexing): correct reader lazy-array and cache contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate wire boundaries and clarify format contracts Assisted-by: Codex:GPT-6 * fix(indexing): validate selector bounds and shared dependencies Correct mathematical API documentation to match supported coordinate, grid, and chunk projection contracts. Assisted-by: Codex:GPT-6 * docs(indexing): reconcile reader contracts and record audit fixes Assisted-by: Codex:GPT-6 * docs(indexing): reconcile audit with current partition implementation Retain the existing unsigned selector fix and update the unsupported mixed-dependency error assertion for general intersection routing. Assisted-by: Codex:GPT-6 * docs(indexing): clarify planning coverage and benchmark measurement boundaries Assisted-by: Codex:GPT-6 * fix(indexing): group signed chunk coordinates without collisions Use lexicographic tuple grouping when chunk indices contain negative values. Cover shared one-axis and two-axis array dependencies, repeated points, and extreme signed coordinates. Assisted-by: Codex:GPT-6 * docs(indexing): state remaining planner limits precisely Assisted-by: Codex:GPT-6 * feat(indexing): execute partitions with their reader context Assisted-by: Codex:GPT-6 * docs(indexing): number audit changelog entries for PR 4345 Assisted-by: Codex:GPT-6 * docs(indexing): describe current contracts in docstrings Remove implementation history and unsupported historical claims from source and test docstrings. Distinguish immutable coordinate mappings from mutable source values. Assisted-by: Codex:GPT-6 * fix(indexing): plan every view read with its source grid Remove Partition.result and partition-local source windows. Preserve the source grid in derived views and supply projections for every LazyArray reader call, including unpartitioned reads. Assisted-by: Codex:GPT-6
…ers#4350) * feat(indexing): make LazyArray indexing lazy by default Add synchronous writes through composed selections and an explicit eager adapter for array consumers. Cover deferred reads, source mutation, masks, aliases, and repeated destinations. Assisted-by: Codex:GPT-6 Rebased-onto: upstream/main after zarr-developers#4345-zarr-developers#4349 merged; conflicts resolved with ClaudeCode:claude-fable-5-1 * fix(indexing): plan fancy writes against the source's write grid The non-affine write fallback assigned one element at a time, which on a chunked source is one chunk read-modify-write per element: a 50-row orthogonal write into a 1000x1000 zarr array with 100x100 chunks cost 49,999 chunk writes and 27 s against 100 writes and 10 ms natively. `write_into` now scatters in bulk. NumPy sources get one fancy assignment. A readable source with a write grid, which `LazyArray.write` discovers from `write_chunk_sizes` then `chunks`, is written one cell at a time: the cell's touched hull is read once, updated in memory with last-occurrence-wins semantics, and written back with one basic slice. The same write into zarr now costs 100 chunk writes. Sources with no grid, transforms the planner cannot factor, and sources that cannot be read keep the per-element path, which never reads. Also: order the rank check before the value copy while still validating values for an empty selection; keep the payload of a masked zero-rank affine write, which `np.flip` with no axes was turning into the `masked` singleton; note that writes bypass the reader and pin that with a test; note that iterated elements are views; add API pages for the writer and eager adapter; and describe the shipped indexing surface in the unreleased `LazyArray` changelog fragment instead of its removed `.lazy` accessor. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(indexing): keep literal domains on views, positional keys unchanged `LazyArray` re-zeroed the domain of every derived view, so a view forgot which coordinates it was cut from and placement needed a side channel. TensorStore keeps literal coordinates on views; NumPy users expect positional keys. These are separable: the key dialect says how a key is read, the domain says what the view remembers. Views now keep their literal domain: `a[10:20]` has domain `[10, 20)` and `a[10:20][2:5]` has `[12, 15)`, while `a[10:20][0]` is still the first element because positional keys are normalized against the domain's origin, which `normalize_positional_selection` already did. Two literal keys join the NumPy ones, as in TensorStore's `__getitem__`: an `IndexDomain` restricts the view to coordinates of its own domain (empty intervals outside it are refused too) and an `IndexTransform` composes onto it. Box partitions keep the request's coordinates, so a part view's domain is a sub-domain of its parent's; a part placed by index arrays keeps a fresh zero-origin domain, and `out_selection` is the placement in both cases. A reversed view shows the negative origin the algebra already produced. Two frames stay zero-origin by construction so no consumer has to subtract an origin: `ReadContext` re-bases its transform, and `parts()` re-bases each projection's `cell_transform` from the request's literal domain to positions in the view's result buffer, which is what `Partition.projection` has always documented. A table-driven test states the expected literal domain for every indexing form and nested chain, checked against the transform algebra. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(indexing): state that a key's type picks relative or absolute indexing Make the one rule about indexing a LazyArray prominent: NumPy keys are positions relative to the view, an `IndexDomain` key names absolute coordinates of the view's domain, an `IndexTransform` key composes onto it, no key type has two readings, and every view keeps its absolute domain whichever key produced it. A guide section with a runnable snippet carries the table and the pandas `ix` / TensorStore comparison; the module docstring, README, and API page point at it. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…elopers#4363) * fix: replace runtime assert statements with explicit checks Asserts are stripped under `python -O`, so checks that matter for correctness or type narrowing must be explicit. Two were load-bearing: `GroupMetadata.from_dict` asserted on node_type and the array-to-group fallback in `zarr.api.asynchronous.open` caught the AssertionError, and `make_store` asserted on mode ahead of the real validation. Redundant asserts are deleted, narrowing asserts are restructured so mypy narrows on its own, and the rest become explicit raises. Ruff S101 is enabled with `tests/` and `src/zarr/testing/` excluded. Assisted-by: ClaudeCode:claude-fable-5-1 * docs: add changelog fragment for zarr-developers#4363 Assisted-by: ClaudeCode:claude-fable-5-1 * ci: scope the no-assert lint to runtime code across the monorepo The S101 per-file ignores only covered the root tests/ and src/zarr/testing/. The packages under packages/ inherit the root ruff config, so the rule fired on 1524 asserts in their tests, examples, and test-support modules and broke the ruff, Lint, pre-commit.ci, and zarr-http-server jobs. Widen the ignores to **/tests/**, **/examples/**, and **/testing/**, and exclude zarr-indexing's runtime source for now; its own asserts are tracked as a separate change. Assisted-by: ClaudeCode:claude-fable-5-1
zarr-developers#4365) * fix(zarr-metadata): v2 array document is open and filters may be empty Two v2 structural rules were stricter than the spec, and the shared conformance corpus (zarr-metadata.js, conformance/v2_array.json cases 5 and 8) has been updated first; this brings the reference implementation back into agreement. - Members outside the .zarray definition were rejected. The spec: "Other keys SHOULD NOT be present within the metadata object and SHOULD be ignored by implementations" — a recommendation, unlike .zgroup's "Other keys MUST NOT be present". Extras are now tolerated by the validator, dropped by the model, and permitted by the Pydantic schema (the TypedDict is open, like the v3 one). The on-disk `.zarray` rule that `attributes` belongs in `.zattrs` is unchanged. - filters: [] was rejected ("expected at least one filter"). The spec: "A list of JSON objects providing codec configurations, or null" — an empty list is a list. The Pydantic schema's min_length is dropped. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): changelog fragment for zarr-developers#4365 Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): link the spec text the v2 open-array change cites Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): link the spec text behind every spec statement Every docstring or comment that cites the Zarr spec or a zarr-extensions README now carries a commit-pinned permalink with a line range (zarr-specs fc7dd9c; zarr-extensions 4da7b37, the registry commit the TypeScript port vendors). Unpinned zarr-extensions `tree/main` page links are pinned the same way. Three statements were wrong or stale and are corrected: - zstd: `checksum` was typed required "per the proposed specification" (zarr-specs PR #256, never merged). The published zarr-extensions entry makes it optional ("Should be omitted if false"; schema requires only `level`), so it is now `NotRequired[bool]`. - v3 consolidated metadata was described as "not a spec artifact"; since zarr-specs zarr-developers#373 the core spec names the field and fixes its envelope (core/index.rst L802-L816); the entry format remains a convention. - The v2 array `*Partial` docstring spoke of a "closed shape"; the array document is open (other keys SHOULD be ignored), unlike `.zgroup`. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…developers#4371) Bumps the python-dependencies group with 5 updates: | Package | From | To | | --- | --- | --- | | [numpy](https://github.com/numpy/numpy) | `2.5.2` | `2.5.3` | | [hypothesis](https://github.com/HypothesisWorks/hypothesis) | `6.167.1` | `6.168.0` | | [uv](https://github.com/astral-sh/uv) | `0.12.9` | `0.12.12` | | [towncrier](https://github.com/twisted/towncrier) | `25.8.0` | `26.9.0` | | [ruff](https://github.com/astral-sh/ruff) | `0.16.5` | `0.16.6` | Updates `numpy` from 2.5.2 to 2.5.3 - [Release notes](https://github.com/numpy/numpy/releases) - [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst) - [Commits](numpy/numpy@v2.5.2...v2.5.3) Updates `hypothesis` from 6.167.1 to 6.168.0 - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](HypothesisWorks/hypothesis@v6.167.1...v6.168.0) Updates `uv` from 0.12.9 to 0.12.12 - [Release notes](https://github.com/astral-sh/uv/releases) - [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md) - [Commits](astral-sh/uv@0.12.9...0.12.12) Updates `towncrier` from 25.8.0 to 26.9.0 - [Release notes](https://github.com/twisted/towncrier/releases) - [Changelog](https://github.com/twisted/towncrier/blob/trunk/NEWS.rst) - [Commits](twisted/towncrier@25.8.0...26.9.0) Updates `ruff` from 0.16.5 to 0.16.6 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.16.5...0.16.6) --- updated-dependencies: - dependency-name: numpy dependency-version: 2.5.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python-dependencies - dependency-name: hypothesis dependency-version: 6.168.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: python-dependencies - dependency-name: uv dependency-version: 0.12.12 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python-dependencies - dependency-name: towncrier dependency-version: 26.9.0 dependency-type: direct:development update-type: version-update:semver-major dependency-group: python-dependencies - dependency-name: ruff dependency-version: 0.16.6 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pers#4368) * fix(ci): preserve branch zarr in xarray downstream tests * chore: use pull request number for changelog fragment --------- Co-authored-by: glaziermag <glaziermag@users.noreply.github.com> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
… path Review follow-ups to the Justfile centralization: - `just gpu` read HATCH_ENV, so an exported CPU test environment silently redirected `pytest -m gpu` into an env built without the gpu feature. It now reads GPU_HATCH_ENV, which nothing else sets. - Pin `uvx prek` to 0.5.3 and restore hook-environment caching in the lint workflow, so lint no longer floats on whatever prek PyPI serves that day. - Restore the mypy hook's `uv run --frozen mypy` entry: routing it through `just typecheck` made just a prerequisite for committing in every existing clone, and expanded to the identical command. - Run the changelog filename check with `uv run --no-project python` instead of building the `dev` hatch env (test + remote-tests + docs + mypy) for a script that imports only sys and pathlib. - Extend `just just-check` over packages/*/justfile, which the root delegation recipes depend on and the formatting gate did not cover. - Set up Python in the codspeed job before pip-installing into it, matching the other workflows; that runner image was never given one. - Revert the release and nightly-wheel jobs to `hatch build`, so the publishing path does not fetch rust-just to run a pure alias. - Renumber the changelog fragment to this pull request and credit zarr-developers#4096 in the body, so the rendered note does not link to an unrelated upstream PR. Assisted-by: ClaudeCode:claude-opus-5
…one just process `just hooks-install` ran `uvx prek install`, and prek writes a hook shim that hard-codes the binary it was installed from, falling back to `prek` on PATH. Under uvx that path is an entry in uv's archive cache, so the first `uv cache prune` (or a bump of the pinned version) broke every commit with `prek: not found`, and nothing put `prek` on PATH for the fallback. Install it as a pinned `uv tool` instead: a changed pin upgrades in place, a repeated run is a no-op, and the shim points at a path that stays. `setup` re-invoked `just list-env`, which dropped a `just hatch_env=... setup` override on the floor because the child process re-read HATCH_ENV. Run it as a subsequent dependency so both steps see the same value. Also retire the last two comments naming the removed hatch scripts. Assisted-by: ClaudeCode:claude-fable-5-1
towncrier renders the fragment name as a link into zarr-developers/zarr-python, so the fork's PR number would have pointed the released note at an unrelated upstream pull request. zarr-developers#4372 is the PR that merges this work. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: update pre-commit hooks updates: - [github.com/astral-sh/ruff-pre-commit: v0.16.0 → v0.16.6](astral-sh/ruff-pre-commit@v0.16.0...v0.16.6) - [github.com/codespell-project/codespell: v2.4.2 → v2.4.3](codespell-project/codespell@v2.4.2...v2.4.3) - [github.com/DavidAnson/markdownlint-cli2: v0.22.1 → v0.23.2](DavidAnson/markdownlint-cli2@v0.22.1...v0.23.2) - [github.com/scientific-python/cookie: 2026.06.18 → 2026.08.14](scientific-python/cookie@2026.06.18...2026.08.14) - [github.com/zizmorcore/zizmor-pre-commit: v1.26.1 → v1.30.0](zizmorcore/zizmor-pre-commit@v1.26.1...v1.30.0) - [github.com/twisted/towncrier: 25.8.0 → 26.9.0](twisted/towncrier@25.8.0...26.9.0) * style: reformat a README example for ruff 0.16.6 ruff-format v0.16.6 formats Python code blocks in Markdown, and normalizes this inline comment to PEP 8's two spaces. Fallout from the hook bump in this pull request, not a behavior change. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: pin just-version in setup-just steps zizmor v1.30.0 adds the `unpinned-tools` audit, which flags all twelve `extractions/setup-just` steps across the three package workflows: pinning the action by SHA still leaves the action free to install whatever just is newest at run time, so a just release can change CI without a commit here. Pinned to 1.58.0, which is what the action resolves to today, so this changes nothing about the current builds while making that choice explicit. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Adding `actions/setup-python` to this job so that `pip install rust-just` had an interpreter also changed which interpreter hatch built `test.py3.12-minimal` on: the runner's own Python was replaced by /opt/hostedtoolcache/Python/3.12.5/arm64. CodSpeed measured the result as a 12.97% degradation across 26 benchmarks, spread -12.76% to -13.99% -- a range of 1.2 points over slice indexing, sharded Morton indexing, three array shapes and two store types. Nothing in this branch touches src/, tests/benchmarks/ or packages/; that uniformity is an interpreter swap, not a regression, and CodSpeed flagged it as "different runtime environments detected". Install just with extractions/setup-just instead, matching the three package workflows. No interpreter is set up, so hatch resolves the same Python the baseline used, and `just-version` is pinned to satisfy zizmor's unpinned-tools audit. The benchmarks need a re-run to produce a comparison against a matching environment; the numbers on the previous run should not be acknowledged as a regression. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…zarr-developers#4370) Bumps the actions group with 4 updates in the / directory: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv), [github-community-projects/issue-metrics](https://github.com/github-community-projects/issue-metrics), [scientific-python/upload-nightly-action](https://github.com/scientific-python/upload-nightly-action) and [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action). Updates `astral-sh/setup-uv` from 10.0.1 to 10.1.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@20cfd1b...bec219d) Updates `github-community-projects/issue-metrics` from 5.0.1 to 5.0.2 - [Release notes](https://github.com/github-community-projects/issue-metrics/releases) - [Commits](github-community-projects/issue-metrics@61084fa...a7dc2fb) Updates `scientific-python/upload-nightly-action` from 0.6.4 to 0.6.5 - [Release notes](https://github.com/scientific-python/upload-nightly-action/releases) - [Commits](scientific-python/upload-nightly-action@e76cfec...16fa02e) Updates `zizmorcore/zizmor-action` from 0.6.3 to 0.6.4 - [Release notes](https://github.com/zizmorcore/zizmor-action/releases) - [Commits](zizmorcore/zizmor-action@70fb788...cc914d7) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 10.1.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: github-community-projects/issue-metrics dependency-version: 5.0.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: scientific-python/upload-nightly-action dependency-version: 0.6.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: zizmorcore/zizmor-action dependency-version: 0.6.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Conflict in .github/workflows/docs.yml: zarr-developers#4370 bumped astral-sh/setup-uv on the line above the `uv sync --group docs` step this branch replaces with the two `uv tool install` lines. Took the bumped pin and kept the branch's install steps. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-ups from an adversarial audit of this branch. Supply chain. `rust-just` is a third-party repackaging of just on PyPI (Repository: github.com/gnpaone/rust-just) with no attestation on any of its 17 wheels, and this branch had introduced it in ten places including the self-hosted GPU runner and Read the Docs. All 23 just installs across the repo now come from casey/just's own release binaries via extractions/setup-crate@7577c1bd (v2.0.1), a node action whose SHA pins every line of executed code -- rather than extractions/setup-just, a composite wrapper that astral-sh/python-build-standalone#771 moved off for that reason. This also retires the 12 pre-existing setup-just call sites, so the repo has one mechanism instead of four. setup-crate does not verify checksums (extractions/setup-just#20 is open); Read the Docs, where no action is available, fetches the release tarball and verifies it against the SHA256SUMS casey/just publishes. Lockfile. docs.yml had stopped resolving from uv.lock: `uv sync --group docs` became `hatch run docs:`, and hatch never reads the lockfile, so the entire docs toolchain floated -- including numcodecs[msgpack], which carries no version constraint. Dependabot's uv ecosystem exists to keep that lock fresh and no job exercised it any more. The eight docs and changelog recipes now run `uv run --frozen --group docs`, so single-version tooling comes from the lock (as mypy already did) while hatch keeps the test environments, which exist per interpreter and per dependency set and cannot live in one lockfile. The hatch docs environment is removed as unused, and neither docs.yml nor Read the Docs installs hatch at all now. Argument forwarding. The three package justfiles splatted `{{ args }}` unquoted, so `just zarr-metadata test -k 'a and b'` reached pytest as `-k a` plus two stray paths, and command substitution in an argument executed. They now use `set positional-arguments` and `"$@"` like the root, which is what the contributing guide already claimed. Also, regressions from my own earlier commits on this branch: `just lint` had dropped `--show-diff-on-failure --color=always`, which j178/prek-action passed by default, so a hook that rewrote a file failed with no diff; a four-line comment above `hooks-install` displaced its `just --list` description, since just reads only the last comment line; the prek cache key did not cover prek_version and had a restore-keys fallback that could reuse a store built by a different prek; `just just-check` ran before `just lint`, so a cosmetic formatting nit hid every real lint result; and the `packages/*/justfile` glob lacked nullglob, which breaks in the sdist, where /Justfile ships but /packages deliberately does not. Docs: the install instructions used `pip install`, which fails on an externally-managed interpreter; `just just-check` was an undocumented required check; two sentences still told contributors to activate an environment that no longer exists; and the changelog fragment did not mention that the Hatch script tables were removed. Adds a .gitattributes rule so a Windows checkout does not get a CRLF Justfile that `just --fmt --check` rejects on every line. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Development commands currently live in Hatch scripts, workflow YAML, and contributor instructions. This draft makes the root
Justfilethe command interface:just test,just coverage,just benchmark,just docs-check,just lint,just build, and changelog/lock helpers define the operations used locally and by root CI.Hatch continues to manage Python environments and dependencies.
HATCH_ENV=test.py3.13-minimal just coverage, for example, selects an environment while the recipe defines the command. The Hatch script tables are removed; contributor instructions, root workflows, Read the Docs, and the type-check hook now call just. Root recipesjust zarr-metadata,just zarr-indexing, andjust zarr-http-serverdelegate to the existing package justfiles, preserving their working directories and command definitions. For example,just zarr-metadata testruns the package test recipe.This extracts the Justfile portions of Max Jones's commits
98dcde719834bcd6fc010ffc4b07be5ae02fc2e4and693f694e50b9ae4cd7c6b0d93238b9ee9f8f6865from zarr-developers#4096, preserving attribution in two commits, then adapts them to current main without taking the Hatch-removal changes. Recipes preserve quoted arguments and ship with their Python helpers in the source distribution.Validation:
The full test matrix, Windows execution, and GPU execution have not been run locally. This is a draft for reviewing the command interface and CI transition.