From 37576062215653e83faa8da4fb3d1b431fbc8911 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 20 Aug 2026 16:47:42 +0000 Subject: [PATCH 01/12] Add git / stress-ng / pjdfstest targets and a backends x targets CI matrix The CI ran exactly one suite (`git annex test`) per backend. That answers "is git-annex broken on this filesystem?" but not "at which layer?" -- a rename-semantics quirk shows up as a pile of git-annex failures with no indication whether git, or POSIX itself, is already unhappy. Add three complementary targets underneath it: stress-ng 20 curated filesystem stressors with --verify, one at a time so an unsupported operation (xattr on vfat) reports as a skip rather than poisoning the run. ~2 min, apt only. pjdfstest POSIX conformance: 238 scripts, ~8800 assertions naming the syscall and the errno. ~3.5 min on ext4. git 174 scripts of git's own testsuite (t0*.sh t1*.sh, ~10k assertions) -- the plumbing git-annex stands on. Only the trash directories go on the mount, via the suite's --root; the build stays on the runner's disk. ~2 min on ext4. Structure: - bin/ci/matrix.sh is the single source of truth: the backend rows, the target columns, pinned upstream refs, per-target timeout / loop size / cron minute. - bin/ci/install-target.sh does runner-side prep (apt package, or a shallow fetch + build at a pinned ref into $EVAL_UNDER_SRC_DIR). - bin/ci/target-.sh is the suite itself, run inside the mount. - bin/ci/run-under.sh grows a third argument selecting the target and applies its timeout and loop-image size. - bin/ci/gen-dispatchers.sh generates the 20 per-cell dispatcher workflows and the README matrix from matrix.sh; `--check` reports stale or orphaned files. GitHub publishes one badge per workflow file, not per matrix cell, so the grid needs one file per cell. The five existing dispatchers are renamed to carry an explicit -git-annex suffix so all 20 cells are named uniformly. Two notes on pinning: - git is pinned to the v2.55.0 tag. - pjdfstest is pinned to a commit, not its one upstream tag. That tag ("0.1", 2016) no longer compiles: major()/minor()/makedev() moved to in glibc 2.28 and the tree builds with -Werror, so the implicit declarations are hard errors on any current toolchain. Master builds clean, so pin a commit on it rather than carry a patch. README badge URLs pointed at the pre-rename yarikoptic/beegfs-test repo; the generated matrix points them at con/eval-under. Validated locally: all three new targets run green on ext4 (stress-ng 20/20 through a real loop mount via run-under.sh; pjdfstest 8827/8827; git 10366 tests, 0 failed), shellcheck clean across bin/ci/*.sh, bin/eval-under*, provision/*.sh, all 21 workflows parse, and reuse lint still reports 100% compliance. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01SHhrxxVz3TJFwYMXuxei2H --- .github/workflows/_test-under.yaml | 45 +++- ....yaml => test-beegfs-7.4.6-git-annex.yaml} | 16 +- .github/workflows/test-beegfs-7.4.6-git.yaml | 42 ++++ .../test-beegfs-7.4.6-pjdfstest.yaml | 42 ++++ .../test-beegfs-7.4.6-stress-ng.yaml | 42 ++++ ....yaml => test-beegfs-8.1.0-git-annex.yaml} | 16 +- .github/workflows/test-beegfs-8.1.0-git.yaml | 42 ++++ .../test-beegfs-8.1.0-pjdfstest.yaml | 42 ++++ .../test-beegfs-8.1.0-stress-ng.yaml | 42 ++++ ...xt4.yaml => test-loop-ext4-git-annex.yaml} | 18 +- .github/workflows/test-loop-ext4-git.yaml | 40 ++++ .../workflows/test-loop-ext4-pjdfstest.yaml | 40 ++++ .../workflows/test-loop-ext4-stress-ng.yaml | 40 ++++ ...fat.yaml => test-loop-vfat-git-annex.yaml} | 18 +- .github/workflows/test-loop-vfat-git.yaml | 40 ++++ .../workflows/test-loop-vfat-pjdfstest.yaml | 40 ++++ .../workflows/test-loop-vfat-stress-ng.yaml | 40 ++++ ...{test-nfs.yaml => test-nfs-git-annex.yaml} | 17 +- .github/workflows/test-nfs-git.yaml | 40 ++++ .github/workflows/test-nfs-pjdfstest.yaml | 40 ++++ .github/workflows/test-nfs-stress-ng.yaml | 40 ++++ README.md | 134 +++++++++--- bin/ci/dump-failure-logs.sh | 29 ++- bin/ci/gen-dispatchers.sh | 194 ++++++++++++++++++ bin/ci/install-target.sh | 134 ++++++++++++ bin/ci/matrix.sh | 137 +++++++++++++ bin/ci/run-under.sh | 58 ++++-- bin/ci/target-git-annex.sh | 36 ++++ bin/ci/target-git.sh | 88 ++++++++ bin/ci/target-pjdfstest.sh | 67 ++++++ bin/ci/target-stress-ng.sh | 155 ++++++++++++++ provision/setup.sh | 18 ++ 32 files changed, 1704 insertions(+), 88 deletions(-) rename .github/workflows/{test-beegfs-7.4.6.yaml => test-beegfs-7.4.6-git-annex.yaml} (54%) create mode 100644 .github/workflows/test-beegfs-7.4.6-git.yaml create mode 100644 .github/workflows/test-beegfs-7.4.6-pjdfstest.yaml create mode 100644 .github/workflows/test-beegfs-7.4.6-stress-ng.yaml rename .github/workflows/{test-beegfs-8.1.0.yaml => test-beegfs-8.1.0-git-annex.yaml} (54%) create mode 100644 .github/workflows/test-beegfs-8.1.0-git.yaml create mode 100644 .github/workflows/test-beegfs-8.1.0-pjdfstest.yaml create mode 100644 .github/workflows/test-beegfs-8.1.0-stress-ng.yaml rename .github/workflows/{test-loop-ext4.yaml => test-loop-ext4-git-annex.yaml} (53%) create mode 100644 .github/workflows/test-loop-ext4-git.yaml create mode 100644 .github/workflows/test-loop-ext4-pjdfstest.yaml create mode 100644 .github/workflows/test-loop-ext4-stress-ng.yaml rename .github/workflows/{test-loop-vfat.yaml => test-loop-vfat-git-annex.yaml} (53%) create mode 100644 .github/workflows/test-loop-vfat-git.yaml create mode 100644 .github/workflows/test-loop-vfat-pjdfstest.yaml create mode 100644 .github/workflows/test-loop-vfat-stress-ng.yaml rename .github/workflows/{test-nfs.yaml => test-nfs-git-annex.yaml} (54%) create mode 100644 .github/workflows/test-nfs-git.yaml create mode 100644 .github/workflows/test-nfs-pjdfstest.yaml create mode 100644 .github/workflows/test-nfs-stress-ng.yaml create mode 100755 bin/ci/gen-dispatchers.sh create mode 100755 bin/ci/install-target.sh create mode 100644 bin/ci/matrix.sh create mode 100755 bin/ci/target-git-annex.sh create mode 100755 bin/ci/target-git.sh create mode 100755 bin/ci/target-pjdfstest.sh create mode 100755 bin/ci/target-stress-ng.sh diff --git a/.github/workflows/_test-under.yaml b/.github/workflows/_test-under.yaml index d9e7d93..22b1389 100644 --- a/.github/workflows/_test-under.yaml +++ b/.github/workflows/_test-under.yaml @@ -1,14 +1,19 @@ -name: (reusable) git-annex test under +name: (reusable) under # Reusable body for the eval-under framework. Takes a backend -# (beegfs / nfs / loop) plus a backend-version (a BeeGFS point release, -# a loop filesystem name, or `n/a` for NFS), installs the matching -# client-side packages, fetches the freshest con/git-annex daily, and -# runs the full `git annex test` under the requested filesystem. +# (beegfs / nfs / loop), a backend-version (a BeeGFS point release, +# a loop filesystem name, or `n/a` for NFS), and a target (which test +# suite to run under that filesystem): git-annex, git, stress-ng, or +# pjdfstest. # -# Invoked by per-flavour dispatcher workflows so each backend/version +# Installs the backend's client-side packages, prepares the target +# (apt package or pinned-tag source build), and runs the suite with +# TMPDIR/HOME on the mounted filesystem. +# +# Invoked by per-cell dispatcher workflows so each backend x target # combination gets its own badge URL. (GitHub only exposes one badge -# per workflow file, not per matrix cell.) +# per workflow file, not per matrix cell.) Those dispatchers are +# generated -- see bin/ci/gen-dispatchers.sh. # # All non-trivial shell logic lives in bin/ci/*.sh so it can be shellcheck'd # and exercised locally in the Vagrant VM. See .claude/CLAUDE.md. @@ -27,6 +32,13 @@ on: For nfs: literal "n/a" (no version to pin). required: true type: string + target: + description: >- + Test suite to run under the backend: + git-annex | git | stress-ng | pjdfstest. + required: false + default: "git-annex" + type: string jobs: test: @@ -45,7 +57,13 @@ jobs: - name: Install backend client dependencies run: bin/ci/install-backend.sh "${{ inputs.backend }}" "${{ inputs.backend-version }}" + - name: Install target ${{ inputs.target }} + run: bin/ci/install-target.sh "${{ inputs.target }}" + + # Only the git-annex target needs the daily build (and the token + # that fetching it requires). - name: Fetch latest git-annex daily build from con/git-annex + if: inputs.target == 'git-annex' env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: bin/ci/install-git-annex-daily.sh @@ -55,18 +73,23 @@ jobs: git config --global user.email test@github.land git config --global user.name "GitHub Almighty" - - name: Run git annex test under ${{ inputs.backend }} - run: sudo -E bin/ci/run-under.sh "${{ inputs.backend }}" "${{ inputs.backend-version }}" + - name: Run ${{ inputs.target }} under ${{ inputs.backend }} + run: >- + sudo -E bin/ci/run-under.sh + "${{ inputs.backend }}" "${{ inputs.backend-version }}" "${{ inputs.target }}" - name: Dump failure logs if: failure() - run: bin/ci/dump-failure-logs.sh "${{ inputs.backend }}" "${{ inputs.backend-version }}" + run: >- + bin/ci/dump-failure-logs.sh + "${{ inputs.backend }}" "${{ inputs.backend-version }}" "${{ inputs.target }}" - name: Upload logs if: always() uses: actions/upload-artifact@v4 with: - name: logs-${{ inputs.backend }}-${{ inputs.backend-version }} + name: logs-${{ inputs.backend }}-${{ inputs.backend-version }}-${{ inputs.target }} path: | /var/log/beegfs-* + /opt/eval-under-src/git/t/test-results/** if-no-files-found: ignore diff --git a/.github/workflows/test-beegfs-7.4.6.yaml b/.github/workflows/test-beegfs-7.4.6-git-annex.yaml similarity index 54% rename from .github/workflows/test-beegfs-7.4.6.yaml rename to .github/workflows/test-beegfs-7.4.6-git-annex.yaml index 82b83cf..91a6188 100644 --- a/.github/workflows/test-beegfs-7.4.6.yaml +++ b/.github/workflows/test-beegfs-7.4.6-git-annex.yaml @@ -1,7 +1,10 @@ -name: BeeGFS 7.4.6 +name: BeeGFS 7.4.6 / git-annex test -# Thin dispatcher so this flavour gets a distinct README badge. -# Delegates all work to _test-under.yaml (reusable). +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-7.4.6 x git-annex) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). on: push: @@ -11,20 +14,20 @@ on: - "fixtures/beegfs/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-beegfs-7.4.6.yaml" + - ".github/workflows/test-beegfs-7.4.6-git-annex.yaml" pull_request: paths: - "bin/**" - "fixtures/beegfs/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-beegfs-7.4.6.yaml" + - ".github/workflows/test-beegfs-7.4.6-git-annex.yaml" workflow_dispatch: schedule: - cron: "17 5 * * 1" # Mon 05:17 UTC concurrency: - group: test-beegfs-7.4.6-${{ github.ref }} + group: beegfs-7.4.6-git-annex-${{ github.ref }} cancel-in-progress: true jobs: @@ -33,6 +36,7 @@ jobs: with: backend: "beegfs" backend-version: "7.4.6" + target: "git-annex" permissions: contents: read actions: read diff --git a/.github/workflows/test-beegfs-7.4.6-git.yaml b/.github/workflows/test-beegfs-7.4.6-git.yaml new file mode 100644 index 0000000..8dc32e3 --- /dev/null +++ b/.github/workflows/test-beegfs-7.4.6-git.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 7.4.6 / git testsuite + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-7.4.6 x git) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-git.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-git.yaml" + workflow_dispatch: + schedule: + - cron: "32 5 * * 1" # Mon 05:32 UTC + +concurrency: + group: beegfs-7.4.6-git-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "7.4.6" + target: "git" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-beegfs-7.4.6-pjdfstest.yaml b/.github/workflows/test-beegfs-7.4.6-pjdfstest.yaml new file mode 100644 index 0000000..5247035 --- /dev/null +++ b/.github/workflows/test-beegfs-7.4.6-pjdfstest.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 7.4.6 / pjdfstest + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-7.4.6 x pjdfstest) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-pjdfstest.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-pjdfstest.yaml" + workflow_dispatch: + schedule: + - cron: "2 5 * * 1" # Mon 05:02 UTC + +concurrency: + group: beegfs-7.4.6-pjdfstest-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "7.4.6" + target: "pjdfstest" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-beegfs-7.4.6-stress-ng.yaml b/.github/workflows/test-beegfs-7.4.6-stress-ng.yaml new file mode 100644 index 0000000..dbd6738 --- /dev/null +++ b/.github/workflows/test-beegfs-7.4.6-stress-ng.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 7.4.6 / stress-ng + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-7.4.6 x stress-ng) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-stress-ng.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-7.4.6-stress-ng.yaml" + workflow_dispatch: + schedule: + - cron: "47 5 * * 1" # Mon 05:47 UTC + +concurrency: + group: beegfs-7.4.6-stress-ng-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "7.4.6" + target: "stress-ng" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-beegfs-8.1.0.yaml b/.github/workflows/test-beegfs-8.1.0-git-annex.yaml similarity index 54% rename from .github/workflows/test-beegfs-8.1.0.yaml rename to .github/workflows/test-beegfs-8.1.0-git-annex.yaml index 6705492..9c562db 100644 --- a/.github/workflows/test-beegfs-8.1.0.yaml +++ b/.github/workflows/test-beegfs-8.1.0-git-annex.yaml @@ -1,7 +1,10 @@ -name: BeeGFS 8.1.0 +name: BeeGFS 8.1.0 / git-annex test -# Thin dispatcher so this flavour gets a distinct README badge. -# Delegates all work to _test-under.yaml (reusable). +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-8.1.0 x git-annex) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). on: push: @@ -11,20 +14,20 @@ on: - "fixtures/beegfs/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-beegfs-8.1.0.yaml" + - ".github/workflows/test-beegfs-8.1.0-git-annex.yaml" pull_request: paths: - "bin/**" - "fixtures/beegfs/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-beegfs-8.1.0.yaml" + - ".github/workflows/test-beegfs-8.1.0-git-annex.yaml" workflow_dispatch: schedule: - cron: "17 5 * * 1" # Mon 05:17 UTC concurrency: - group: test-beegfs-8.1.0-${{ github.ref }} + group: beegfs-8.1.0-git-annex-${{ github.ref }} cancel-in-progress: true jobs: @@ -33,6 +36,7 @@ jobs: with: backend: "beegfs" backend-version: "8.1.0" + target: "git-annex" permissions: contents: read actions: read diff --git a/.github/workflows/test-beegfs-8.1.0-git.yaml b/.github/workflows/test-beegfs-8.1.0-git.yaml new file mode 100644 index 0000000..78927c4 --- /dev/null +++ b/.github/workflows/test-beegfs-8.1.0-git.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 8.1.0 / git testsuite + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-8.1.0 x git) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-git.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-git.yaml" + workflow_dispatch: + schedule: + - cron: "32 5 * * 1" # Mon 05:32 UTC + +concurrency: + group: beegfs-8.1.0-git-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "8.1.0" + target: "git" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-beegfs-8.1.0-pjdfstest.yaml b/.github/workflows/test-beegfs-8.1.0-pjdfstest.yaml new file mode 100644 index 0000000..748b13f --- /dev/null +++ b/.github/workflows/test-beegfs-8.1.0-pjdfstest.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 8.1.0 / pjdfstest + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-8.1.0 x pjdfstest) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-pjdfstest.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-pjdfstest.yaml" + workflow_dispatch: + schedule: + - cron: "2 5 * * 1" # Mon 05:02 UTC + +concurrency: + group: beegfs-8.1.0-pjdfstest-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "8.1.0" + target: "pjdfstest" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-beegfs-8.1.0-stress-ng.yaml b/.github/workflows/test-beegfs-8.1.0-stress-ng.yaml new file mode 100644 index 0000000..371ec5f --- /dev/null +++ b/.github/workflows/test-beegfs-8.1.0-stress-ng.yaml @@ -0,0 +1,42 @@ +name: BeeGFS 8.1.0 / stress-ng + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (beegfs-8.1.0 x stress-ng) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-stress-ng.yaml" + pull_request: + paths: + - "bin/**" + - "fixtures/beegfs/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-beegfs-8.1.0-stress-ng.yaml" + workflow_dispatch: + schedule: + - cron: "47 5 * * 1" # Mon 05:47 UTC + +concurrency: + group: beegfs-8.1.0-stress-ng-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "beegfs" + backend-version: "8.1.0" + target: "stress-ng" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-ext4.yaml b/.github/workflows/test-loop-ext4-git-annex.yaml similarity index 53% rename from .github/workflows/test-loop-ext4.yaml rename to .github/workflows/test-loop-ext4-git-annex.yaml index 3da3ad2..3fc5097 100644 --- a/.github/workflows/test-loop-ext4.yaml +++ b/.github/workflows/test-loop-ext4-git-annex.yaml @@ -1,11 +1,10 @@ -name: Loop ext4 +name: Loop ext4 / git-annex test -# Thin dispatcher so this flavour gets a distinct README badge. -# Delegates all work to _test-under.yaml (reusable). +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. # -# Baseline flavour: ext4 loop mount, close to the default Linux root -# filesystem behaviour. Regressions here point at git-annex itself -# rather than filesystem-specific edge cases. +# Thin dispatcher so this matrix cell (loop-ext4 x git-annex) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). on: push: @@ -14,19 +13,19 @@ on: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-loop-ext4.yaml" + - ".github/workflows/test-loop-ext4-git-annex.yaml" pull_request: paths: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-loop-ext4.yaml" + - ".github/workflows/test-loop-ext4-git-annex.yaml" workflow_dispatch: schedule: - cron: "17 5 * * 1" # Mon 05:17 UTC concurrency: - group: test-loop-ext4-${{ github.ref }} + group: loop-ext4-git-annex-${{ github.ref }} cancel-in-progress: true jobs: @@ -35,6 +34,7 @@ jobs: with: backend: "loop" backend-version: "ext4" + target: "git-annex" permissions: contents: read actions: read diff --git a/.github/workflows/test-loop-ext4-git.yaml b/.github/workflows/test-loop-ext4-git.yaml new file mode 100644 index 0000000..4983539 --- /dev/null +++ b/.github/workflows/test-loop-ext4-git.yaml @@ -0,0 +1,40 @@ +name: Loop ext4 / git testsuite + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-ext4 x git) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-git.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-git.yaml" + workflow_dispatch: + schedule: + - cron: "32 5 * * 1" # Mon 05:32 UTC + +concurrency: + group: loop-ext4-git-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "ext4" + target: "git" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-ext4-pjdfstest.yaml b/.github/workflows/test-loop-ext4-pjdfstest.yaml new file mode 100644 index 0000000..75a223c --- /dev/null +++ b/.github/workflows/test-loop-ext4-pjdfstest.yaml @@ -0,0 +1,40 @@ +name: Loop ext4 / pjdfstest + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-ext4 x pjdfstest) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-pjdfstest.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-pjdfstest.yaml" + workflow_dispatch: + schedule: + - cron: "2 5 * * 1" # Mon 05:02 UTC + +concurrency: + group: loop-ext4-pjdfstest-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "ext4" + target: "pjdfstest" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-ext4-stress-ng.yaml b/.github/workflows/test-loop-ext4-stress-ng.yaml new file mode 100644 index 0000000..1de9066 --- /dev/null +++ b/.github/workflows/test-loop-ext4-stress-ng.yaml @@ -0,0 +1,40 @@ +name: Loop ext4 / stress-ng + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-ext4 x stress-ng) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-stress-ng.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-ext4-stress-ng.yaml" + workflow_dispatch: + schedule: + - cron: "47 5 * * 1" # Mon 05:47 UTC + +concurrency: + group: loop-ext4-stress-ng-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "ext4" + target: "stress-ng" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-vfat.yaml b/.github/workflows/test-loop-vfat-git-annex.yaml similarity index 53% rename from .github/workflows/test-loop-vfat.yaml rename to .github/workflows/test-loop-vfat-git-annex.yaml index 71e84bf..b2dc6e8 100644 --- a/.github/workflows/test-loop-vfat.yaml +++ b/.github/workflows/test-loop-vfat-git-annex.yaml @@ -1,11 +1,10 @@ -name: Loop vfat +name: Loop vfat / git-annex test -# Thin dispatcher so this flavour gets a distinct README badge. -# Delegates all work to _test-under.yaml (reusable). +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. # -# `--fs vfat` is git-annex's canonical "crippled filesystem" flavour -# (no fifos, no symlinks, case-insensitive) and triggers adjusted- -# branch handling in git-annex init. +# Thin dispatcher so this matrix cell (loop-vfat x git-annex) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). on: push: @@ -14,19 +13,19 @@ on: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-loop-vfat.yaml" + - ".github/workflows/test-loop-vfat-git-annex.yaml" pull_request: paths: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-loop-vfat.yaml" + - ".github/workflows/test-loop-vfat-git-annex.yaml" workflow_dispatch: schedule: - cron: "17 5 * * 1" # Mon 05:17 UTC concurrency: - group: test-loop-vfat-${{ github.ref }} + group: loop-vfat-git-annex-${{ github.ref }} cancel-in-progress: true jobs: @@ -35,6 +34,7 @@ jobs: with: backend: "loop" backend-version: "vfat" + target: "git-annex" permissions: contents: read actions: read diff --git a/.github/workflows/test-loop-vfat-git.yaml b/.github/workflows/test-loop-vfat-git.yaml new file mode 100644 index 0000000..481d482 --- /dev/null +++ b/.github/workflows/test-loop-vfat-git.yaml @@ -0,0 +1,40 @@ +name: Loop vfat / git testsuite + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-vfat x git) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-git.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-git.yaml" + workflow_dispatch: + schedule: + - cron: "32 5 * * 1" # Mon 05:32 UTC + +concurrency: + group: loop-vfat-git-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "vfat" + target: "git" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-vfat-pjdfstest.yaml b/.github/workflows/test-loop-vfat-pjdfstest.yaml new file mode 100644 index 0000000..a3f7474 --- /dev/null +++ b/.github/workflows/test-loop-vfat-pjdfstest.yaml @@ -0,0 +1,40 @@ +name: Loop vfat / pjdfstest + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-vfat x pjdfstest) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-pjdfstest.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-pjdfstest.yaml" + workflow_dispatch: + schedule: + - cron: "2 5 * * 1" # Mon 05:02 UTC + +concurrency: + group: loop-vfat-pjdfstest-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "vfat" + target: "pjdfstest" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-loop-vfat-stress-ng.yaml b/.github/workflows/test-loop-vfat-stress-ng.yaml new file mode 100644 index 0000000..c6dbd46 --- /dev/null +++ b/.github/workflows/test-loop-vfat-stress-ng.yaml @@ -0,0 +1,40 @@ +name: Loop vfat / stress-ng + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (loop-vfat x stress-ng) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-stress-ng.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-loop-vfat-stress-ng.yaml" + workflow_dispatch: + schedule: + - cron: "47 5 * * 1" # Mon 05:47 UTC + +concurrency: + group: loop-vfat-stress-ng-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "loop" + backend-version: "vfat" + target: "stress-ng" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-nfs.yaml b/.github/workflows/test-nfs-git-annex.yaml similarity index 54% rename from .github/workflows/test-nfs.yaml rename to .github/workflows/test-nfs-git-annex.yaml index e62ad29..5a9d81d 100644 --- a/.github/workflows/test-nfs.yaml +++ b/.github/workflows/test-nfs-git-annex.yaml @@ -1,10 +1,10 @@ -name: NFS +name: NFS (localhost) / git-annex test -# Thin dispatcher so this flavour gets a distinct README badge. -# Delegates all work to _test-under.yaml (reusable). +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. # -# NFS backend uses `nfs-kernel-server` for a localhost loopback export; -# no version to pin (kernel + nfs-utils are ambient on the runner OS). +# Thin dispatcher so this matrix cell (nfs x git-annex) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). on: push: @@ -13,19 +13,19 @@ on: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-nfs.yaml" + - ".github/workflows/test-nfs-git-annex.yaml" pull_request: paths: - "bin/**" - "provision/**" - ".github/workflows/_test-under.yaml" - - ".github/workflows/test-nfs.yaml" + - ".github/workflows/test-nfs-git-annex.yaml" workflow_dispatch: schedule: - cron: "17 5 * * 1" # Mon 05:17 UTC concurrency: - group: test-nfs-${{ github.ref }} + group: nfs-git-annex-${{ github.ref }} cancel-in-progress: true jobs: @@ -34,6 +34,7 @@ jobs: with: backend: "nfs" backend-version: "n/a" + target: "git-annex" permissions: contents: read actions: read diff --git a/.github/workflows/test-nfs-git.yaml b/.github/workflows/test-nfs-git.yaml new file mode 100644 index 0000000..01d48fa --- /dev/null +++ b/.github/workflows/test-nfs-git.yaml @@ -0,0 +1,40 @@ +name: NFS (localhost) / git testsuite + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (nfs x git) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-git.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-git.yaml" + workflow_dispatch: + schedule: + - cron: "32 5 * * 1" # Mon 05:32 UTC + +concurrency: + group: nfs-git-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "nfs" + backend-version: "n/a" + target: "git" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-nfs-pjdfstest.yaml b/.github/workflows/test-nfs-pjdfstest.yaml new file mode 100644 index 0000000..b461b86 --- /dev/null +++ b/.github/workflows/test-nfs-pjdfstest.yaml @@ -0,0 +1,40 @@ +name: NFS (localhost) / pjdfstest + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (nfs x pjdfstest) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-pjdfstest.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-pjdfstest.yaml" + workflow_dispatch: + schedule: + - cron: "2 5 * * 1" # Mon 05:02 UTC + +concurrency: + group: nfs-pjdfstest-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "nfs" + backend-version: "n/a" + target: "pjdfstest" + permissions: + contents: read + actions: read diff --git a/.github/workflows/test-nfs-stress-ng.yaml b/.github/workflows/test-nfs-stress-ng.yaml new file mode 100644 index 0000000..2012d11 --- /dev/null +++ b/.github/workflows/test-nfs-stress-ng.yaml @@ -0,0 +1,40 @@ +name: NFS (localhost) / stress-ng + +# GENERATED FILE -- edit bin/ci/matrix.sh and re-run +# bin/ci/gen-dispatchers.sh instead of editing this by hand. +# +# Thin dispatcher so this matrix cell (nfs x stress-ng) gets a distinct +# README badge. All work is delegated to _test-under.yaml (reusable). + +on: + push: + branches: [main, master] + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-stress-ng.yaml" + pull_request: + paths: + - "bin/**" + - "provision/**" + - ".github/workflows/_test-under.yaml" + - ".github/workflows/test-nfs-stress-ng.yaml" + workflow_dispatch: + schedule: + - cron: "47 5 * * 1" # Mon 05:47 UTC + +concurrency: + group: nfs-stress-ng-${{ github.ref }} + cancel-in-progress: true + +jobs: + call: + uses: ./.github/workflows/_test-under.yaml + with: + backend: "nfs" + backend-version: "n/a" + target: "stress-ng" + permissions: + contents: read + actions: read diff --git a/README.md b/README.md index c4d2114..476a1f9 100644 --- a/README.md +++ b/README.md @@ -7,23 +7,81 @@ timestamp granularity, locking, permissions, adjusted-branch fallbacks, `root_squash` interactions) in tools like git-annex, DataLad, and rsync -- classes of bugs that don't show up in plain-ext4 CI. -git-annex is the immediate demo target: this repo runs the full -`git annex test` suite against every backend on every push. The harness -itself is backend-agnostic -- new backends are dropped in as -`bin/eval-under-` scripts (see below). +git-annex is the immediate demo target: this repo runs `git annex test` +-- plus git's own testsuite, pjdfstest, and a curated set of stress-ng +filesystem stressors -- against every backend on every push. The harness +itself is both backend- and suite-agnostic: new filesystems drop in as +`bin/eval-under-` scripts, new suites as `bin/ci/target-.sh` +(see below). ## CI status -| Backend | Status | -| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| BeeGFS 7.4.6 | [![BeeGFS 7.4.6](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-beegfs-7.4.6.yaml/badge.svg)](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-beegfs-7.4.6.yaml) | -| BeeGFS 8.1.0 | [![BeeGFS 8.1.0](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-beegfs-8.1.0.yaml/badge.svg)](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-beegfs-8.1.0.yaml) | -| NFS (localhost) | [![NFS](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-nfs.yaml/badge.svg)](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-nfs.yaml) | -| Loop vfat | [![Loop vfat](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-loop-vfat.yaml/badge.svg)](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-loop-vfat.yaml) | -| Loop ext4 | [![Loop ext4](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-loop-ext4.yaml/badge.svg)](https://github.com/yarikoptic/beegfs-test/actions/workflows/test-loop-ext4.yaml) | - -Each row runs the **full `git annex test`** against a fresh daily build -from [con/git-annex](https://github.com/con/git-annex). + +| Backend | git-annex test | git testsuite | stress-ng | pjdfstest | +| --- | --- | --- | --- | --- | +| BeeGFS 7.4.6 | [![BeeGFS 7.4.6 / git-annex](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-git-annex.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-git-annex.yaml) | [![BeeGFS 7.4.6 / git](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-git.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-git.yaml) | [![BeeGFS 7.4.6 / stress-ng](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-stress-ng.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-stress-ng.yaml) | [![BeeGFS 7.4.6 / pjdfstest](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-pjdfstest.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-7.4.6-pjdfstest.yaml) | +| BeeGFS 8.1.0 | [![BeeGFS 8.1.0 / git-annex](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-git-annex.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-git-annex.yaml) | [![BeeGFS 8.1.0 / git](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-git.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-git.yaml) | [![BeeGFS 8.1.0 / stress-ng](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-stress-ng.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-stress-ng.yaml) | [![BeeGFS 8.1.0 / pjdfstest](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-pjdfstest.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-beegfs-8.1.0-pjdfstest.yaml) | +| NFS (localhost) | [![NFS (localhost) / git-annex](https://github.com/con/eval-under/actions/workflows/test-nfs-git-annex.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-nfs-git-annex.yaml) | [![NFS (localhost) / git](https://github.com/con/eval-under/actions/workflows/test-nfs-git.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-nfs-git.yaml) | [![NFS (localhost) / stress-ng](https://github.com/con/eval-under/actions/workflows/test-nfs-stress-ng.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-nfs-stress-ng.yaml) | [![NFS (localhost) / pjdfstest](https://github.com/con/eval-under/actions/workflows/test-nfs-pjdfstest.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-nfs-pjdfstest.yaml) | +| Loop vfat | [![Loop vfat / git-annex](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-git-annex.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-git-annex.yaml) | [![Loop vfat / git](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-git.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-git.yaml) | [![Loop vfat / stress-ng](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-stress-ng.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-stress-ng.yaml) | [![Loop vfat / pjdfstest](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-pjdfstest.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-vfat-pjdfstest.yaml) | +| Loop ext4 | [![Loop ext4 / git-annex](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-git-annex.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-git-annex.yaml) | [![Loop ext4 / git](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-git.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-git.yaml) | [![Loop ext4 / stress-ng](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-stress-ng.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-stress-ng.yaml) | [![Loop ext4 / pjdfstest](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-pjdfstest.yaml/badge.svg)](https://github.com/con/eval-under/actions/workflows/test-loop-ext4-pjdfstest.yaml) | + + +Rows are **backends** (which filesystem the work happens on), columns +are **targets** (which suite is run on it). Every cell is its own +workflow file so it gets its own badge -- GitHub publishes one badge per +workflow file, not per matrix cell. Those files are generated from +`bin/ci/matrix.sh` by `bin/ci/gen-dispatchers.sh`; don't edit them by +hand. + +A red cell is not automatically a bug: vfat has no symlinks, ownership, +or xattrs, and NFS has its own locking and close-to-open rules. The +matrix exists to make *which* filesystem breaks *which* layer visible at +a glance. + +## Test targets + +Budgets below are what the suite itself costs on the loop-ext4 cell; a +sync-heavy backend (NFS, BeeGFS) runs the same work considerably slower, +which is why each target carries its own timeout in `bin/ci/matrix.sh`. + +| Target | What it runs | Prep | Budget | +| ----------- | ----------------------------------------------------- | ----------------------------------- | ------ | +| `git-annex` | The full `git annex test` suite | Daily build from [con/git-annex][ga] | tens of minutes | +| `git` | 174 scripts of git's own testsuite (`t0*.sh t1*.sh`, ~10k assertions) | Source build, pinned tag | ~4 min build + ~2 min on ext4 | +| `stress-ng` | 20 curated filesystem stressors, one at a time, `--verify` | `apt install stress-ng` | ~2 min | +| `pjdfstest` | POSIX conformance: 238 scripts, ~8800 assertions | Source build, pinned commit | ~3.5 min on ext4 | + +[ga]: https://github.com/con/git-annex + +Why these four, in order of how deep they sit: + +- **stress-ng** is the cheapest and the bluntest: it hammers `rename`, + `link`/`symlink`, `locka`/`lockf`, `xattr`, `chmod`/`chown`, `utime` + with verification on, so a filesystem that returns success while doing + the wrong thing is caught before anything is built on top of it. Each + stressor runs separately, so "vfat has no xattrs" reports as a skip + rather than poisoning the run. +- **pjdfstest** is the specification check: it tells you *which syscall* + returned *which errno* where POSIX says otherwise. Where `git annex + test` says "something is wrong", this says `rename` returns `EEXIST` + instead of `ENOTEMPTY`. +- **git** is the integration check for the layer git-annex actually + stands on -- index, refs, object store, attributes. Git's suite is the + canonical filesystem-picky testbed and is maintained far more + rigorously than anything hand-written here. Only the trash directories + (where the tests' file operations happen) live on the mount; the build + itself stays on the runner's disk. +- **git-annex** is the original motivation and the top of the stack. + +Pinned upstream refs live in `bin/ci/matrix.sh` and are bumped +deliberately: with a moving testsuite, a newly-red cell is ambiguous -- +did the filesystem regress, or did upstream add a test? git is pinned to +a release tag. pjdfstest is pinned to a commit rather than its one +upstream tag (`0.1`, 2016), which no longer compiles: `major()`, +`minor()` and `makedev()` moved to `` in glibc 2.28 and +the tree builds with `-Werror`, so those implicit declarations are hard +errors on any current toolchain. Master builds clean, so we pin a commit +on it instead of carrying a patch. ## Motivation @@ -37,10 +95,10 @@ doesn't ([`eval_under_nfs`](https://github.com/datalad/datalad/blob/maint/tools/ [`eval_under_testloopfs`](https://github.com/datalad/datalad/blob/maint/tools/eval_under_testloopfs)). This repo consolidates and generalises that pattern: one dispatcher -(`bin/eval-under`), one reusable CI workflow (`_test-under.yaml`), and -a small backend script per filesystem. New filesystems slot in -uniformly; the git-annex-under-BeeGFS coverage that motivated the repo -is now just one of several dispatcher workflows. +(`bin/eval-under`), one reusable CI workflow (`_test-under.yaml`), a +small backend script per filesystem, and a small target script per test +suite. Both axes slot in uniformly; the git-annex-under-BeeGFS coverage +that motivated the repo is now one cell of a generated matrix. ## CLI usage @@ -82,8 +140,12 @@ the full flag / env-var / default table per backend. | `fixtures/beegfs/docker-compose-v7.yml` | BeeGFS v7 test cluster (mgmtd + meta + storage), `network_mode: host` | | `fixtures/beegfs/docker-compose-v8.yml` | Same, for BeeGFS v8.x (different mgmtd command style / gRPC control plane) | | `fixtures/beegfs/beegfs-*.conf.template` | Minimal client + helperd confs for the throwaway cluster | -| `.github/workflows/_test-under.yaml` | Reusable workflow parameterised on `backend` + `backend-version` | -| `.github/workflows/test-*.yaml` | Per-flavour dispatchers (one badge each) | +| `bin/ci/matrix.sh` | Single source of truth: backends x targets, pinned upstream refs, per-target knobs | +| `bin/ci/install-target.sh` | Runner-side prep for a target (apt package, or source build at a pinned tag) | +| `bin/ci/target-.sh` | The suite itself, run inside the mount by `bin/ci/run-under.sh` | +| `bin/ci/gen-dispatchers.sh` | Regenerates the per-cell workflows + the README matrix from `matrix.sh` | +| `.github/workflows/_test-under.yaml` | Reusable workflow parameterised on `backend` + `backend-version` + `target` | +| `.github/workflows/test-*.yaml` | Generated per-cell dispatchers (one badge each) | | `drafts/git-annex-test-beegfs.yaml` | Copy-target workflow for `datalad/git-annex` (external PR target) | ## Local iteration (VM) @@ -102,6 +164,13 @@ sudo bin/eval-under beegfs --set-home -- bash -c ' ' sudo bin/eval-under nfs --set-home -- git annex test sudo bin/eval-under loop --fs vfat --set-home -- git annex test + +# Or run a whole CI cell exactly as the runner would. install-target.sh +# is the one-off prep (source builds land in $EVAL_UNDER_SRC_DIR, not on +# the mount); run-under.sh then wraps the suite in the backend. +bin/ci/install-target.sh pjdfstest +sudo -E bin/ci/run-under.sh loop ext4 pjdfstest +sudo -E bin/ci/run-under.sh nfs n/a stress-ng ``` Optional: install `act` in the VM to replay the GitHub workflow locally. @@ -127,9 +196,28 @@ filesystem testing. 3. At the end, run the wrapped command with `TMPDIR`, `DATALAD_TESTS_TEMP_DIR`, and (if `--set-home`) `HOME` pointing at the mount. -4. Add a `.github/workflows/test-.yaml` dispatcher and a - row to the CI badge table above. -5. Update `provision/setup.sh` if the backend needs new host packages. +4. Add a row to `EVAL_UNDER_BACKENDS` in `bin/ci/matrix.sh`, then run + `bin/ci/gen-dispatchers.sh` -- that writes the per-target dispatcher + workflows and refreshes the README matrix above. Commit the result. +5. Teach `bin/ci/install-backend.sh` how to install its client packages. +6. Update `provision/setup.sh` if the backend needs new host packages. + +## Adding a new test target + +1. Write `bin/ci/target-.sh`. It runs *inside* the mount, with + `TMPDIR` (and `HOME`, with `--set-home`) already pointing at the + filesystem under test. Exit non-zero on failure; skip -- loudly -- + rather than fail on operations the filesystem genuinely cannot do. +2. Teach `bin/ci/install-target.sh` how to prepare it on the runner. + Build source trees into `$EVAL_UNDER_SRC_DIR` (the runner's own disk), + never onto the mount: only the suite's I/O should exercise the + filesystem under test. Pin any upstream checkout to a tag. +3. Add the name to `EVAL_UNDER_TARGETS` in `bin/ci/matrix.sh`, along with + its `target_label`, `target_timeout`, `target_loop_size_mb`, and + `target_cron_minute` cases. +4. Run `bin/ci/gen-dispatchers.sh` and commit the generated workflows + + README matrix column. +5. `shellcheck bin/ci/*.sh bin/eval-under*` before committing. ## Upstream targets diff --git a/bin/ci/dump-failure-logs.sh b/bin/ci/dump-failure-logs.sh index 2114638..d4330e8 100755 --- a/bin/ci/dump-failure-logs.sh +++ b/bin/ci/dump-failure-logs.sh @@ -9,15 +9,20 @@ # without masking the underlying failure. # # usage: -# bin/ci/dump-failure-logs.sh +# bin/ci/dump-failure-logs.sh [target] # -# beegfs: `docker compose logs` + dmesg-filtered-for-beegfs -# nfs/loop: full dmesg tail +# backend side: +# beegfs: `docker compose logs` + dmesg-filtered-for-beegfs +# nfs/loop: full dmesg tail +# target side: +# git: the failing tests' own output from t/test-results/ set -uo pipefail BACKEND="${1:?backend required}" VERSION="${2:-}" +TARGET="${3:-}" +SRC_DIR="${EVAL_UNDER_SRC_DIR:-/opt/eval-under-src}" case "$BACKEND" in beegfs) @@ -41,4 +46,22 @@ case "$BACKEND" in ;; esac +# git's testsuite keeps a per-script .out next to a .counts summary; the +# aggregated `make` output only says which scripts failed, not why. +if [ "$TARGET" = "git" ]; then + results="$SRC_DIR/git/t/test-results" + echo "=== git testsuite failures ($results) ===" + if [ -d "$results" ]; then + for counts in "$results"/*.counts; do + [ -e "$counts" ] || continue + grep -q '^failed 0$' "$counts" && continue + echo "--- $(basename "${counts%.counts}") ---" + cat "$counts" || true + tail -100 "${counts%.counts}.out" 2>/dev/null || true + done + else + echo "no test-results directory (the suite never started?)" + fi +fi + exit 0 diff --git a/bin/ci/gen-dispatchers.sh b/bin/ci/gen-dispatchers.sh new file mode 100755 index 0000000..c46720b --- /dev/null +++ b/bin/ci/gen-dispatchers.sh @@ -0,0 +1,194 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# Maintenance helper (NOT a runner-side step): regenerate the per-cell +# dispatcher workflows and the README CI matrix from bin/ci/matrix.sh. +# +# GitHub exposes one badge per *workflow file*, not per matrix cell, so a +# backends x targets grid in the README needs one thin dispatcher file +# per cell. Hand-maintaining 20 near-identical YAML files is how they +# drift; generating them is how they don't. +# +# usage: +# bin/ci/gen-dispatchers.sh # write workflows + README table +# bin/ci/gen-dispatchers.sh --check # exit 1 if anything is stale +# +# Run it after editing EVAL_UNDER_BACKENDS / EVAL_UNDER_TARGETS in +# bin/ci/matrix.sh, and commit the result. + +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +root="$(cd "$here/../.." && pwd)" +# matrix.sh is a sourced library, resolved at runtime relative to $here. +# shellcheck source=bin/ci/matrix.sh disable=SC1091 +. "$here/matrix.sh" + +WF_DIR="$root/.github/workflows" +README="$root/README.md" +BEGIN_MARK='' +END_MARK='' + +CHECK=0 +case "${1:-}" in + --check) CHECK=1 ;; + "") ;; + *) echo "usage: $0 [--check]" >&2; exit 2 ;; +esac + +# One trap for every scratch path we create below. +scratch=() +cleanup() { [ "${#scratch[@]}" -eq 0 ] || rm -rf "${scratch[@]}"; } +trap cleanup EXIT + +outdir="$WF_DIR" +if [ "$CHECK" = 1 ]; then + outdir="$(mktemp -d)" + scratch+=("$outdir") +fi + +grep -qF "$BEGIN_MARK" "$README" || { + echo "ERROR: $README has no '$BEGIN_MARK' marker" >&2 + exit 1 +} + +badge_url() { + local file="$1" + echo "https://github.com/$EVAL_UNDER_REPO_SLUG/actions/workflows/$file" +} + +emit_dispatcher() { + local backend="$1" version="$2" label="$3" target="$4" file="$5" + local slug fixtures="" + slug="$(backend_slug "$backend" "$version")" + + # BeeGFS is the only backend with checked-in fixtures worth watching. + if [ "$backend" = "beegfs" ]; then + fixtures=' - "fixtures/beegfs/**" +' + fi + + cat > "$outdir/$file" < "$new_readme" + +# --- report / write -------------------------------------------------- + +if [ "$CHECK" = 1 ]; then + rc=0 + for file in "${generated[@]}"; do + if ! diff -q "$WF_DIR/$file" "$outdir/$file" >/dev/null 2>&1; then + echo "stale: .github/workflows/$file" >&2 + rc=1 + fi + done + # Any dispatcher that is no longer part of the matrix. + for existing in "$WF_DIR"/test-*.yaml; do + base="$(basename "$existing")" + # shellcheck disable=SC2076 # literal match is intended + if [[ ! " ${generated[*]} " =~ " $base " ]]; then + echo "orphan: .github/workflows/$base" >&2 + rc=1 + fi + done + if ! diff -q "$README" "$new_readme" >/dev/null 2>&1; then + echo "stale: README.md CI matrix" >&2 + rc=1 + fi + [ "$rc" = 0 ] && echo "up to date: ${#generated[@]} dispatchers + README matrix" + exit "$rc" +fi + +cp "$new_readme" "$README" +echo "wrote ${#generated[@]} dispatchers into .github/workflows/ and refreshed the README matrix" +printf ' %s\n' "${generated[@]}" diff --git a/bin/ci/install-target.sh b/bin/ci/install-target.sh new file mode 100755 index 0000000..321fe88 --- /dev/null +++ b/bin/ci/install-target.sh @@ -0,0 +1,134 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# Runner-side install/build of an eval-under *test target* (the suite we +# run under the backend filesystem). Counterpart of install-backend.sh, +# which installs the backend (the filesystem) instead. +# +# Everything built here lands on the runner's own disk +# ($EVAL_UNDER_SRC_DIR), never on the filesystem under test -- only the +# suite's own I/O should exercise the mount. +# +# usage: +# bin/ci/install-target.sh +# +# target = git-annex | git | stress-ng | pjdfstest +# +# env overrides: +# EVAL_UNDER_SRC_DIR where to clone/build (/opt/eval-under-src) +# EVAL_UNDER_GIT_REF git tag to build (see bin/ci/matrix.sh) +# EVAL_UNDER_PJDFSTEST_REF pjdfstest tag to build (see bin/ci/matrix.sh) +# +# Idempotent enough for CI re-runs: an already-built tree is left alone. + +set -euo pipefail +export DEBIAN_FRONTEND=noninteractive + +here="$(cd "$(dirname "$0")" && pwd)" +# matrix.sh is a sourced library, resolved at runtime relative to $here. +# SC1091 is only silenceable by running shellcheck with -x; disable it +# here so a plain `shellcheck bin/ci/*.sh` stays clean. +# shellcheck source=bin/ci/matrix.sh disable=SC1091 +. "$here/matrix.sh" + +TARGET="${1:?target required (git-annex|git|stress-ng|pjdfstest)}" +target_known "$TARGET" || { + echo "unknown target: $TARGET (expected: ${EVAL_UNDER_TARGETS[*]})" >&2 + exit 1 +} + +# Give unattended-upgrades a moment on ubuntu-22.04 runners rather than +# hard-failing on a dpkg lock (same rationale as install-backend.sh). +APT_LOCK_TIMEOUT=(-o "DPkg::Lock::Timeout=60") + +apt_install() { + sudo apt-get "${APT_LOCK_TIMEOUT[@]}" install -y --no-install-recommends "$@" +} + +apt_update() { + sudo apt-get "${APT_LOCK_TIMEOUT[@]}" update -qq +} + +# Shallow-fetch a pinned ref into $EVAL_UNDER_SRC_DIR/, or report +# that it is already there. Sets $checkout. +# +# init + fetch + checkout rather than `clone --branch`, because --branch +# only accepts a branch or tag name: this way a ref can equally be a +# commit SHA (which is what pjdfstest needs -- see EVAL_UNDER_PJDFSTEST_REF +# in bin/ci/matrix.sh). +checkout="" +fetch_pinned() { + local name="$1" url="$2" ref="$3" + checkout="$EVAL_UNDER_SRC_DIR/$name" + if [ -d "$checkout/.git" ]; then + echo "I: $name already checked out at $checkout" + return 0 + fi + sudo mkdir -p "$EVAL_UNDER_SRC_DIR" + sudo chown "$(id -u):$(id -g)" "$EVAL_UNDER_SRC_DIR" + echo "I: fetching $url @ $ref -> $checkout" + mkdir -p "$checkout" + git -C "$checkout" init -q + git -C "$checkout" remote add origin "$url" + git -C "$checkout" fetch -q --depth 1 origin "$ref" + git -C "$checkout" checkout -q FETCH_HEAD + git -C "$checkout" --no-pager log -1 --format='I: %H %s' +} + +install_git_annex() { + # Nothing to build: the daily git-annex build is installed by + # bin/ci/install-git-annex-daily.sh, which needs a GH token and so + # stays a separate workflow step. + echo "I: target git-annex needs no extra build step here" + echo "I: (git-annex itself comes from bin/ci/install-git-annex-daily.sh)" +} + +install_git() { + apt_update + # Build deps for a functional git (curl/expat/gettext matter for the + # t0xxx/t1xxx range: without them the corresponding tests silently skip). + apt_install build-essential gettext perl \ + zlib1g-dev libssl-dev libcurl4-openssl-dev libexpat1-dev + + fetch_pinned git https://github.com/git/git "$EVAL_UNDER_GIT_REF" + + if [ -x "$checkout/git" ]; then + echo "I: git already built ($("$checkout/git" --version))" + return 0 + fi + # NO_TCLTK: gitk/git-gui are irrelevant here and pull in a tcl + # toolchain. Everything else stays default so the testsuite's + # prerequisites resolve the way upstream CI sees them. + make -C "$checkout" -j"$(nproc)" NO_TCLTK=1 + "$checkout/git" --version +} + +install_stress_ng() { + apt_update + apt_install stress-ng + stress-ng --version +} + +install_pjdfstest() { + apt_update + apt_install build-essential autoconf automake libtool perl + + fetch_pinned pjdfstest https://github.com/pjd/pjdfstest "$EVAL_UNDER_PJDFSTEST_REF" + + if [ -x "$checkout/pjdfstest" ]; then + echo "I: pjdfstest already built" + return 0 + fi + ( cd "$checkout" && autoreconf -ifs && ./configure && make pjdfstest ) + test -x "$checkout/pjdfstest" +} + +case "$TARGET" in + git-annex) install_git_annex ;; + git) install_git ;; + stress-ng) install_stress_ng ;; + pjdfstest) install_pjdfstest ;; +esac diff --git a/bin/ci/matrix.sh b/bin/ci/matrix.sh new file mode 100644 index 0000000..592c8a2 --- /dev/null +++ b/bin/ci/matrix.sh @@ -0,0 +1,137 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# Shared CI matrix definition for eval-under. Sourced (not executed) by +# bin/ci/{install-target,run-under,target-*,gen-dispatchers}.sh so the +# cell list, the pinned upstream refs, and the per-target knobs live in +# exactly one place. +# +# The matrix is backends x targets. A "backend" answers "which +# filesystem?" (bin/eval-under-); a "target" answers "which test +# suite do we run under it?" (bin/ci/target-.sh). +# +# git-annex full `git annex test` (the suite that motivated the repo) +# git a subset of git's own testsuite, built from a pinned tag +# stress-ng curated filesystem-class stressors, one at a time +# pjdfstest POSIX filesystem conformance suite, at a pinned tag +# +# shellcheck shell=bash + +# Order matters: it is the column order of the README CI matrix. +EVAL_UNDER_TARGETS=(git-annex git stress-ng pjdfstest) + +# Backend cells, "||". Order matters: +# it is the row order of the README CI matrix. Version is the literal +# "n/a" for backends with nothing to pin (see bin/ci/install-backend.sh). +# Consumed only by bin/ci/gen-dispatchers.sh, hence "unused" here. +# shellcheck disable=SC2034 +EVAL_UNDER_BACKENDS=( + "beegfs|7.4.6|BeeGFS 7.4.6" + "beegfs|8.1.0|BeeGFS 8.1.0" + "nfs|n/a|NFS (localhost)" + "loop|vfat|Loop vfat" + "loop|ext4|Loop ext4" +) + +# GitHub repo the README badges point at. Consumed by gen-dispatchers.sh. +# shellcheck disable=SC2034 +EVAL_UNDER_REPO_SLUG="${EVAL_UNDER_REPO_SLUG:-con/eval-under}" + +# Where install-target.sh unpacks/builds source trees on the *runner* +# (deliberately not on the filesystem under test -- we want the build to +# be fast and the suite's I/O to be the only thing hitting the mount). +EVAL_UNDER_SRC_DIR="${EVAL_UNDER_SRC_DIR:-/opt/eval-under-src}" + +# Pinned upstream refs. Bump deliberately, never "latest": a moving +# testsuite makes a red cell ambiguous (did the filesystem regress, or +# did upstream add a test?). +EVAL_UNDER_GIT_REF="${EVAL_UNDER_GIT_REF:-v2.55.0}" +# pjd/pjdfstest carries exactly one tag upstream, "0.1" (2016), and it no +# longer builds: major()/minor()/makedev() moved to in +# glibc 2.28 and the tree compiles with -Werror, so the implicit +# declarations are hard errors on anything modern. Master builds clean, +# so pin a commit on master instead -- same determinism, minus the patch +# we would otherwise have to carry. Bump deliberately. +EVAL_UNDER_PJDFSTEST_REF="${EVAL_UNDER_PJDFSTEST_REF:-85a8aea9e685999ef0540392fd80535f873d7ff7}" + +# Filename-safe identifier for a backend cell: "beegfs-7.4.6", "nfs", +# "loop-vfat". Used for workflow filenames and concurrency groups. +backend_slug() { + local backend="$1" version="$2" + if [ "$version" = "n/a" ]; then + echo "$backend" + else + echo "$backend-$version" + fi +} + +target_known() { + local t + for t in "${EVAL_UNDER_TARGETS[@]}"; do + [ "$t" = "$1" ] && return 0 + done + return 1 +} + +# Human-readable column header / workflow-name fragment. +target_label() { + case "$1" in + git-annex) echo "git-annex test" ;; + git) echo "git testsuite" ;; + stress-ng) echo "stress-ng" ;; + pjdfstest) echo "pjdfstest" ;; + *) echo "$1" ;; + esac +} + +# Wall-clock budget for the wrapped suite, in seconds. Kept below the +# workflow-level timeout so a runaway suite produces a `timeout` exit +# code (and our logs) rather than a bare GitHub cancellation. +target_timeout() { + case "$1" in + git-annex) echo 2400 ;; + # 174 scripts, ~2 min on ext4; a sync-heavy backend is far slower. + git) echo 2400 ;; + stress-ng) echo 900 ;; + # ~8500 assertions, run serially; ext4 does it in minutes but a + # sync-heavy NFS or BeeGFS mount is an order of magnitude slower. + pjdfstest) echo 2400 ;; + *) echo 1800 ;; + esac +} + +# Loop-backing-image size (MB) for `eval-under loop --size`. git-annex +# keeps the historical 100MB; the others need more room (git's trash +# directories, stress-ng's scratch files) or are I/O-bound on inode +# count rather than bytes. +target_loop_size_mb() { + case "$1" in + git-annex) echo 100 ;; + git) echo 512 ;; + # copy-file's minimum --copy-file-bytes is 128M, and it needs + # source + destination, so 512 leaves uncomfortably little slack. + stress-ng) echo 768 ;; + pjdfstest) echo 256 ;; + *) echo 100 ;; + esac +} + +# Minute-of-hour for a target's weekly schedule, so the 20 cells do not +# all stampede the runner pool at 05:17 Monday. +target_cron_minute() { + case "$1" in + git-annex) echo 17 ;; + git) echo 32 ;; + stress-ng) echo 47 ;; + pjdfstest) echo 2 ;; + *) echo 17 ;; + esac +} + +# Does this target need the git-annex daily build installed on the runner? +target_needs_git_annex() { + [ "$1" = "git-annex" ] +} diff --git a/bin/ci/run-under.sh b/bin/ci/run-under.sh index bd321c5..b350f4d 100755 --- a/bin/ci/run-under.sh +++ b/bin/ci/run-under.sh @@ -4,35 +4,67 @@ # # Generated with Claude Code 2.1.233 / Claude Opus 4.7 # -# Invoke `bin/eval-under BACKEND [OPTS] --set-home` around a full -# `git annex test` run. Handles the per-backend option translation -# (--version for beegfs, --fs for loop, nothing for nfs) so the -# workflow YAML doesn't need a case. +# Invoke `bin/eval-under BACKEND [OPTS] --set-home` around one of the +# test targets in bin/ci/target-.sh. Handles the per-backend +# option translation (--version for beegfs, --fs/--size for loop, +# nothing for nfs) and the per-target timeout, so the workflow YAML +# doesn't need a case. # # usage: -# bin/ci/run-under.sh +# bin/ci/run-under.sh [target] +# +# backend = beegfs | nfs | loop +# version = for beegfs: point release (e.g. 7.4.6, 8.1.0) +# for loop: filesystem type (e.g. vfat, ext4) +# for nfs: literal "n/a" +# target = git-annex (default) | git | stress-ng | pjdfstest +# +# env overrides: +# EVAL_UNDER_TIMEOUT seconds for the wrapped suite +# EVAL_UNDER_LOOP_SIZE_MB loop backing image size +# EVAL_UNDER_SRC_DIR where install-target.sh built the suites # # Runs as the current user; expects to be launched under sudo when the # backend requires root (beegfs/loop mount, NFS server bring-up). set -euo pipefail +here="$(cd "$(dirname "$0")" && pwd)" +# matrix.sh is a sourced library, resolved at runtime relative to $here. +# shellcheck source=bin/ci/matrix.sh disable=SC1091 +. "$here/matrix.sh" + BACKEND="${1:?backend required}" VERSION="${2:?version required}" +TARGET="${3:-git-annex}" + +target_known "$TARGET" || { + echo "unknown target: $TARGET (expected: ${EVAL_UNDER_TARGETS[*]})" >&2 + exit 1 +} + +# The target scripts re-derive their own defaults from matrix.sh, but an +# override handed to us must survive into the wrapped child. +export EVAL_UNDER_SRC_DIR + +TIMEOUT="${EVAL_UNDER_TIMEOUT:-$(target_timeout "$TARGET")}" opts=() case "$BACKEND" in beegfs) opts=(--version "$VERSION") ;; - loop) opts=(--fs "$VERSION") ;; + loop) opts=(--fs "$VERSION" + --size "${EVAL_UNDER_LOOP_SIZE_MB:-$(target_loop_size_mb "$TARGET")}") ;; nfs) opts=() ;; *) echo "unknown backend: $BACKEND" >&2; exit 1 ;; esac +runner="$here/target-$TARGET.sh" +[ -x "$runner" ] || { echo "no target runner at $runner" >&2; exit 1; } + +echo "I: $(target_label "$TARGET") under $BACKEND/$VERSION (timeout ${TIMEOUT}s)" + # Sudo is expected to be in place already (workflow uses `sudo -E`); the -# script itself just forwards. Timeout keeps runaway `git annex test` -# invocations from hitting the workflow-level timeout with no signal. -# The single-quoted bash -c body is deliberate: $HOME must expand in the -# eval-under-launched child shell (where HOME=/home), not here. -# shellcheck disable=SC2016 -exec ./bin/eval-under "$BACKEND" "${opts[@]}" --set-home -- \ - timeout 2400 bash -c 'cd "$HOME" && git annex version | head -1 && git annex test' +# script itself just forwards. The timeout keeps a runaway suite from +# hitting the workflow-level timeout with no signal of its own. +exec "$here/../eval-under" "$BACKEND" "${opts[@]}" --set-home -- \ + timeout "$TIMEOUT" "$runner" diff --git a/bin/ci/target-git-annex.sh b/bin/ci/target-git-annex.sh new file mode 100755 index 0000000..c981a5b --- /dev/null +++ b/bin/ci/target-git-annex.sh @@ -0,0 +1,36 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# eval-under *target*: the full `git annex test` suite. +# +# Runs INSIDE the eval-under wrapper, i.e. with TMPDIR / HOME already +# pointing at the filesystem under test. Do not invoke directly for CI +# purposes -- go through bin/ci/run-under.sh git-annex. +# +# usage: +# bin/ci/target-git-annex.sh +# +# env (set by eval-under, honoured here): +# HOME /home -- the suite runs here, so every repo it +# creates lives on the filesystem under test +# TMPDIR + +set -euo pipefail + +cd "$HOME" + +# `git annex test` builds real repos and needs a committer identity. The +# workflow configures one for the runner user, but eval-under --set-home +# repoints HOME at the mount, so the runner's ~/.gitconfig is out of +# scope. Seed one here rather than depend on the caller's HOME. +git config --global --get user.email >/dev/null 2>&1 \ + || git config --global user.email test@github.land +git config --global --get user.name >/dev/null 2>&1 \ + || git config --global user.name "GitHub Almighty" + +git annex version | head -1 + +exec git annex test diff --git a/bin/ci/target-git.sh b/bin/ci/target-git.sh new file mode 100755 index 0000000..206237a --- /dev/null +++ b/bin/ci/target-git.sh @@ -0,0 +1,88 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# eval-under *target*: a subset of git's own testsuite. +# +# Rationale: git-annex sits on top of git plumbing, so a filesystem that +# breaks git's index/refs/object handling breaks git-annex in ways that +# `git annex test` reports only indirectly. Git's suite is the canonical +# filesystem-picky testbed and is far more rigorously maintained than +# anything we could hand-write. +# +# The git build lives on the runner's disk (bin/ci/install-target.sh); +# only the trash directories -- where every test's actual file operation +# happens -- are placed on the filesystem under test, via the suite's +# own `--root` option. +# +# Runs INSIDE the eval-under wrapper. Go through +# bin/ci/run-under.sh git. +# +# usage: +# bin/ci/target-git.sh +# +# env: +# EVAL_UNDER_SRC_DIR where install-target.sh built git +# EVAL_UNDER_GIT_TESTS glob(s) selecting the subset, evaluated inside +# git's t/ directory. Default: 't0*.sh t1*.sh' +# -- 174 scripts / ~10k assertions covering +# plumbing (t0xxx: init, index, attributes, +# object store) and the porcelain layer built +# straight on it (t1xxx: refs, config, fsck, +# worktrees, sparse-checkout). That is exactly +# the layer git-annex stands on, and it runs in +# under two minutes on ext4. Space-separated +# globs, e.g. 't00*.sh t13*.sh', to narrow it. +# EVAL_UNDER_GIT_JOBS parallel test jobs (default: nproc) +# TMPDIR -- trash directories go under here + +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +# matrix.sh is a sourced library, resolved at runtime relative to $here. +# shellcheck source=bin/ci/matrix.sh disable=SC1091 +. "$here/matrix.sh" + +SRC="$EVAL_UNDER_SRC_DIR/git" +TESTS_GLOB="${EVAL_UNDER_GIT_TESTS:-t0*.sh t1*.sh}" +JOBS="${EVAL_UNDER_GIT_JOBS:-$(nproc)}" + +[ -x "$SRC/git" ] || { + echo "ERROR: no git build at $SRC -- run bin/ci/install-target.sh git first" >&2 + exit 3 +} + +# Trash directories (one per test script) on the filesystem under test. +root="${TMPDIR:-/tmp}/git-testsuite" +mkdir -p "$root" + +# Expand the glob ourselves inside t/ and hand `make` an explicit list. +# Passing the raw glob as T= happens to work via GNU make's wildcard +# expansion of prerequisites, but an explicit list fails loudly (empty +# selection) instead of silently running the whole suite. +cd "$SRC/t" +shopt -s nullglob +# shellcheck disable=SC2206 # deliberate word-split + glob of the pattern +selected=( $TESTS_GLOB ) +shopt -u nullglob + +if [ "${#selected[@]}" -eq 0 ]; then + echo "ERROR: EVAL_UNDER_GIT_TESTS='$TESTS_GLOB' matched no tests in $SRC/t" >&2 + exit 3 +fi + +echo "I: git $("$SRC/git" --version | awk '{print $3}') @ ${EVAL_UNDER_GIT_REF}" +echo "I: ${#selected[@]} test scripts selected by '$TESTS_GLOB'" +echo "I: trash directories under $root" + +# `make -C t` runs each selected script and aggregates. --root puts the +# per-test trash directory on the mount; test-results/ stays on the +# runner disk (it is bookkeeping, not filesystem exercise). +# +# Note: the suite runs as root here (eval-under needs root to mount), so +# git's SANITY prerequisite is off and the handful of tests asserting +# "cannot write to a chmod-000 path" are skipped upstream-style rather +# than failing. +exec make -j"$JOBS" T="${selected[*]}" GIT_TEST_OPTS="--root=$root" diff --git a/bin/ci/target-pjdfstest.sh b/bin/ci/target-pjdfstest.sh new file mode 100755 index 0000000..e3721f3 --- /dev/null +++ b/bin/ci/target-pjdfstest.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# eval-under *target*: pjdfstest, the POSIX filesystem conformance suite. +# +# ~8500 assertions over chmod/chown/link/mkdir/mknod/open/rename/rmdir/ +# symlink/truncate/unlink -- i.e. exactly the syscalls whose corner cases +# (EEXIST vs ENOTEMPTY, sticky-bit rules, ctime updates, rename-onto-open-fd) +# break git-annex on network filesystems. Where `git annex test` says +# "something is wrong", pjdfstest says which syscall and which errno. +# +# Runs INSIDE the eval-under wrapper, and must run as root: most of the +# suite is about privileged-vs-unprivileged behaviour and it refuses to +# produce meaningful results otherwise. +# +# Go through bin/ci/run-under.sh pjdfstest. +# +# usage: +# bin/ci/target-pjdfstest.sh +# +# env: +# EVAL_UNDER_SRC_DIR where install-target.sh built pjdfstest +# EVAL_UNDER_PJDFSTEST_ARGS extra `prove` args (default: -r, add -v +# for the per-assertion firehose) +# TMPDIR -- the suite runs with cwd here + +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +# matrix.sh is a sourced library, resolved at runtime relative to $here. +# shellcheck source=bin/ci/matrix.sh disable=SC1091 +. "$here/matrix.sh" + +SRC="$EVAL_UNDER_SRC_DIR/pjdfstest" +PROVE_ARGS="${EVAL_UNDER_PJDFSTEST_ARGS:--r}" + +[ -x "$SRC/pjdfstest" ] || { + echo "ERROR: no pjdfstest build at $SRC -- run bin/ci/install-target.sh pjdfstest first" >&2 + exit 3 +} +[ "$(id -u)" -eq 0 ] || { + echo "ERROR: pjdfstest must run as root (privileged-vs-not is half the suite)" >&2 + exit 3 +} +command -v prove >/dev/null || { + echo "ERROR: prove(1) not found -- install perl" >&2 + exit 3 +} + +# The suite operates on the current working directory, so cwd *is* the +# filesystem under test. A dedicated subdirectory keeps its debris away +# from anything else the mount holds. +work="${TMPDIR:-/tmp}/pjdfstest" +mkdir -p "$work" +cd "$work" + +echo "I: pjdfstest @ ${EVAL_UNDER_PJDFSTEST_REF} ($SRC)" +echo "I: running in $work on $(stat -f -c %T . 2>/dev/null || echo 'unknown fs')" + +# prove exits non-zero if any assertion fails. Failures on a crippled +# filesystem (vfat has no ownership, no symlinks, no sub-second times) +# are expected and are the signal, not a flake. +# shellcheck disable=SC2086 # $PROVE_ARGS is a deliberate option list +exec prove $PROVE_ARGS "$SRC/tests" diff --git a/bin/ci/target-stress-ng.sh b/bin/ci/target-stress-ng.sh new file mode 100755 index 0000000..e62eb59 --- /dev/null +++ b/bin/ci/target-stress-ng.sh @@ -0,0 +1,155 @@ +#!/bin/bash +# SPDX-FileCopyrightText: 2026 Yaroslav Halchenko +# SPDX-License-Identifier: MIT +# +# Generated with Claude Code +# +# eval-under *target*: curated stress-ng filesystem stressors. +# +# Cheapest cell in the matrix (apt package, no build) and the broadest: +# stress-ng hammers the syscall surface git-annex depends on -- rename, +# link/symlink, locking, xattr, chmod/chown, utime -- with verification +# enabled, so a filesystem that returns success while doing the wrong +# thing is caught here rather than three layers up in a testsuite. +# +# Each stressor runs on its own so one unsupported operation (say xattr +# on vfat) is reported as a skip instead of poisoning the whole run. +# +# Runs INSIDE the eval-under wrapper. Go through +# bin/ci/run-under.sh stress-ng. +# +# usage: +# bin/ci/target-stress-ng.sh +# +# env: +# EVAL_UNDER_STRESS_NG_SECONDS per-stressor runtime (default 5) +# EVAL_UNDER_STRESS_NG_WORKERS workers per stressor (default 1) +# EVAL_UNDER_STRESS_NG_ONLY space-separated subset of stressor +# names to run (default: all below) +# TMPDIR -- scratch files go under here +# +# Exit status: 0 if every stressor passed or skipped, 1 if any failed. + +set -uo pipefail + +SECONDS_PER="${EVAL_UNDER_STRESS_NG_SECONDS:-5}" +WORKERS="${EVAL_UNDER_STRESS_NG_WORKERS:-1}" + +command -v stress-ng >/dev/null || { + echo "ERROR: stress-ng not installed -- run bin/ci/install-target.sh stress-ng" >&2 + exit 3 +} + +# "|". Only stressors that operate on +# --temp-path (i.e. on the filesystem under test) are listed; the CPU / +# VM / kernel-surface classes tell us nothing about a filesystem. +# +# Byte-size caps are deliberate: the loop backend's backing image is a +# few hundred MB, and a stressor that fills it reports ENOSPC noise +# rather than a semantics bug. +STRESSORS=( + "chmod|" + "chown|" + "copy-file|--copy-file-bytes 128M" # 128M is stress-ng's minimum + "dentry|--dentries 1024" + "dir|" + "fallocate|--fallocate-bytes 8M" + "fcntl|" + "filename|" + "hdd|--hdd-bytes 8M" + "link|" + "locka|" + "lockf|" + "mknod|" + "open|" + "rename|" + "seek|--seek-size 8M" + "symlink|" + "sync-file|--sync-file-bytes 8M" + "utime|" + "xattr|" +) + +work="${TMPDIR:-/tmp}/stress-ng" +mkdir -p "$work" +cd "$work" || exit 3 + +echo "I: $(stress-ng --version)" +echo "I: temp-path $work (${SECONDS_PER}s x ${WORKERS} worker(s) per stressor)" + +# The runner's stress-ng is whatever its distro ships (0.13.x on +# ubuntu-22.04, 0.17.x on 24.04), and both the stressor set and the +# tuning options drift between releases. Ask the binary what it supports +# rather than pinning our list to one version: a stressor this build +# lacks is a skip, and a tuning option it lacks is simply dropped. +AVAILABLE="$(stress-ng --stressors 2>/dev/null || true)" +HELP_TEXT="$(stress-ng --help 2>&1 || true)" + +supported_stressor() { + # An empty --stressors listing means this build predates the option; + # in that case assume everything is available and let it fail loudly. + [ -z "$AVAILABLE" ] && return 0 + [[ " $AVAILABLE " == *" $1 "* ]] +} + +# Echo only the "--opt value" pairs this build actually knows about. +filter_opts() { + local out=() opt val + while [ $# -gt 0 ]; do + opt="$1"; val="${2:-}"; shift 2 || shift + if [[ " $HELP_TEXT " == *" $opt "* ]]; then + out+=("$opt" "$val") + else + echo "I: dropping unsupported option $opt" >&2 + fi + done + printf '%s ' "${out[@]:-}" +} + +declare -a passed=() skipped=() failed=() + +run_one() { + local name="$1" extra="$2" rc + echo + echo "=== stress-ng --$name ===" + if ! supported_stressor "$name"; then + echo "I: not in this stress-ng build; skipping" + skipped+=("$name (not built)") + return 0 + fi + # shellcheck disable=SC2086 # both are deliberate option lists + extra="$(filter_opts $extra)" + # shellcheck disable=SC2086 # $extra is a deliberate option list + stress-ng "--$name" "$WORKERS" \ + --timeout "$SECONDS_PER" \ + --verify --metrics-brief \ + --temp-path "$work" \ + $extra + rc=$? + case "$rc" in + 0) passed+=("$name") ;; + # 3 = EXIT_NO_RESOURCE, 4 = EXIT_NOT_IMPLEMENTED. Both mean "this + # filesystem/kernel cannot do it", which is information, not a + # regression -- vfat has no xattrs and never will. + 3|4) skipped+=("$name (rc=$rc)") ;; + *) failed+=("$name (rc=$rc)") ;; + esac +} + +only="${EVAL_UNDER_STRESS_NG_ONLY:-}" +for entry in "${STRESSORS[@]}"; do + name="${entry%%|*}" + extra="${entry#*|}" + if [ -n "$only" ] && [[ " $only " != *" $name "* ]]; then + continue + fi + run_one "$name" "$extra" +done + +echo +echo "=== stress-ng summary ($work) ===" +printf 'passed (%2d): %s\n' "${#passed[@]}" "${passed[*]:-none}" +printf 'skipped (%2d): %s\n' "${#skipped[@]}" "${skipped[*]:-none}" +printf 'failed (%2d): %s\n' "${#failed[@]}" "${failed[*]:-none}" + +[ "${#failed[@]}" -eq 0 ] diff --git a/provision/setup.sh b/provision/setup.sh index 07dd6fa..d6f7ab6 100755 --- a/provision/setup.sh +++ b/provision/setup.sh @@ -10,6 +10,9 @@ # - nfs-kernel-server (eval-under-nfs) # - dosfstools + xfsprogs + btrfs-progs (eval-under-loop, per fs) # - git-annex + git (Ubuntu apt versions, enough to smoke-test the wrapper) +# - stress-ng + build deps for the git / pjdfstest targets +# (the source builds themselves stay opt-in: run +# `bin/ci/install-target.sh {git,pjdfstest}` inside the VM) # # Idempotent: safe to re-run via `vagrant provision`. @@ -40,12 +43,25 @@ apt-get install -y --no-install-recommends \ nfs-kernel-server \ dosfstools xfsprogs btrfs-progs +log "eval-under test-target dependencies" +# stress-ng: bin/ci/target-stress-ng.sh (apt is the whole install). +# autoconf/automake/libtool: building pjdfstest from its pinned tag. +# gettext + lib*-dev: building git from its pinned tag. +# The clones + builds are deliberately NOT done here (minutes of +# provisioning for something not every VM needs) -- run +# `bin/ci/install-target.sh git` / `... pjdfstest` in the VM on demand. +apt-get install -y --no-install-recommends \ + stress-ng \ + autoconf automake libtool \ + gettext zlib1g-dev libssl-dev libcurl4-openssl-dev libexpat1-dev + log "Docker CE + compose plugin" if ! command -v docker >/dev/null; then install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg \ -o /etc/apt/keyrings/docker.asc chmod a+r /etc/apt/keyrings/docker.asc + # shellcheck disable=SC1091 # /etc/os-release provided by the OS codename="$(. /etc/os-release && echo "$VERSION_CODENAME")" echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $codename stable" \ > /etc/apt/sources.list.d/docker.list @@ -59,6 +75,7 @@ systemctl enable --now docker log "BeeGFS APT repo (v${BEEGFS_VERSION})" # Prefer a list file for the current Ubuntu codename; fall back to jammy # if the newer codename isn't yet published (BeeGFS DKMS still builds). +# shellcheck disable=SC1091 # /etc/os-release provided by the OS codename="$(. /etc/os-release && echo "$VERSION_CODENAME")" list_url="https://www.beegfs.io/release/beegfs_${BEEGFS_VERSION}/dists/beegfs-${codename}.list" if ! curl -fsI "$list_url" >/dev/null 2>&1; then @@ -101,6 +118,7 @@ log "Provisioning complete" echo " git-annex : $(git-annex version | head -1)" echo " docker : $(docker --version)" echo " beegfs mod: $(modinfo -F version beegfs 2>/dev/null || echo '?')" +echo " stress-ng : $(stress-ng --version 2>/dev/null | head -1 || echo missing)" echo " mkfs.vfat : $(command -v mkfs.vfat || echo missing)" echo " mkfs.xfs : $(command -v mkfs.xfs || echo missing)" echo " exportfs : $(command -v exportfs || echo missing)" From 944f9e9e3c04e1d2add9638e7ceba2e38f317490 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 20 Aug 2026 16:54:48 +0000 Subject: [PATCH 02/12] NFS: add --no-root-squash for the root-requiring targets The NFS backend deliberately drops back to the invoking user and exports with the default root_squash: that is what a normal user's NFS home looks like, and reproducing it is half the point of the backend. git-annex and git want exactly that. pjdfstest and stress-ng do not. pjdfstest is half privileged-vs- unprivileged assertions and refuses to run non-root at all (the first CI run of this PR died on precisely that); stress-ng's chown and mknod stressors need CAP_CHOWN / CAP_MKNOD. Squashing every privileged syscall to nobody leaves both with nothing to measure. So give the backend an explicit opt-out: --no-root-squash / EVAL_UNDER_NFS_NO_ROOT_SQUASH exports with no_root_squash *and* keeps the wrapped command's privileges. matrix.sh grows target_needs_root(), and run-under.sh passes the flag for the two targets that need it. The loop and beegfs backends already run the wrapped command as root, so they are unaffected. Export options and mount options are separate namespaces -- sync/async are valid in both but no_root_squash is export-only and mount(8) rejects it -- so the two option strings are now built separately. Also trim the on-failure dmesg dump: `dmesg | tail -100` on a hosted runner is ~100 lines of hyperv/pci/apparmor boot spam that buries the actual failure. Dump warnings and errors, plus lines naming the filesystem under test. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01SHhrxxVz3TJFwYMXuxei2H --- bin/ci/dump-failure-logs.sh | 13 ++++++++++-- bin/ci/matrix.sh | 16 +++++++++++++++ bin/ci/run-under.sh | 6 +++++- bin/eval-under-nfs | 40 +++++++++++++++++++++++++++++++++---- 4 files changed, 68 insertions(+), 7 deletions(-) diff --git a/bin/ci/dump-failure-logs.sh b/bin/ci/dump-failure-logs.sh index d4330e8..f502752 100755 --- a/bin/ci/dump-failure-logs.sh +++ b/bin/ci/dump-failure-logs.sh @@ -38,8 +38,17 @@ case "$BACKEND" in sudo dmesg | grep -i beegfs | tail -50 || true ;; nfs|loop) - echo "=== dmesg (last 100) ===" - sudo dmesg | tail -100 || true + # Filtered rather than `dmesg | tail -100`: on a hosted runner the + # last 100 kernel lines are almost entirely boot spam (hyperv, pci, + # apparmor), which buries the failure in the job log. Warnings and + # errors plus anything naming the filesystem under test is what + # actually matters here. + echo "=== dmesg (warnings and errors, last 40) ===" + sudo dmesg --level=emerg,alert,crit,err,warn 2>/dev/null | tail -40 || true + echo "=== dmesg (mentioning $BACKEND/$VERSION, last 30) ===" + sudo dmesg 2>/dev/null \ + | grep -iE "loop|nfs|${VERSION:-nomatch}" \ + | tail -30 || true ;; *) echo "unknown backend: $BACKEND" >&2 diff --git a/bin/ci/matrix.sh b/bin/ci/matrix.sh index 592c8a2..f63c040 100644 --- a/bin/ci/matrix.sh +++ b/bin/ci/matrix.sh @@ -131,6 +131,22 @@ target_cron_minute() { esac } +# Does this target need to run as root to mean anything? +# +# pjdfstest is half privileged-vs-unprivileged assertions and refuses to +# run otherwise; stress-ng's chown/mknod stressors need CAP_CHOWN / +# CAP_MKNOD. The loop and beegfs backends already run the wrapped command +# as root, but the NFS backend deliberately drops back to the invoking +# user and exports with root_squash -- which is exactly right for +# git-annex/git and useless for these two. run-under.sh passes +# --no-root-squash for them. +target_needs_root() { + case "$1" in + pjdfstest|stress-ng) return 0 ;; + *) return 1 ;; + esac +} + # Does this target need the git-annex daily build installed on the runner? target_needs_git_annex() { [ "$1" = "git-annex" ] diff --git a/bin/ci/run-under.sh b/bin/ci/run-under.sh index b350f4d..8580891 100755 --- a/bin/ci/run-under.sh +++ b/bin/ci/run-under.sh @@ -54,7 +54,11 @@ case "$BACKEND" in beegfs) opts=(--version "$VERSION") ;; loop) opts=(--fs "$VERSION" --size "${EVAL_UNDER_LOOP_SIZE_MB:-$(target_loop_size_mb "$TARGET")}") ;; - nfs) opts=() ;; + nfs) opts=() + # See target_needs_root() in matrix.sh: root-requiring suites + # need an export that does not squash root, and need to keep + # their privileges rather than being dropped to the invoker. + target_needs_root "$TARGET" && opts=(--no-root-squash) ;; *) echo "unknown backend: $BACKEND" >&2; exit 1 ;; esac diff --git a/bin/eval-under-nfs b/bin/eval-under-nfs index 349d0ea..bee892d 100755 --- a/bin/eval-under-nfs +++ b/bin/eval-under-nfs @@ -23,6 +23,10 @@ SET_HOME="${EVAL_UNDER_HOME_ON_MOUNT:-0}" # back if the new name is unset. SYNC="${EVAL_UNDER_NFS_SYNC:-${DATALAD_TESTS_NFS_SYNC:-}}" +# Export with no_root_squash AND keep the wrapped command running as root, +# instead of the default (root_squash + drop back to the invoking user). +NO_ROOT_SQUASH="${EVAL_UNDER_NFS_NO_ROOT_SQUASH:-0}" + usage() { cat <<'EOF' Usage: eval-under-nfs [OPTIONS] -- CMD [ARGS...] @@ -53,6 +57,20 @@ Options (flag / env var / default / purpose): during clone/save/get, and `sync` pushed datalad CI jobs past 6h. Enable to reproduce that slowdown locally. + --no-root-squash EVAL_UNDER_NFS_NO_ROOT_SQUASH (unset) + Export with `no_root_squash` and run the wrapped command as root + rather than dropping back to the invoking user. + + The default is the opposite on purpose: `root_squash` plus an + unprivileged wrapped command is what a normal user's NFS home + looks like, and reproducing it is half the point of this backend. + But a suite that tests privileged behaviour (pjdfstest's + chown/mknod/sticky-bit assertions, stress-ng's chown/mknod + stressors) has nothing to measure when every privileged syscall is + squashed to nobody -- for those, this flag is what makes the run + meaningful. Also matches how HPC sites actually export to trusted + compute nodes. + -h, --help Print this help. @@ -78,6 +96,7 @@ while [ $# -gt 0 ]; do --mount-point) MNT="$2"; shift 2 ;; --set-home) SET_HOME=1; shift ;; --sync) SYNC=1; shift ;; + --no-root-squash) NO_ROOT_SQUASH=1; shift ;; -h|--help) usage; exit 0 ;; --) shift; break ;; *) echo "unknown arg: $1" >&2; usage >&2; exit 2 ;; @@ -92,6 +111,14 @@ else NFS_OPTS="rw,async" fi +# Export options and mount options are different namespaces: `sync`/`async` +# are valid in both, but `no_root_squash` is export-only and mount(8) would +# reject it. Keep them separate. +EXPORT_OPTS="$NFS_OPTS" +if [ "$NO_ROOT_SQUASH" != 0 ]; then + EXPORT_OPTS="$EXPORT_OPTS,no_root_squash" +fi + if [ "$(id -u)" -ne 0 ]; then command -v sudo >/dev/null || { echo "must run as root (or have sudo available)" >&2; exit 2; } @@ -131,8 +158,8 @@ start_export() { # under root_squash a later chown as root would be denied. "${SUDO[@]}" chown "$INVOKER_UID:$INVOKER_GID" "$ORIG" - log "exportfs -o $NFS_OPTS localhost:$ORIG" - "${SUDO[@]}" exportfs -o "$NFS_OPTS" "localhost:$ORIG" + log "exportfs -o $EXPORT_OPTS localhost:$ORIG" + "${SUDO[@]}" exportfs -o "$EXPORT_OPTS" "localhost:$ORIG" } mount_nfs() { @@ -168,7 +195,8 @@ if [ "$SET_HOME" = 1 ]; then # lands as nobody:nogroup and the wrapped command can't cd into it. # (We could be running as effective-root either via `sudo