The shared conformance corpus, read and run against this client

.github/workflows/ci.yml

@@ -127,6 +127,59 @@ jobs:
 
       - run: java -jar zudb-bench/target/benchmarks.jar -f 1 -wi 1 -i 1 -r 1s -w 1s
 
+  # The cross client corpus, run against the engine this job builds.
+  # Every client answers the same fourteen hundred cases and a report is
+  # diffed line for line against the other four, so this is the job that
+  # says this client agrees with them rather than only with itself.
+  #
+  # The cases come from the same checkout the library was built from,
+  # which is the pairing that makes the report mean anything. A corpus
+  # ahead of the library reports the engine catching up to its own cases
+  # as this client failing, and a library ahead of the corpus reports
+  # nothing at all. This client builds the engine rather than shipping an
+  # archive of it, so both are the same checkout and there is no revision
+  # to pin.
+  #
+  # A job of its own rather than a step in the one above, because the run
+  # is fourteen hundred databases and the job above runs its suite three
+  # times.
+  corpus:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+
+      - uses: actions/checkout@v5
+        with:
+          repository: tamnd/zu
+          path: engine
+
+      - uses: actions/setup-java@v5
+        with:
+          distribution: temurin
+          java-version: "25"
+          cache: maven
+
+      - uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: engine
+
+      - name: Build libzu
+        working-directory: engine
+        run: cargo build --release -p zu-capi
+
+      - name: Where the library landed
+        run: |
+          set -eu
+          lib="$(ls engine/target/release/libzu.so)"
+          test -n "$lib"
+          echo "ZU_LIBRARY=$GITHUB_WORKSPACE/$lib" >> "$GITHUB_ENV"
+
+      # Verbose, because the run logs every case the engine has not
+      # caught up to and a release branch is read for exactly that.
+      - run: mvn $MAVEN_ARGS -pl zudb-corpus -am test -Dsurefire.useFile=false
+        env:
+          ZU_CASES: ${{ github.workspace }}/engine/conformance/cases
+
   # The JNI provider on the JDKs it exists for. Panama is not there on
   # 17 or 21, so on those two this is the only way to call the engine at
   # all, and a client that claims 17 and is only ever tested on 25 is a

README.md

@@ -360,6 +360,17 @@ ZU_LIBRARY=/path/to/libzu.dylib java -jar zudb-bench/target/benchmarks.jar
 
 `ZU_LIBRARY` rather than `-Dzu.library` there, because JMH forks a JVM of its own and a fork inherits the environment rather than the system properties.
 
+The cross client corpus is a directory of cases in the engine's repository, versioned with it and answered by every client in the family. It is a command as well as a test:
+
+```sh
+mvn -pl zudb-corpus -am package -DskipTests
+ZU_LIBRARY=/path/to/libzu.dylib java --enable-native-access=ALL-UNNAMED \
+    -cp "zudb/target/classes:zudb-ffm/target/classes:zudb-corpus/target/classes" \
+    dev.zudb.corpus.Main /path/to/zu/conformance/cases
+```
+
+It prints a line per case that did not pass and then a summary, and the lines are the reference runner's word for word so that two clients disagreeing is a diff. `-strict` makes a case the engine has not caught up to fail the run, which is what a release branch wants, and `-work` keeps the databases rather than removing them. The same run happens under `mvn test` when `ZU_CASES` points at the cases, and skips when it does not, so a checkout of this repository alone is still green.
+
 The leak run is a script rather than a test, because what reads the result is the allocator rather than an assertion:
 
 ```sh
@@ -404,6 +415,7 @@ Inside this repository:
 | The Panama provider | `zudb-ffm` |
 | The JNI provider, and the C shim it calls through | `zudb-jni` |
 | The cases every provider owes, run by both of them | `zudb-tck` |
+| The cross client corpus, read and run against this client | `zudb-corpus` |
 | Arrow, over the C Data Interface | `zudb-arrow` |
 | JMH benchmarks | `zudb-bench` |
 | The staged libraries, built by the release rather than by a clone | `zudb-native` |

pom.xml

@@ -51,6 +51,7 @@
     <module>zudb</module>
     <module>zudb-tck</module>
     <module>zudb-ffm</module>
+    <module>zudb-corpus</module>
     <module>zudb-jni</module>
     <module>zudb-arrow</module>
     <module>zudb-bench</module>

zudb-corpus/pom.xml

@@ -0,0 +1,84 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  The shared conformance corpus, run through this client.
+
+  The corpus is a directory of YAML files versioned with the engine and
+  shipped to every client in the family. A case is a statement and what
+  running it must produce, so it is a corpus every client can run and no
+  client has to have written.
+
+  These are main sources for the same reason zudb-tck's are: a test
+  source set is not something another module can depend on without a
+  test jar, and this one is both a command a person runs against a
+  corpus directory and a fixture a test can call. Nothing here is
+  published.
+
+  It compiles to 25 rather than to the 17 the API targets, because
+  reading a result as Arrow through the C Data Interface is what the
+  Arrow half of the corpus asks for, and doing that without a dependency
+  on arrow-java wants java.lang.foreign.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <parent>
+    <groupId>dev.zudb</groupId>
+    <artifactId>zudb-parent</artifactId>
+    <version>0.11.0-SNAPSHOT</version>
+  </parent>
+
+  <artifactId>zudb-corpus</artifactId>
+  <name>zu for the JVM: the shared corpus</name>
+  <description>The conformance corpus the engine and every other client run, read and run here.</description>
+
+  <dependencies>
+    <dependency>
+      <groupId>dev.zudb</groupId>
+      <artifactId>zudb</artifactId>
+    </dependency>
+    <!-- A provider at test scope and not at compile scope, because this
+      module is the API and the corpus and neither of them picks a
+      provider. Somebody running the command puts one on the path
+      alongside a library, the same choice they make for their own
+      program. The tests need one to run at all, and the FFM provider is
+      the one that fits: this module already compiles to 25 for the sake
+      of reading a result through the C Data Interface. -->
+    <dependency>
+      <groupId>dev.zudb</groupId>
+      <artifactId>zudb-ffm</artifactId>
+      <version>${project.version}</version>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+
+  <build>
+    <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-compiler-plugin</artifactId>
+        <configuration>
+          <release>${zu.release.ffm}</release>
+          <compilerArgs>
+            <arg>-Xlint:all,-requires-automatic,-requires-transitive-automatic</arg>
+            <arg>-Werror</arg>
+          </compilerArgs>
+        </configuration>
+      </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-surefire-plugin</artifactId>
+        <configuration>
+          <!-- On the class path and not the module path, because this is
+               a command a person runs and the way it is run is a class
+               path with a provider and a library on it. The grant names
+               the unnamed module for the same reason, and it is the
+               grant the README tells that person to pass. -->
+          <useModulePath>false</useModulePath>
+          <argLine>--enable-native-access=ALL-UNNAMED ${zu.test.args}</argLine>
+        </configuration>
+      </plugin>
+    </plugins>
+  </build>
+</project>