Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions .github/workflows/install.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# Installing this client on a machine that has nothing on it.
#
# Every other job in this repository runs against a checkout: the engine
# is built beside it, an environment variable points at the library, and
# the tests are told where everything is so that they test the binding
# rather than the search. That is the right thing for a test suite and
# it is the wrong thing entirely for the question a new user asks, which
# is whether three lines in a pom and a JDK are enough.
#
# So this job answers that question somewhere else. A container with a
# JDK and Maven in it and nothing else, no toolchain, no engine, no
# variable, and the program off the front page built against the
# artifacts this commit would publish. It runs nightly rather than on
# every push, because what breaks it is rarely a commit here: it is a
# base image that changed, a JDK that tightened a default, a
# dependency's dependency that moved.
#
# Both providers get a row, because the claim this client makes that no
# other client of this engine makes is that it installs on a JDK from
# 17 and on a JDK from 25, over two different bindings, out of the same
# dependency. A row each is what keeps that from being a paragraph.
name: Install

on:
schedule:
# Nightly, an hour that is nobody's working day and not on the hour,
# where a queue forms.
- cron: "41 4 * * *"
workflow_dispatch:
pull_request:
paths:
- .github/workflows/install.yml
- scripts/install.sh
- README.md
- "**/pom.xml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
MAVEN_ARGS: -B -ntp

jobs:
clean-machine:
strategy:
fail-fast: false
matrix:
include:
- provider: zudb-ffm
image: maven:3-eclipse-temurin-25
java: "25"
- provider: zudb-jni
image: maven:3-eclipse-temurin-17
java: "17"
name: ${{ matrix.provider }} on JDK ${{ matrix.java }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5

- uses: actions/checkout@v5
with:
repository: tamnd/zu
path: engine

# This JDK is the one that builds the artifacts. The one that
# installs them is inside the container and is a different JDK,
# which is the point of the row.
- uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "25"
cache: maven

- uses: Swatinem/rust-cache@v2
with:
workspaces: engine

- name: Build libzu
working-directory: engine
run: cargo build --release -p zu-capi

# One platform, this runner's, because a runner builds the one it
# is. The release is where all seven have to be there, and the
# enforcer that says so is stood down here for the same reason it
# is in the natives job.
- name: Stage the one platform this runner is
run: |
set -eu
mkdir -p zudb-native/lib/linux-amd64
cp engine/target/release/libzu.so zudb-native/lib/linux-amd64/libzu.so

# The JNI row needs a shim, and a shim needs a C compiler, which is
# exactly the tool the container is asserted not to have. That is
# not a contradiction: the compiler is ours to have at build time
# and the user's not to need at install time, and the shim ends up
# in the jar.
- name: Build the JNI shim
run: ./scripts/build-shim.sh

# A file repository cannot serve a snapshot without the metadata a
# deploy writes, and a version with SNAPSHOT in it is not what a
# reader of the page would type anyway. So the artifacts are
# staged under the version this pom is on its way to.
- name: Which version this would be
run: |
set -eu
snapshot="$(mvn $MAVEN_ARGS help:evaluate -Dexpression=project.version -q -DforceStdout)"
echo "VERSION=${snapshot%-SNAPSHOT}" >> "$GITHUB_ENV"

- name: Stage the artifacts
run: |
set -eu
mvn $MAVEN_ARGS versions:set -DnewVersion="$VERSION" -DgenerateBackupPoms=false
# A cached local repository can hold this version from a run
# before this one, and an install that quietly resolved that
# is an install nobody tested.
rm -rf "$HOME/.m2/repository/dev/zudb"
mvn $MAVEN_ARGS -Pnatives -Denforcer.skip=true -DskipTests \
-pl zudb,zudb-ffm,zudb-jni,zudb-native -am clean install
mkdir -p "$RUNNER_TEMP/repo"
cp -r "$HOME/.m2/repository/dev" "$RUNNER_TEMP/repo/"
# Notes the local repository keeps about where it downloaded
# something from. A repository being served has no use for
# them and they are one more way for a resolve to go
# sideways.
find "$RUNNER_TEMP/repo" -name _remote.repositories -delete
# A repository serves a checksum beside every file, and a
# resolver that finds none says so at length. Central has
# them, so this has them, or else the install being tested is
# noisier than the one a user gets.
find "$RUNNER_TEMP/repo" -type f \
-exec sh -c 'sha1sum "$1" | cut -d" " -f1 > "$1.sha1"' _ {} \;
find "$RUNNER_TEMP/repo" -name '*.jar' -o -name '*.pom' | sort

- name: Install it the way a reader of the page would
run: |
set -eu
mkdir -p "$RUNNER_TEMP/app"
docker run --rm \
-v "$PWD:/src:ro" \
-v "$RUNNER_TEMP/repo:/repo:ro" \
-v "$RUNNER_TEMP/app:/app" \
-e SRC=/src -e REPO=/repo -e APP=/app \
-e VERSION="$VERSION" \
-e PROVIDER=${{ matrix.provider }} \
-e ABSENT="rustc cargo cc gcc g++ make cmake ninja" \
${{ matrix.image }} \
bash /src/scripts/install.sh
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,8 @@ The library inside the jar is a resource, and no loader on any platform can map

A GraalVM native image needs no configuration for any of this. Both artifacts carry their own reachability metadata: `zudb-ffm` lists every signature it binds, because an image has no linker in it and each downcall stub is machine code the builder has to be told to write, and `zudb-native` registers the libraries so that one ends up inside the image rather than being looked for on a machine that does not have it. Use a classifier rather than the platform-complete jar, or the image carries seven libraries and uses one. CI builds an image on Linux and macOS every run and makes it answer a query, because a metadata file that is wrong produces an image that builds clean and dies on the first call.

None of that is taken on trust. A nightly job installs this client the way this page says to, in a container holding a JDK and Maven and nothing else, and runs the program at the top of the page against the artifacts the commit would publish. Before it starts it asserts the machine has no libzu on it, no header, no pkg-config file, no `ZU_LIBRARY`, and no compiler of any kind, because an install that works for a reason the reader does not have is the one failure a test on our own machine cannot see. Then it takes the library artifact away again and checks that what comes back names the line to add, since a job that has only ever passed is a job nobody has watched fail.

On the module path the artifact needs `--add-modules dev.zudb.natives`. Nothing `requires` it, since there is no code in it to require, and a jar nothing requires is a jar that is never resolved and whose resources are therefore invisible. The search says so itself when it comes up empty on a module path, so the failure names the flag rather than leaving a user to work out why the same classpath run worked.

## How it binds
Expand All @@ -296,7 +298,18 @@ An SDK that requires a recent JDK in 2026 excludes a large part of the enterpris

A `ServiceLoader` picks the provider at run time and application code never names one, and the same suite of cases runs against both every build, so a difference between the two is a red job rather than something you find. Add `zudb-jni` beside `zudb` on 17 through 21, add `zudb-ffm` on 25 and later, or add both and let the loader pick. The FFM artifact targets Java 25 rather than the Java 22 that finalised the API, because 22 has been out of support since September 2024 and shipping against an unsupported release only moves the problem. CI runs 17, 21, 25, and 26.

One thing to know before your first run: from JDK 24, native access must be granted explicitly. The jars carry `Enable-Native-Access: ALL-UNNAMED` for the class path case, the module path case wants `--enable-native-access=dev.zudb.ffm` or `--enable-native-access=dev.zudb.jni` for whichever provider is in play, and the FFM provider checks `Module::isNativeAccessEnabled` before the first downcall so that the failure is an exception naming the flag rather than a JVM warning on stderr three frames from any of our code.
On 17 through 21, the second dependency at the top of this page is the only line that changes:

```xml
<dependency>
<groupId>dev.zudb</groupId>
<artifactId>zudb-jni</artifactId>
<version>${zu.version}</version>
<scope>runtime</scope>
</dependency>
```

One thing to know before your first run: from JDK 24, native access is granted by whoever starts the JVM rather than by the library being called. On a class path that means `--enable-native-access=ALL-UNNAMED` on the command line, or the same thing as an `Enable-Native-Access` line in the manifest of the jar that `java -jar` names. On a module path it names the provider instead, `--enable-native-access=dev.zudb.ffm` or `--enable-native-access=dev.zudb.jni` for whichever one is in play. The FFM provider checks `Module::isNativeAccessEnabled` before the first downcall, so a run without the grant is an exception naming the flag rather than a JVM warning on stderr three frames from any of our code. The JNI provider is not restricted this way on a class path, which is one more thing the 17 artifact is quieter about.

## Errors

Expand Down Expand Up @@ -385,7 +398,7 @@ Inside this repository:
| JMH benchmarks | `zudb-bench` |
| The staged libraries, built by the release rather than by a clone | `zudb-native` |
| Every published name and the shape it is published in | `api/surface.txt` |
| Building the shim, staging the libraries | `scripts` |
| Building the shim, staging the libraries, installing on a clean machine | `scripts` |

`api/surface.txt` is generated, one line per exported type and per member a caller outside the module can name, in the spirit of the `api/go1.N.txt` files Go holds itself to. `SurfaceTest` regenerates it and compares, so a change to the API is a change to that file in the same commit, where it is the first thing in the diff rather than something a user finds after the release. Write it down with `mvn -pl zudb test -Dzu.surface.write=true` and review it like any other file: a name that arrived is a minor release, a name that went or changed shape is a major one or a mistake, and the gate says which of the three a diff is while it is still a diff. It reads the compiled classes of the API module and links nothing, so it answers on a clone with no library staged and no Rust installed.

Expand Down
Loading
Loading