Skip to content

Install what a release publishes, on a machine that has nothing on it - #20

Merged
tamnd merged 1 commit into
mainfrom
clean-install
Aug 19, 2026
Merged

Install what a release publishes, on a machine that has nothing on it#20
tamnd merged 1 commit into
mainfrom
clean-install

Conversation

@tamnd

@tamnd tamnd commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Every job in this repository runs on a hosted image, and a hosted image is the least representative computer in the world. It has a compiler, a Rust toolchain, a git, a Python somebody chose deliberately, and a few hundred libraries a wheel can quietly link against and get away with. So the failures that only a user's machine sees are exactly the ones nothing here looks for. A file the build put in the tree and left out of the wheel, read out of the checkout instead and never missed. A stub or a py.typed that the sdist ships and the wheel does not. An extension linked against a symbol version the manylinux image had and a slim image has not. A dependency that arrived because something else in the job had already pulled it in.

This is the DX3 line that asks for a clean-machine install test per SDK, nightly, in a container with only the language runtime.

What it does

A nightly that builds the two Linux wheels a release would build, inside the pypa images and against the oldest interpreter the stable ABI covers, and then installs each of them in a container that holds an interpreter, a package manager and nothing else.

image wheel
python:3.11-slim manylinux_2_28_x86_64
python:3.14-slim manylinux_2_28_x86_64
python:3.14-alpine musllinux_1_2_x86_64

The floor this package supports, the newest release, and musl, which is where a wheel tagged wrongly installs anyway and then fails to import. The index is off, so nothing can arrive to cover for a mistake, and pip has to find everything in the one directory the wheel is in.

docker run rather than a job container, because a job container has the runner's own Node mounted into it and half the reason to use a slim image is that nothing is mounted into it.

What runs inside

tools/smoke.py. Standard library only, no pytest, no fixtures, no checkout. It asserts that the package it imported came out of site-packages rather than out of a tree, that the compiled half and the stub and the py.typed are all beside it, and then it does what a reader of the README would do on their first afternoon:

  • a graph on a file, reopened afterwards, so that a wheel which only ever wrote to a page cache is caught
  • both bulk paths, load and the appender
  • a statement that cannot parse, which has to come back with the standard's own code on it
  • the DB-API front door and the asyncio one, since both are submodules and a wheel that shipped one of them and not the other passes everything else
  • and last, the check only a machine like that one can make: that importing zudb brought no dataframe library with it, and that asking for a DataFrame without pandas is a refusal naming the extra to install

The image is checked rather than believed

Every row asserts that cc, gcc, clang, rustc, cargo, make and git are all absent. The day a base image starts shipping a compiler is the day this job silently stops being about anything, and nothing else would notice.

The gate is validated

The same program in the same image with nothing installed has to fail. That is what a wheel which did not build, did not upload or did not install looks like from inside the container, and it is also the only way to know the job is running the program at all: a bind mount pointing at nothing and a container whose exit code went nowhere both look exactly like success.

Verified locally against colima that none of the three images carries any of the seven tools, and that the negative case exits 1 rather than passing. tools/smoke.py itself was run against a real wheel installed into an empty virtualenv, which is how the two assertions it originally got wrong were found.

And in the suite

tools/smoke.py runs piece by piece from tests/test_smoke.py as well, because a container is the least convenient place in the world to discover that a line of it went stale. What the suite cannot run is the no-dependencies check, which needs a machine with no pandas on it, and the came-from-an-install check when the working copy is a maturin develop rather than a wheel. Both say so in a skip reason.

Why nightly

What this catches is drift in things outside this repository. A base image whose glibc moved, a manylinux policy that went forward, a pip that changed how it reads a tag. None of that arrives in a diff anybody here writes. It runs on the pull request that changes the workflow or the program too, since a nightly that broke is a nightly nobody reads for a week.

Every job in this repository runs on a hosted image, and a hosted image
is the least representative computer in the world. It has a compiler, a
Rust toolchain, a git, a Python somebody chose deliberately, and a few
hundred libraries a wheel can quietly link against and get away with. So
the failures that only a user's machine sees are exactly the ones
nothing here looks for. A file the build put in the tree and left out of
the wheel, read out of the checkout instead and never missed. A stub or
a py.typed that the sdist ships and the wheel does not. An extension
linked against a symbol version the manylinux image had and a slim image
has not. A dependency that arrived because something else in the job had
already pulled it in.

So: a nightly that builds the two Linux wheels a release would build,
inside the pypa images and against the oldest interpreter the stable ABI
covers, and then installs each of them in a container that holds an
interpreter, a package manager and nothing else. Three rows, which are
the floor this package supports, the newest release, and musl, where a
wheel tagged wrongly installs anyway and then fails to import. The
index is off, so nothing can arrive to cover for a mistake, and pip has
to find everything in the one directory the wheel is in.

What runs inside is tools/smoke.py. Standard library only, no pytest, no
fixtures, no checkout: it asserts that the package it imported came out
of site-packages rather than out of a tree, that the compiled half and
the stub and the py.typed are all beside it, and then it does what a
reader of the README would do on their first afternoon. A graph on a
file, reopened afterwards so that a wheel which only ever wrote to a
page cache is caught. Both bulk paths. A statement that cannot parse,
which has to come back with the standard's own code on it. The DB-API
front door and the asyncio one, since both are submodules and a wheel
that shipped one of them and not the other passes everything else. Last,
the check only a machine like that one can make: that importing zudb
brought no dataframe library with it and that asking for a DataFrame
without pandas is a refusal naming the extra to install.

The images are checked rather than believed. Every row asserts that cc,
gcc, clang, rustc, cargo, make and git are all absent, because the day a
base image starts shipping a compiler is the day this job silently stops
being about anything.

The gate is validated the way the other gates here are: the failure it
exists to catch has to fail it. The same program in the same image with
nothing installed, which is what a wheel that did not build, did not
upload or did not install looks like from inside the container, and
which is also the only way to know the job is running the program at
all. A bind mount pointing at nothing and a container whose exit code
went nowhere both look exactly like success.

tools/smoke.py also runs in the suite, piece by piece, because a
container is the least convenient place to discover that a line of it
went stale. What the suite cannot run is the no-dependencies check,
which needs a machine with no pandas on it, and the came-from-an-install
check when the working copy is a maturin develop rather than a wheel.
Both say so.

Nightly rather than on every push. What this catches is drift in things
outside this repository, a base image whose glibc moved, a manylinux
policy that went forward, a pip that changed how it reads a tag, and
none of that arrives in a diff anybody here writes.
@tamnd
tamnd merged commit c5723a0 into main Aug 19, 2026
11 of 13 checks passed
@tamnd
tamnd deleted the clean-install branch August 19, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant