Install what a release publishes, on a machine that has nothing on it - #20
Merged
Conversation
Every job in this repository runs on a hosted image, and a hosted image is the least representative computer in the world. It has a compiler, a Rust toolchain, a git, a Python somebody chose deliberately, and a few hundred libraries a wheel can quietly link against and get away with. So the failures that only a user's machine sees are exactly the ones nothing here looks for. A file the build put in the tree and left out of the wheel, read out of the checkout instead and never missed. A stub or a py.typed that the sdist ships and the wheel does not. An extension linked against a symbol version the manylinux image had and a slim image has not. A dependency that arrived because something else in the job had already pulled it in. So: a nightly that builds the two Linux wheels a release would build, inside the pypa images and against the oldest interpreter the stable ABI covers, and then installs each of them in a container that holds an interpreter, a package manager and nothing else. Three rows, which are the floor this package supports, the newest release, and musl, where a wheel tagged wrongly installs anyway and then fails to import. The index is off, so nothing can arrive to cover for a mistake, and pip has to find everything in the one directory the wheel is in. What runs inside is tools/smoke.py. Standard library only, no pytest, no fixtures, no checkout: it asserts that the package it imported came out of site-packages rather than out of a tree, that the compiled half and the stub and the py.typed are all beside it, and then it does what a reader of the README would do on their first afternoon. A graph on a file, reopened afterwards so that a wheel which only ever wrote to a page cache is caught. Both bulk paths. A statement that cannot parse, which has to come back with the standard's own code on it. The DB-API front door and the asyncio one, since both are submodules and a wheel that shipped one of them and not the other passes everything else. Last, the check only a machine like that one can make: that importing zudb brought no dataframe library with it and that asking for a DataFrame without pandas is a refusal naming the extra to install. The images are checked rather than believed. Every row asserts that cc, gcc, clang, rustc, cargo, make and git are all absent, because the day a base image starts shipping a compiler is the day this job silently stops being about anything. The gate is validated the way the other gates here are: the failure it exists to catch has to fail it. The same program in the same image with nothing installed, which is what a wheel that did not build, did not upload or did not install looks like from inside the container, and which is also the only way to know the job is running the program at all. A bind mount pointing at nothing and a container whose exit code went nowhere both look exactly like success. tools/smoke.py also runs in the suite, piece by piece, because a container is the least convenient place to discover that a line of it went stale. What the suite cannot run is the no-dependencies check, which needs a machine with no pandas on it, and the came-from-an-install check when the working copy is a maturin develop rather than a wheel. Both say so. Nightly rather than on every push. What this catches is drift in things outside this repository, a base image whose glibc moved, a manylinux policy that went forward, a pip that changed how it reads a tag, and none of that arrives in a diff anybody here writes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every job in this repository runs on a hosted image, and a hosted image is the least representative computer in the world. It has a compiler, a Rust toolchain, a git, a Python somebody chose deliberately, and a few hundred libraries a wheel can quietly link against and get away with. So the failures that only a user's machine sees are exactly the ones nothing here looks for. A file the build put in the tree and left out of the wheel, read out of the checkout instead and never missed. A stub or a py.typed that the sdist ships and the wheel does not. An extension linked against a symbol version the manylinux image had and a slim image has not. A dependency that arrived because something else in the job had already pulled it in.
This is the DX3 line that asks for a clean-machine install test per SDK, nightly, in a container with only the language runtime.
What it does
A nightly that builds the two Linux wheels a release would build, inside the pypa images and against the oldest interpreter the stable ABI covers, and then installs each of them in a container that holds an interpreter, a package manager and nothing else.
python:3.11-slimpython:3.14-slimpython:3.14-alpineThe floor this package supports, the newest release, and musl, which is where a wheel tagged wrongly installs anyway and then fails to import. The index is off, so nothing can arrive to cover for a mistake, and pip has to find everything in the one directory the wheel is in.
docker runrather than a job container, because a job container has the runner's own Node mounted into it and half the reason to use a slim image is that nothing is mounted into it.What runs inside
tools/smoke.py. Standard library only, no pytest, no fixtures, no checkout. It asserts that the package it imported came out of site-packages rather than out of a tree, that the compiled half and the stub and the py.typed are all beside it, and then it does what a reader of the README would do on their first afternoon:loadand the appenderzudbbrought no dataframe library with it, and that asking for a DataFrame without pandas is a refusal naming the extra to installThe image is checked rather than believed
Every row asserts that
cc,gcc,clang,rustc,cargo,makeandgitare all absent. The day a base image starts shipping a compiler is the day this job silently stops being about anything, and nothing else would notice.The gate is validated
The same program in the same image with nothing installed has to fail. That is what a wheel which did not build, did not upload or did not install looks like from inside the container, and it is also the only way to know the job is running the program at all: a bind mount pointing at nothing and a container whose exit code went nowhere both look exactly like success.
Verified locally against colima that none of the three images carries any of the seven tools, and that the negative case exits 1 rather than passing.
tools/smoke.pyitself was run against a real wheel installed into an empty virtualenv, which is how the two assertions it originally got wrong were found.And in the suite
tools/smoke.pyruns piece by piece fromtests/test_smoke.pyas well, because a container is the least convenient place in the world to discover that a line of it went stale. What the suite cannot run is the no-dependencies check, which needs a machine with no pandas on it, and the came-from-an-install check when the working copy is amaturin developrather than a wheel. Both say so in a skip reason.Why nightly
What this catches is drift in things outside this repository. A base image whose glibc moved, a manylinux policy that went forward, a pip that changed how it reads a tag. None of that arrives in a diff anybody here writes. It runs on the pull request that changes the workflow or the program too, since a nightly that broke is a nightly nobody reads for a week.