Generate the API reference from the wheel that ships

A reference written by hand beside the code is wrong by the second
release, and wrong in the way that costs the most: it looks maintained.
So this one is generated, by pdoc, from the installed package, and the
release builds it from the wheel it is about to publish rather than from
the checkout the wheel came out of.

Half of this package is compiled and that is the one thing the
inspection cannot see through. `zudb.connect` is a function object pyo3
built, so what pdoc reads off it is `(path, *, read_only=False, ...)`
and the types are in `_zudb.pyi`, where the checkers read them. pdoc
looks for a stub named after the module it is documenting, which is
`zudb`, and the stub is named after the module the names were defined
in, which is `zudb._zudb`. So `tools/reference.py` lays the stub where
pdoc looks, under the `-stubs` name PEP 561 reserves for it, deriving it
rather than writing it: there is no second declaration of the surface to
keep in step, and the only edit is the one relative import, which a stub
loaded outside the import machinery has no package to resolve.

The check on that is the part worth having. Without the overlay ten
public signatures lose every annotation, and the reference still builds,
still looks finished, and is wrong about every one of them. Nobody reads
their own generated docs closely enough to catch it, so the build fails
instead, naming the signatures it would have published bare.

`tags` now downloads the wheels by name instead of everything the run
produced, because the rule it applies is that the directory holds the
grid and nothing outside it, and the reference job puts a directory of
HTML in the same run.

Author: tamnd

.github/workflows/ci.yml

@@ -116,8 +116,10 @@ jobs:
       - run: pip install ".[all]"
       # griffe reads the stub and inspects the installed extension, so
       # the check runs against the wheel rather than against the
-      # checkout it was built from.
-      - run: pip install pytest griffe ipython
+      # checkout it was built from. pdoc is here for the same reason:
+      # the reference is generated from the installed package, and the
+      # test that says so has nothing to read without it.
+      - run: pip install pytest griffe ipython pdoc==16.0.0
       - run: pytest
 
   # The same suite again, over an extension built with AddressSanitizer.

.github/workflows/wheels.yml

@@ -14,6 +14,7 @@ on:
       - pyproject.toml
       - rust-toolchain.toml
       - tools/wheel_tags.py
+      - tools/reference.py
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
@@ -196,17 +197,62 @@ jobs:
           name: sdist
           path: dist/*.tar.gz
 
+  # The reference, built from the wheel this release is publishing and
+  # not from the checkout it came out of. Half of this package is
+  # compiled, so a reference generated from source would be a reference
+  # of the half a reader can read already, and the interesting half
+  # exists only once it is built.
+  #
+  # One wheel is enough and it is the stable-ABI Linux one, because
+  # what is being read is the public surface and every wheel in the
+  # grid has the same one. A grid of fourteen references would be
+  # fourteen copies of one page and thirteen chances for a reader to
+  # find the wrong one.
+  reference:
+    needs: wheel
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.14"
+      - uses: actions/download-artifact@v4
+        with:
+          name: wheel-abi3-manylinux_2_28_x86_64
+          path: dist
+      - run: pip install --no-index --find-links dist zudb
+      # ipython because `zudb.magic` imports it at the top and a module
+      # that cannot be imported cannot be documented, and it is the
+      # `%%gql` half of this client rather than an extra.
+      - run: pip install pdoc==16.0.0 ipython
+      - run: python tools/reference.py reference
+      - uses: actions/upload-artifact@v4
+        with:
+          name: reference
+          path: reference/
+          if-no-files-found: error
+
   # The grid, checked as a grid. Every cell filled and nothing outside
   # it, which is the check a single row cannot do.
   tags:
     needs: [wheel, sdist]
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
+      # By name rather than everything this run produced, because the
+      # check is that the directory holds the grid and nothing else,
+      # and the reference job puts a directory of HTML in the same run.
+      # A rule that says "nothing outside the grid" reads whatever it
+      # is handed, so what it is handed has to be the grid.
       - uses: actions/download-artifact@v4
         with:
+          pattern: wheel-*
           path: dist
           merge-multiple: true
+      - uses: actions/download-artifact@v4
+        with:
+          name: sdist
+          path: dist
       - uses: actions/setup-python@v6
         with:
           python-version: "3.14"

README.md

@@ -221,6 +221,12 @@ The wheel carries `py.typed` and a stub for the compiled module, so mypy, pyrigh
 
 The stub is checked against the module it describes in CI: griffe reads the stub as text and the installed extension by inspection, and the two have to agree on every name, every parameter and every default. A stub is a promise no interpreter checks, so something has to.
 
+## Reference
+
+Every name, its signature and what its docstring says, generated from the installed package rather than written by hand beside it. The release builds it from the wheel it is about to publish, so the pages describe that wheel and not the checkout it came out of, and `python tools/reference.py <directory>` builds the same pages here.
+
+Half of this package is compiled, which is the one thing an inspection cannot see through: `zudb.connect` is a function object pyo3 built, and its types live in `_zudb.pyi` where the checkers read them. So the stub is laid where the generator looks for one, and the build fails if it stops landing, because a reference that quietly lost every annotation is a reference that still builds and is wrong about every signature on the page.
+
 ## What works today
 
 The list above is what this client is for. What it does so far is the core of it: `connect`, `execute` and `sql` with named parameters, results that iterate and fetch, values as Python objects both ways including dates, times, datetimes and durations, `Node`, `Rel` and `Path` as classes, `load` for building a graph with edges in it, an appender for growing one, transactions as a context manager that commits at the end of a block and rolls back when it raises, every condition as an exception class carrying its code, its position and its documentation link, results as Arrow columns and as pandas and polars frames, `register` for putting a frame under a name a statement can match on and reading it where it lies, stubs inside the wheel with a gate that keeps them true, the GIL released around every statement, every load and every copy out, `Ctrl-C` and `interrupt()` stopping a statement without touching the connection under it, `zudb.aio` for the same calls awaited on an event loop, results, nodes, rels and paths that draw themselves in a notebook with `%gql` and `%%gql` to run statements in one, and `zudb.dbapi` for code written against PEP 249. Each one landed with the tests that say it works.

pyproject.toml

@@ -47,6 +47,9 @@ dev = [
   # Reads the stub as text and the built extension by inspection, which
   # is what checks one against the other.
   "griffe>=2",
+  # The reference generator. `tools/reference.py` drives it, and the
+  # release builds the pages from the wheel it is about to publish.
+  "pdoc>=16",
   # For `%gql` and `%%gql`, which are tested against a real shell
   # rather than against a stub of one. Not a dependency of the wheel:
   # anyone with a notebook has IPython already, and nobody else needs

tests/test_reference.py

@@ -0,0 +1,92 @@
+"""The generated reference, and the one thing about it worth asserting.
+
+Most of what a documentation generator does is not this suite's
+business: pdoc's templates are pdoc's, and a test that counted headings
+would fail on the week pdoc renders one differently. What is this
+client's business is that the reference says what the package says, and
+half of the package is compiled, so the interesting failure is the
+quiet one: pdoc reads a pyo3 function object, finds no annotations on
+it, and publishes a whole reference in which nothing has a type.
+
+That is a reference that builds, looks finished, and is wrong about
+every signature on the page. So the check is in the tool, and this is
+the check on the check: it fires when the stub overlay is missing and
+it passes when it is there.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+import reference
+
+pdoc = pytest.importorskip("pdoc", reason="the reference generator, a dev dependency")
+
+
+def test_without_the_overlay_the_signatures_have_no_types_on_them() -> None:
+    """The failure the tool exists to catch, uncaught."""
+    assert reference.bare("zudb"), "pdoc read types off a compiled module by itself"
+
+
+def test_with_the_overlay_every_public_signature_is_typed() -> None:
+    """And caught. The overlay is what the tool puts on sys.path."""
+    with reference.stubs():
+        assert reference.bare("zudb") == []
+
+
+def test_the_overlay_is_the_stub_the_type_checkers_read(tmp_path: Path) -> None:
+    """Derived from `_zudb.pyi` and not a second copy of it.
+
+    Every line but the one relative import, which a stub loaded outside
+    the import machinery has no package to resolve.
+    """
+    stub = (package() / "_zudb.pyi").read_text(encoding="utf-8").splitlines()
+    written = (
+        (reference.overlay(package(), tmp_path) / "__init__.pyi")
+        .read_text(encoding="utf-8")
+        .splitlines()
+    )
+    assert len(written) == len(stub)
+    moved = [(one, two) for one, two in zip(stub, written, strict=True) if one != two]
+    assert moved == [("from .types import Value", "from zudb.types import Value")]
+
+
+def test_the_reference_covers_what_a_reader_can_import(tmp_path: Path) -> None:
+    """One page per module, and the compiled module is not one.
+
+    `zudb._zudb` is where the classes are defined and it is not a name
+    anybody imports, so it is not in the list. Its names are documented
+    on the `zudb` page, which is where they are exported from.
+    """
+    import zudb
+
+    reference.build(tmp_path)
+    for name in reference.MODULES:
+        assert (tmp_path / (name.replace(".", "/") + ".html")).is_file(), name
+    assert "zudb._zudb" not in reference.MODULES
+    assert set(reference.MODULES) - {"zudb"} == {
+        f"zudb.{name}" for name in ("aio", "dbapi", "errors", "magic", "types")
+    }
+    # The version the wheel says it is, on the page the wheel produced.
+    assert zudb.__version__ in (tmp_path / "zudb.html").read_text(encoding="utf-8")
+
+
+def test_the_reference_replaces_what_was_there(tmp_path: Path) -> None:
+    """A page for a module that went away is worse than no page.
+
+    It is a name a reader can still find, still in the search index
+    built beside it, and gone from the package.
+    """
+    stale = tmp_path / "zudb" / "gone.html"
+    stale.parent.mkdir(parents=True)
+    stale.write_text("a module this package used to have", encoding="utf-8")
+    reference.build(tmp_path)
+    assert not stale.exists()
+
+
+def package() -> Path:
+    """Where the installed package is, which is what is documented."""
+    import zudb
+
+    return Path(zudb.__file__ or "").parent

tools/reference.py

@@ -0,0 +1,187 @@
+"""The API reference, generated from the package that ships.
+
+A reference written by hand beside the code is a reference that is
+wrong by the second release, and wrong in the way that costs the most:
+it looks maintained. So this one is built from the installed package,
+by inspection, and the release builds it from the same wheel it is
+about to publish.
+
+    python tools/reference.py docs/api
+
+pdoc is the generator because it reads a package the way the
+interpreter does and asks for nothing in return: no configuration file,
+no theme to keep, no second copy of the module list. What it produces
+is HTML that opens from disk, which matters for a client whose docs go
+out as an artifact of the release rather than as a deployment.
+
+Half of this package is compiled, and that is the one thing the
+inspection cannot see through. `zudb.connect` is a function object
+pyo3 built, so its signature is `(path, *, read_only=False, ...)` and
+the types are in `_zudb.pyi`, where the type checkers read them. pdoc
+knows about stub files and looks for one named after the module it is
+documenting, which is `zudb`, and the stub is named after the module
+the names were defined in, which is `zudb._zudb`. So the stub is put
+where pdoc looks, under the `-stubs` name PEP 561 reserves for exactly
+this, and the one relative import in it is rewritten because a stub
+loaded outside the import machinery has no package to be relative to.
+
+Derived rather than written, which is the point: there is no second
+declaration of the surface to keep in step, and `bare()` below fails
+the build if the overlay stops landing. Without that check a reference
+that quietly lost every type annotation would still be a reference that
+built, and nobody reads their own generated docs closely enough to
+notice.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import shutil
+import sys
+import tempfile
+from collections.abc import Iterator
+from pathlib import Path
+
+#: What the reference covers, which is what a reader can import. The
+#: compiled module is not in the list: it is an implementation detail
+#: with an underscore on it, and every name it defines is re-exported
+#: from `zudb` and documented there.
+MODULES = [
+    "zudb",
+    "zudb.aio",
+    "zudb.dbapi",
+    "zudb.errors",
+    "zudb.magic",
+    "zudb.types",
+]
+
+#: The name pdoc looks under, and PEP 561's name for a stub package
+#: distributed apart from the package it describes. `zudb/__init__.pyi`
+#: beside the real `__init__.py` would work as well and would be a
+#: second file in the wheel claiming to be the public surface, which is
+#: the thing this whole file exists to avoid.
+STUBS = "zudb-stubs"
+
+
+def overlay(package: Path, into: Path) -> Path:
+    """Write the stub pdoc will find for `zudb`, and answer where.
+
+    The content is `_zudb.pyi`, unchanged except for its one relative
+    import. pdoc loads a stub with a loader of its own rather than
+    through `import`, so the module has no package and `from .types
+    import Value` raises before anything is read; naming the package
+    absolutely is the whole edit.
+    """
+    stub = (package / "_zudb.pyi").read_text(encoding="utf-8")
+    written = into / STUBS
+    written.mkdir(parents=True, exist_ok=True)
+    (written / "__init__.pyi").write_text(
+        stub.replace("from .types import", "from zudb.types import"), encoding="utf-8"
+    )
+    return written
+
+
+@contextlib.contextmanager
+def stubs() -> Iterator[Path]:
+    """The overlay, on `sys.path`, for as long as the block runs.
+
+    pdoc remembers where a module's stub was and where it was not, and
+    the answer changes here twice in one process. So the caches that
+    hold it are dropped on the way in and on the way out, which is what
+    lets a test ask the question both ways and get both answers.
+    """
+    import pdoc.doc
+    import pdoc.doc_pyi
+    import zudb
+
+    def forget() -> None:
+        pdoc.doc_pyi.find_stub_file.cache_clear()
+        pdoc.doc.Module.from_name.cache_clear()
+
+    with tempfile.TemporaryDirectory() as scratch:
+        written = overlay(Path(zudb.__file__ or "").parent, Path(scratch))
+        sys.path.insert(0, str(written.parent))
+        forget()
+        try:
+            yield written
+        finally:
+            sys.path.remove(str(written.parent))
+            forget()
+
+
+def bare(module: str) -> list[str]:
+    """The names on `module` whose signature carries no annotation.
+
+    Which is the check that the overlay landed. Everything the compiled
+    module defines is annotated in the stub, so after the overlay every
+    parameter of every public function has a type on it, and a name
+    here is a name pdoc documented from the object instead. An empty
+    list is a reference worth publishing.
+
+    What a reader reads is what is counted: the members a namespace
+    declares itself, under names with no underscore on the front.
+    Inherited ones are somebody else's, and `object.__eq__` takes an
+    untyped `value` on every class in the language.
+    """
+    import pdoc.doc
+
+    def functions(namespace: pdoc.doc.Namespace) -> list[pdoc.doc.Function]:
+        found = []
+        for doc in namespace.own_members:
+            if doc.name.startswith("_"):
+                continue
+            if isinstance(doc, pdoc.doc.Function):
+                found.append(doc)
+            elif isinstance(doc, pdoc.doc.Class):
+                found.extend(functions(doc))
+        return found
+
+    untyped = []
+    for one in functions(pdoc.doc.Module.from_name(module)):
+        wanted = [
+            name
+            for name, parameter in one.signature.parameters.items()
+            if name not in ("self", "cls") and parameter.annotation is parameter.empty
+        ]
+        if wanted:
+            untyped.append(f"{one.fullname}({', '.join(wanted)})")
+    return untyped
+
+
+def build(into: Path) -> None:
+    """The reference, in `into`, replacing whatever was there.
+
+    Replacing rather than merging, because a page for a module that was
+    removed is worse than no page at all: it is a name a reader can
+    still find, still linked from the index of a search built beside
+    it, and gone from the package.
+    """
+    import pdoc
+
+    with stubs():
+        untyped = bare("zudb")
+        if untyped:
+            raise SystemExit(
+                "the stub overlay did not land, so the reference would publish "
+                f"{len(untyped)} name(s) with no types on them: {', '.join(sorted(untyped))}"
+            )
+        shutil.rmtree(into, ignore_errors=True)
+        pdoc.pdoc(*MODULES, output_directory=into)
+
+
+def main(argv: list[str]) -> int:
+    if len(argv) != 2:
+        print(f"usage: {argv[0]} <directory>", file=sys.stderr)
+        return 2
+    into = Path(argv[1])
+    build(into)
+    pages = sorted(path for path in into.rglob("*.html"))
+    missing = [name for name in MODULES if not (into / (name.replace(".", "/") + ".html")).exists()]
+    for name in missing:
+        print(f"{name} was documented and has no page", file=sys.stderr)
+    print(f"{len(pages)} pages in {into}, {len(MODULES)} modules")
+    return 1 if missing else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))