Skip to content

Commit e3d9a27

Browse files
committed
docs: make API reference rendering independent of page order
Zensical discovers pages with an unsorted directory walk and renders them all through one shared mkdocstrings handler, so the griffe collection that resolves cross-package re-exports (mcp -> mcp_types) accumulates in filesystem-dependent order. A GitHub runner-image update reshuffled readdir order and every CI docs build started failing with AliasResolutionError. The per-page preload_modules hint on the mcp index page could not help: mkdocstrings-python silently ignores it once the page's root package is already collected, which any earlier `::: mcp.<submodule>` page guarantees. Chase exported aliases at load time instead (load_external_modules: true), drop the dead per-page preload mechanism, and add a gauntlet step that renders the API pages in worst-case orders so an order-dependence regression fails on every machine instead of only on unlucky filesystems.
1 parent e464f72 commit e3d9a27

6 files changed

Lines changed: 146 additions & 35 deletions

File tree

.github/workflows/shared.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -143,8 +143,8 @@ jobs:
143143
# nav entries without a page and pages without a nav entry, `zensical build
144144
# --strict` fails on broken .md links, `pymdownx.snippets: check_paths:
145145
# true` fails on a deleted `docs_src/` include, and the post-build steps
146-
# fail on unresolved cross-references, inventory download failures, and
147-
# broken non-markdown link targets.
146+
# fail on order-dependent API rendering, unresolved cross-references,
147+
# inventory download failures, and broken non-markdown link targets.
148148
# Until this job existed the docs were only ever built post-merge by
149149
# `deploy-docs.yml`, so those failures went green on the PR and broke the next
150150
# deploy of main. This is the check path; `deploy-docs.yml` stays the deploy

mkdocs.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,13 @@ plugins:
188188
handlers:
189189
python:
190190
paths: [src, src/mcp-types]
191+
# Zensical renders pages in undefined (filesystem-dependent) order
192+
# against one shared griffe collection, so a cross-package re-export
193+
# (`mcp` -> `mcp_types`) resolves only if its target package happens
194+
# to have been collected first. Chasing exported aliases into their
195+
# packages at load time makes resolution order-independent;
196+
# scripts/docs/check_render_order.py enforces that property.
197+
load_external_modules: true
191198
options:
192199
relative_crossrefs: true
193200
members_order: source

scripts/docs/build.sh

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,9 @@
44
#
55
# Zensical runs no MkDocs plugins or hooks, so the build is three steps:
66
# materialise the API reference pages and the concrete config, build the
7-
# site strictly, then generate llms.txt and the per-page markdown
8-
# renditions. This script is the single owner of that recipe, dependency
7+
# site strictly (plus the order-independence and cross-reference checks
8+
# Zensical doesn't do itself), then generate llms.txt and the per-page
9+
# markdown renditions. This script is the single owner of that recipe, dependency
910
# sync included — CI (shared.yml, docs-preview.yml) and scripts/build-docs.sh
1011
# all call it. The toolchain detection in docs-preview.yml and build-docs.sh
1112
# keys on this file's path and expects the site under site/.
@@ -31,6 +32,11 @@ rm -rf .cache site
3132
uv run --frozen --no-sync python scripts/docs/build_config.py
3233
uv run --frozen --no-sync zensical build -f mkdocs.gen.yml --strict
3334

35+
# The build above renders pages in one arbitrary (filesystem-dependent)
36+
# order; prove the API reference renders in hostile orders too — see the
37+
# check's docstring for the failure mode this guards.
38+
uv run --frozen --no-sync python scripts/docs/check_render_order.py
39+
3440
# Zensical stays green even under --strict when a cross-reference fails to
3541
# resolve (rendered as literal bracket text) or an objects.inv inventory
3642
# fails to download (every link through it silently degrades to plain text);

scripts/docs/check_render_order.py

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
"""Fail the docs build when API rendering depends on page processing order.
2+
3+
Zensical discovers pages with an unsorted directory walk and renders them
4+
all through one shared mkdocstrings handler, so the griffe collection that
5+
resolves `::: module` blocks accumulates in filesystem-dependent order, and
6+
nothing in the toolchain checks that the order is safe: a cross-package
7+
re-export that only resolved when its target package had been collected
8+
earlier built fine on one machine and died with `AliasResolutionError` on
9+
another (a GitHub runner-image update reshuffled readdir order and broke
10+
every CI docs build this way). `load_external_modules: true` in `mkdocs.yml`
11+
makes resolution order-independent; this check enforces that property,
12+
because a regular build only ever exercises one arbitrary order.
13+
14+
mkdocstrings applies module loading and per-page options only on the first
15+
collect of a package (later pages find the package already collected), so
16+
each package under `docs/api/` gets the two hostile sides of that
17+
asymmetry, each from a fresh handler with an empty collection:
18+
19+
- its subpages first and its package index last, so pages rendering
20+
cross-package re-exports (the index above all) come after a plain
21+
subpage has already collected the package and nothing they declare
22+
themselves can still affect collection;
23+
- its package index alone, so the page with the most re-exports is itself
24+
the first collect over an empty collection.
25+
26+
Only the package's own pages are rendered: resolving `mcp` re-exports
27+
without ever rendering an `mcp_types` page is exactly the property under
28+
test.
29+
30+
Usage:
31+
python scripts/docs/check_render_order.py [--config mkdocs.gen.yml]
32+
33+
Run after `build_config.py` has produced the config and the `docs/api/`
34+
tree.
35+
"""
36+
37+
from __future__ import annotations
38+
39+
import argparse
40+
import traceback
41+
from pathlib import Path
42+
43+
# Sibling modules, same direct-invocation pattern as build_config.py:
44+
# gen_ref_pages owns the docs/api layout, llms_txt owns the page-URL mapping.
45+
import gen_ref_pages
46+
from llms_txt import page_url
47+
from zensical.compat import mkdocstrings as zensical_mkdocstrings
48+
from zensical.config import parse_config
49+
from zensical.markdown.render import render
50+
51+
API_DIR = gen_ref_pages.API_DIR
52+
DOCS_DIR = API_DIR.parent
53+
54+
55+
def _passes(package: str, pages: list[Path]) -> list[tuple[str, list[Path]]]:
56+
"""The labeled render orders exercising both sides of the first-collect asymmetry."""
57+
index = API_DIR / package / "index.md"
58+
if index not in pages:
59+
return [(f"'{package}'", pages)]
60+
subpages = [page for page in pages if page != index]
61+
if not subpages:
62+
return [(f"'{package}' index-alone", [index])]
63+
return [(f"'{package}' index-last", [*subpages, index]), (f"'{package}' index-alone", [index])]
64+
65+
66+
def _render(page: Path) -> None:
67+
"""Render one page the way Zensical's Rust core drives the Python side."""
68+
rel = page.relative_to(DOCS_DIR).as_posix()
69+
render(page.read_text(encoding="utf-8"), rel, page_url(rel))
70+
71+
72+
def main() -> None:
73+
parser = argparse.ArgumentParser(description=__doc__)
74+
parser.add_argument(
75+
"--config", default=str(gen_ref_pages.ROOT / "mkdocs.gen.yml"), help="Built config to render with"
76+
)
77+
args = parser.parse_args()
78+
parse_config(args.config)
79+
80+
packages: dict[str, list[Path]] = {}
81+
for page in sorted(API_DIR.rglob("*.md")):
82+
packages.setdefault(page.relative_to(API_DIR).parts[0], []).append(page)
83+
if not packages:
84+
raise SystemExit(f"check_render_order: no pages under {API_DIR} (run build_config.py first)")
85+
86+
for package in sorted(packages):
87+
for label, order in _passes(package, packages[package]):
88+
# Fresh Handlers -> empty griffe collection. Autorefs anchors
89+
# accumulate across passes, but they play no part in collection
90+
# or alias resolution (check_crossrefs owns link health).
91+
zensical_mkdocstrings.reset()
92+
for position, page in enumerate(order):
93+
try:
94+
_render(page)
95+
# Top-level handler: any exception from any page fails the
96+
# check; the traceback identifies whether the order was at
97+
# fault or something else broke (network, missing file).
98+
except Exception:
99+
traceback.print_exc()
100+
rel = page.relative_to(DOCS_DIR).as_posix()
101+
raise SystemExit(
102+
f"check_render_order: {rel} failed at position {position + 1}/{len(order)} of the"
103+
f" {label} order (traceback above; an AliasResolutionError means API rendering"
104+
" depends on page order — see `load_external_modules` in mkdocs.yml)"
105+
) from None
106+
print(f"check_render_order: {label} order OK ({len(order)} pages)")
107+
108+
109+
if __name__ == "__main__":
110+
main()

scripts/docs/gen_ref_pages.py

Lines changed: 15 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -64,22 +64,21 @@ def _compact_index(module: griffe.Module, documented: set[str]) -> str | None:
6464
"""Build a compact page body for a module that re-exports from outside its own subtree.
6565
6666
mkdocstrings renders a re-export whose canonical documentation lives on
67-
another page as a full duplicate of it: deterministically for aliases
68-
within one top-level package (`mcp.client.auth` re-exporting from
69-
`mcp.shared.auth`), and order-dependently across top-level packages
70-
(`from mcp_types import y` + `__all__` renders the duplicate only when
71-
the other package happens to be loaded already, and silently omits the
72-
member when it isn't). Modules whose exports all live in their own
73-
subtree (`mcp_types` re-exporting its private `._types` module, or a
74-
module whose `__all__` lists only its own definitions) are unaffected
75-
and keep the plain `::: module` stub (return `None`): their page is
76-
itself the canonical rendering.
77-
78-
For an affected module, pin the semantics instead of inheriting the
79-
accident: every export whose canonical page exists elsewhere under the API
80-
reference becomes a link to it, and only exports documented nowhere else
81-
(re-exports from private modules) keep their full body here, via an
82-
explicit `members:` list.
67+
another page as a full duplicate of it, whether the alias stays within
68+
one top-level package (`mcp.client.auth` re-exporting from
69+
`mcp.shared.auth`) or crosses packages (`from mcp_types import y` +
70+
`__all__` — `load_external_modules` in mkdocs.yml has the collector chase
71+
exported cross-package aliases when their package is first collected, so
72+
the target package is loaded regardless of page order). Modules whose
73+
exports all live in their own subtree (`mcp_types` re-exporting its
74+
private `._types` module, or a module whose `__all__` lists only its own
75+
definitions) are unaffected and keep the plain `::: module` stub (return
76+
`None`): their page is itself the canonical rendering.
77+
78+
For an affected module, replace the duplicates: every export whose
79+
canonical page exists elsewhere under the API reference becomes a link to
80+
it, and only exports documented nowhere else (re-exports from private
81+
modules) keep their full body here, via an explicit `members:` list.
8382
"""
8483
prefix = f"{module.path}."
8584
exports: dict[str, griffe.Object | griffe.Alias] = {}
@@ -138,18 +137,7 @@ def _compact_index(module: griffe.Module, documented: set[str]) -> str | None:
138137
entry += f" — {summary}"
139138
sections.setdefault(_KIND_SECTIONS[target.kind], []).append(entry)
140139

141-
# Rendering the stub resolves the cross-package aliases again, in
142-
# mkdocstrings' own collection. On a warm incremental rebuild the target
143-
# package's pages can all be cache hits, so nothing else loads it and the
144-
# resolution crashes (AliasResolutionError); preloading pins it. The
145-
# module's own root package needs no pin: rendering the stub loads it.
146-
preload = sorted(
147-
{member.target_path.split(".")[0] for member in exports.values() if member.is_alias}
148-
- {module.path.split(".")[0]}
149-
)
150140
body = [f"::: {module.path}", " options:"]
151-
if preload:
152-
body += [" preload_modules:", *(f" - {pkg}" for pkg in preload)]
153141
if inline:
154142
body += [" members:", *(f" - {name}" for name in inline)]
155143
else:

scripts/docs/llms_txt.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ def _dest_md_uri(src_uri: str) -> str:
9393
return "index.md" if directory == PurePosixPath(".") else f"{directory}/index.md"
9494

9595

96-
def _page_url(src_uri: str) -> str:
96+
def page_url(src_uri: str) -> str:
9797
"""The directory URL of a page relative to the site root (`servers/tools/`, `""` for the home page)."""
9898
return _dest_md_uri(src_uri).removesuffix("index.md")
9999

@@ -260,7 +260,7 @@ def rewrite(match: re.Match[str]) -> str:
260260
raise _BuildError(f"llms_txt: cannot resolve link target {target!r} in {src_uri}")
261261
if linked.endswith(".md"):
262262
# Pages without a markdown rendition (the api/ stubs) link to their HTML instead.
263-
url = _dest_md_uri(linked) if linked in prose else _page_url(linked)
263+
url = _dest_md_uri(linked) if linked in prose else page_url(linked)
264264
else:
265265
url = linked # assets are published at their docs-relative path
266266
return f"{opening}{site_url}{url}{anchor or ''}{title or ''}{closing}"
@@ -316,7 +316,7 @@ def generate(site_dir: Path) -> None:
316316
# same one `_title` falls back to).
317317
h1 = _prose_h1(markdown)
318318
body = markdown if h1 is None else markdown[: h1.start()] + markdown[h1.end() :]
319-
full += [f"# {title}", "", f"Source: {site_url}{_page_url(src_uri)}", "", body.strip(), ""]
319+
full += [f"# {title}", "", f"Source: {site_url}{page_url(src_uri)}", "", body.strip(), ""]
320320
index.append("")
321321

322322
index += ["## Optional", ""]
@@ -332,7 +332,7 @@ def generate(site_dir: Path) -> None:
332332
f" missing {sorted(generated - listed)}, stale {sorted(listed - generated)}"
333333
)
334334
for src_uri, title, description in _OPTIONAL_PAGES:
335-
index.append(f"- [{title}]({site_url}{_page_url(src_uri)}): {description}")
335+
index.append(f"- [{title}]({site_url}{page_url(src_uri)}): {description}")
336336
index.append("")
337337

338338
(site_dir / "llms.txt").write_text("\n".join(index), encoding="utf-8")

0 commit comments

Comments
 (0)