CI: Run docstring examples via pytest --doctest-modules - #920
Open
mmcky wants to merge 1 commit into
Open
Conversation
Wires the docstring-example sweep from gh-866 into ci.yml so examples cannot silently rot again after the gh-864 fixes. A new quantecon/conftest.py resolves the two blockers raised in gh-866: - An autouse fixture restores NumPy print options after each test, so the game_theory examples that set precision=4 no longer leak process-global state into doctests collected later (previously 8 spurious failures in markov, optimize and random). - collect_ignore excludes util/notebooks.py (fetches over the network) and util/timing.py (prints wall-clock durations), keeping the rendered docs free of `# doctest: +SKIP` directives. The CI step runs on Linux only: one platform is enough to stop drift, and it sidesteps --ignore-glob path-separator issues on Windows. The DeprecationWarning from util/array.py::searchsorted needs no handling since no strict warning filter is configured. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds automated execution of docstring examples as part of CI, turning doctests into continuously-validated specifications for the quantecon/ package. This aims to prevent future drift/rot in documented examples (per #866) by running pytest --doctest-modules in CI and addressing known collection/state-leak blockers.
Changes:
- Add a
quantecon/conftest.pyto (a) reset NumPy print options between tests and (b) exclude modules whose doctests can’t pass deterministically (network / wall-clock output). - Extend the existing GitHub Actions test workflow to run a doctest sweep for
quanteconon Linux.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
quantecon/conftest.py |
Adds pytest config to ignore non-deterministic doctest modules and an autouse fixture to prevent NumPy print-option leaks across doctests. |
.github/workflows/ci.yml |
Adds a Linux-only CI step to run pytest --doctest-modules quantecon while ignoring */tests/* to avoid re-running unit tests. |
Contributor
Author
|
thanks @oyamad -- this was a good idea. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #866.
Wires the docstring-example sweep into
ci.ymlso examples become executable specifications and cannot silently rot again after the fixes in #864.Note
Stacked on #864 (
docs/consistency-audit), since the doctest run only passes with those 31 example fixes in place. After #864 merges, this PR should be retargeted tomain(GitHub does this automatically on branch deletion) — please don't merge it before #864 lands.What this adds
A new
quantecon/conftest.pyresolving both blockers from #866:np.get_printoptions()before each test and restores it after. The sixgame_theoryexamples that callnp.set_printoptions(precision=4)stay exactly as published, and the 8 spurious failures they previously caused inmarkov,optimizeandrandomdisappear. This is the conftest route the issue weakly preferred: invisible to readers, with a docstring in the fixture explaining why it exists for future contributors.collect_ignoreexcludesutil/notebooks.py(fetches over the network) andutil/timing.py(prints wall-clock durations). This keeps the rendered docs free of# doctest: +SKIPdirectives — no cost paid by readers to satisfy CI.util/array.py::searchsortedemitting aDeprecationWarning) needs no handling: no strict warning filter is configured, so the warning is reported but does not fail the run. If a strict filter is ever adopted, this example will need afilterwarningsexemption.A CI step in the existing
testsjob, after the regular pytest run:It runs on Linux only: one platform is enough to stop drift, float reprs are the platform-sensitive part of doctest output, and it sidesteps
--ignore-globpath-separator issues on Windows. The--ignore-glob='*/tests/*'keeps the step from re-running the whole unit-test suite (that already ran in the previous step); the sweep itself takes ~8 seconds.Verification
On this branch:
pytest --doctest-modules quantecon --ignore-glob='*/tests/*'→ 47 passed (was 11 failed, 39 passed before the conftest — the 8 print-option leaks plusnotebooks/timing×2, matching the issue's inventory). A regularpytestrun overquantecon/tests,util/testsandgame_theory/tests/test_pure_nash.pystill passes with the new conftest in place.🤖 Generated with Claude Code