Skip to content

CI: Run docstring examples via pytest --doctest-modules - #920

Open
mmcky wants to merge 1 commit into
docs/consistency-auditfrom
ci/doctest-modules-866
Open

CI: Run docstring examples via pytest --doctest-modules#920
mmcky wants to merge 1 commit into
docs/consistency-auditfrom
ci/doctest-modules-866

Conversation

@mmcky

@mmcky mmcky commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Closes #866.

Wires the docstring-example sweep into ci.yml so examples become executable specifications and cannot silently rot again after the fixes in #864.

Note

Stacked on #864 (docs/consistency-audit), since the doctest run only passes with those 31 example fixes in place. After #864 merges, this PR should be retargeted to main (GitHub does this automatically on branch deletion) — please don't merge it before #864 lands.

What this adds

A new quantecon/conftest.py resolving both blockers from #866:

  • Blocker 1 (print-state leak): an autouse fixture snapshots np.get_printoptions() before each test and restores it after. The six game_theory examples that call np.set_printoptions(precision=4) stay exactly as published, and the 8 spurious failures they previously caused in markov, optimize and random disappear. This is the conftest route the issue weakly preferred: invisible to readers, with a docstring in the fixture explaining why it exists for future contributors.
  • Blocker 2 (unpassable modules): collect_ignore excludes util/notebooks.py (fetches over the network) and util/timing.py (prints wall-clock durations). This keeps the rendered docs free of # doctest: +SKIP directives — no cost paid by readers to satisfy CI.
  • The third, milder case (util/array.py::searchsorted emitting a DeprecationWarning) needs no handling: no strict warning filter is configured, so the warning is reported but does not fail the run. If a strict filter is ever adopted, this example will need a filterwarnings exemption.

A CI step in the existing tests job, after the regular pytest run:

- name: Run docstring examples (doctest)
  if: runner.os == 'Linux'
  shell: bash -l {0}
  run: |
    pytest --doctest-modules quantecon --ignore-glob='*/tests/*'

It runs on Linux only: one platform is enough to stop drift, float reprs are the platform-sensitive part of doctest output, and it sidesteps --ignore-glob path-separator issues on Windows. The --ignore-glob='*/tests/*' keeps the step from re-running the whole unit-test suite (that already ran in the previous step); the sweep itself takes ~8 seconds.

Verification

On this branch: pytest --doctest-modules quantecon --ignore-glob='*/tests/*'47 passed (was 11 failed, 39 passed before the conftest — the 8 print-option leaks plus notebooks/timing ×2, matching the issue's inventory). A regular pytest run over quantecon/tests, util/tests and game_theory/tests/test_pure_nash.py still passes with the new conftest in place.

🤖 Generated with Claude Code

Wires the docstring-example sweep from gh-866 into ci.yml so examples
cannot silently rot again after the gh-864 fixes.

A new quantecon/conftest.py resolves the two blockers raised in gh-866:

- An autouse fixture restores NumPy print options after each test, so
  the game_theory examples that set precision=4 no longer leak
  process-global state into doctests collected later (previously 8
  spurious failures in markov, optimize and random).
- collect_ignore excludes util/notebooks.py (fetches over the network)
  and util/timing.py (prints wall-clock durations), keeping the
  rendered docs free of `# doctest: +SKIP` directives.

The CI step runs on Linux only: one platform is enough to stop drift,
and it sidesteps --ignore-glob path-separator issues on Windows. The
DeprecationWarning from util/array.py::searchsorted needs no handling
since no strict warning filter is configured.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 2, 2026 09:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds automated execution of docstring examples as part of CI, turning doctests into continuously-validated specifications for the quantecon/ package. This aims to prevent future drift/rot in documented examples (per #866) by running pytest --doctest-modules in CI and addressing known collection/state-leak blockers.

Changes:

  • Add a quantecon/conftest.py to (a) reset NumPy print options between tests and (b) exclude modules whose doctests can’t pass deterministically (network / wall-clock output).
  • Extend the existing GitHub Actions test workflow to run a doctest sweep for quantecon on Linux.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
quantecon/conftest.py Adds pytest config to ignore non-deterministic doctest modules and an autouse fixture to prevent NumPy print-option leaks across doctests.
.github/workflows/ci.yml Adds a Linux-only CI step to run pytest --doctest-modules quantecon while ignoring */tests/* to avoid re-running unit tests.

Comment thread .github/workflows/ci.yml
@mmcky

mmcky commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

thanks @oyamad -- this was a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run docstring examples in CI (--doctest-modules)

2 participants