02-structured-tests

Structured pytest reporter

A small pytest plugin that emits one JSON object per test result and a final summary, either to stdout or to a file. Failure records include a short summary, the failing phase, the first file:line citation, and the last 12 lines of the failure message. Setup and teardown failures are recorded against the affected test.

Use

Drop pytest_jsonl_reporter.py into your repo. Either register it via your conftest.py:

# conftest.py
pytest_plugins = ["pytest_jsonl_reporter"]

or load it explicitly per run:

pytest -p ./pytest_jsonl_reporter.py --jsonl-out=test-results.jsonl

The output stream is one JSON object per line, plus a final kind=summary record:

{"nodeid": "tests/test_foo.py::test_bar", "outcome": "failed", "duration_ms": 12,
 "failure": {"summary": "AssertionError: expected 4 got 5",
             "citation": "tests/test_foo.py:42",
             "tail": ["    assert result == 4", "E   AssertionError: ..."]}}
{"kind": "summary", "duration_ms": 412, "total": 47, "passed": 46, "failed": 1, "skipped": 0, "exit_status": 1}

Prior art

This is a deliberately small reference. Production suites may prefer:

• pytest-json-report: the de facto standard, with a richer schema, hooks for custom metadata, and proper documentation.
• pytest-common-test-report-json: emits the Common Test Report Format JSON schema.

The plugin has no dependency beyond pytest, fits in one file, and composes with the evidence harness in ../03-verification-gate/. Use pytest-json-report when you need its broader schema and plugin ecosystem.

Why

The article's fifth lesson: tools that return structured failure beat tools that bury the cause in 200 lines of stderr. A pytest run is one of the most common tool calls an agent makes; making its output legible is a direct loop improvement.

Article reference: §5 (errors are the work).