release(0.13.5): perf — cancellable flush sleep + CI hygiene (#60)

* perf(ci): cancel flush-thread sleep so shutdown() returns in ms, not 5s

The Transport flush loop used `time.sleep(self.config.flush_interval)` —
uncancellable, so any test or process that called `runtime.shutdown()`
while the thread was mid-sleep blocked on `thread.join()` for the full
default 5s flush_interval. With 1222 tests in the suite and many paths
calling shutdown() (or its fixture teardowns), this multiplied into
~10-15 minutes of pure teardown wall-clock per Python in the matrix.

Replace the bare sleep with `Event.wait`, which returns the instant
`stop()` sets the event. `stop()` now sets the event before
`join()`, and `start()` clears it so a restart-after-stop is
clean. Pin contract in tests/test_transport.py::

    test_stop_interrupts_flush_sleep

…uses a 30s flush_interval; pre-fix this took 30s, post-fix <5s.

CI hygiene in the same commit so the suite can actually use the freed
time:

- ci.yml / publish*.yml: enable pip cache (`cache: pip` +
  `cache-dependency-path: pyproject.toml`) — saves ~60-90s of cold
  install per matrix leg.
- ci.yml: `fail-fast: true` on the matrix — don't burn two more
  runner legs once one Python leg is red.
- ci.yml / coverage / publish*.yml: install `pytest-xdist>=3.6` and
  pass `-n auto` to pytest. `pytest-xdist` is also added to
  `[project.optional-dependencies.dev]` so a local
  `pip install -e .[dev]` brings it in.
- pyproject.toml: drop `-q` from `addopts` so CI logs show the
  full PASSED line per test (`--tb=short` keeps tracebacks compact).
  `-n auto` stays in the workflow, not the addopts, so a developer
  running `pytest tests/test_x.py` gets a single process.

No public API change. The runtime default FlushConfig is unchanged
(5s interval, 50 batch size); production flush cadence is identical.
The fix only shortens the worst-case shutdown latency.

* remove redundant docs

* chore(release): bump version 0.13.4 -> 0.13.5

Pairs with the preceding release/0.13.5 commits:

  * perf(ci): cancel flush-thread sleep (transport.py:816)
  * remove redundant docs (drift.md, sdk-v3-migration-gaps.md)

Wire format unchanged; pure version bump + changelog entry
covering both the perf fix and the CI hygiene so the SDK_MIN_VERSION
floor is up to date.

No on-wire breaking change; backends on 1.0.0 keep working
unchanged. Recommended upgrade path: 0.13.4 -> 0.13.5.

* fix(tests): stop transport flush thread between tests so it doesn't race respx

PR #60 landed the cancellable-sleep fix in Transport._flush_loop and
expected CI wall-clock to drop to 3-5 minutes. The first green run
on PR #60 (PR #60 run #1) actually took 9m 47s — the test step
dominated by a retry storm:

  Request failed (attempt 5/11), retrying in 8.46s: ConnectError
  Request failed (attempt 6/11), retrying in 9.16s: ConnectError
  ...
  Circuit breaker OPEN. Batch of 10 events will be re-queued.

Root cause: `tests/conftest.py:reset_runtime` teardown nulled the
runtime reference WITHOUT calling `runtime.shutdown()`. The
transport flush thread therefore kept running across tests, the
buffer drained through httpx with no respx context active, and the
xdist workers spent the next 9 minutes retry-sending the buffer
against the real (unreachable in CI) backend. `_retry_with_backoff
(max_retries=10, max_delay=10s)` is 65s of pure sleep per failed
batch, and with 4 xdist workers and many buffered batches this
multiplied into 9m 47s — i.e. a CI-noise fix that hid a deeper
lifecycle bug.

Pre-fix CI was already paying this cost (5s shutdown-sleep × 200+
tests ≈ 17 min of teardown per Python leg); the retry storm was
always there but masked by the dominant 5s cost. PR #60's 5s fix
exposed it.

Fix: add `flush: bool = True` to both `Transport.stop()` and
`NullRunRuntime.shutdown()`. When False, the transport thread is
cancelled WITHOUT a final `_do_flush()` / `_persist_to_wal()`.
`tests/conftest.py:reset_runtime` teardown now calls
`inst.shutdown(flush=False)` before nilling the reference. This
makes the conftest teardown a true no-op for the buffer — the test
that wrote the events is responsible for asserting on what it
cared about. The production default (`flush=True`) is preserved,
so the `nullrun.shutdown()` audit contract ("drain in-flight
events") is unchanged.

Pins:

  * `tests/test_transport.py::test_stop_flush_false_skips_final_flush
    ` — buffers an event, calls `stop(flush=False)` with no
    respx active, asserts the call returns in <1s AND the buffer
    is left untouched. Pre-fix this would have hung for 65s+ on
    the first retry.

  * `tests/test_init_contract.py::TestShutdownFlushKwarg::
    test_runtime_shutdown_flush_false_skips_final_flush` — same
    contract at the `NullRunRuntime` level: `shutdown(flush=False
    )` propagates the `flush=False` flag to
    `Transport.stop()`.

Public API additions:

  * `Transport.stop(timeout=10.0, flush: bool = True)` — `flush
    =False` is the new flag.
  * `NullRunRuntime.shutdown(flush: bool = True)` — propagates.
  * `nullrun.shutdown(timeout=2.0, flush: bool = True)` — passes
    `flush` through to the runtime.

No on-wire or production behaviour change. CI step is expected to
drop from ~9m 47s (PR #60 run #1) to ~30-60s on the next run.

Author: maltsev-dev

.github/workflows/ci.yml

@@ -11,7 +11,11 @@ jobs:
     runs-on: ubuntu-latest
     permissions:
       contents: read
+    # 2026-07-08: fail-fast on the first matrix failure instead of
+    # wasting runner minutes on the remaining Python versions when
+    # the suite is already red. Speed gain is per-run, not per-test.
     strategy:
+      fail-fast: true
       matrix:
         python: ["3.10", "3.11", "3.12"]
 
@@ -22,14 +26,29 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
+          # Cache pip's download cache keyed on the lock-relevant
+          # surfaces of pyproject.toml. Skips the ~60-90s cold
+          # install on warm caches; the action also reuses the
+          # cache across matrix legs when the key matches.
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install -e ".[dev]"
+          # xdist ships in the dev tree already; pin it explicitly so
+          # a future deps churn can't drop it without breaking CI.
+          pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest
+        # `-n auto` lets xdist pick a worker count from the runner's
+        # CPU count. With the transport cancellable-sleep fix the
+        # 5s-per-shutdown multiplier is gone, and xdist plus the
+        # existing respx-based mocking keeps the per-test wall clock
+        # near single-thread baseline (no shared state between
+        # workers — ``reset_runtime`` autouse fixture in conftest
+        # is per-process by construction under xdist).
+        run: pytest -n auto --durations=20
 
       - name: Run ruff
         run: ruff check src/
@@ -46,11 +65,16 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: "3.12"
-      - run: pip install -e ".[dev]"
-      - run: coverage run -m pytest
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
+      - run: pip install -e ".[dev]" "pytest-xdist>=3.6"
+      # Single Python leg for coverage — multi-version coverage
+      # reports don't add signal and double the runner time. 3.12
+      # is the modern floor for typing-only changes.
+      - run: coverage run -m pytest -n auto
       - uses: codecov/codecov-action@v4
         if: always()
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           files: ./coverage.xml
-          fail_ci_if_error: false
+          fail_ci_if_error: false

.github/workflows/publish-test.yml

@@ -19,12 +19,14 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
-        run: pip install -e ".[dev]"
+        run: pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest tests/ -v
+        run: pytest tests/ -v -n auto
 
   publish:
     name: Build and publish to TestPyPI
@@ -62,4 +64,4 @@ jobs:
           # re-runs of the same SHA a no-op (matching twine's
           # --skip-existing behaviour). Production PyPI cannot
           # overwrite anyway, so this flag is harmless there too.
-          skip-existing: true
+          skip-existing: true

.github/workflows/publish.yml

@@ -12,6 +12,9 @@ jobs:
     runs-on: ubuntu-latest
     permissions:
       contents: read
+    # 2026-07-08: parallel matrix kept (PyPI publish is a one-shot
+    # event and the runner is already paid for) but pip cache
+    # brought in for parity with ci.yml.
     strategy:
       matrix:
         python-version: ["3.10", "3.11", "3.12"]
@@ -22,12 +25,14 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
-        run: pip install -e ".[dev]"
+        run: pip install -e ".[dev]" "pytest-xdist>=3.6"
 
       - name: Run tests
-        run: pytest tests/ -v
+        run: pytest tests/ -v -n auto
 
   publish:
     name: Build and publish

docs/drift.md

@@ -17,7 +17,18 @@ name = "nullrun"
 # nullrun._singleton (the metaclass-backing descriptor) and
 # nullrun._registry (the runtime registry) so runtime.py stays the
 # orchestrator only. See __version__.py for the full changelog.
-version = "0.13.3"
+# 0.13.3 (2026-07-07): developer-ergonomics — `langgraph` import
+# semantics + cleanup of dead `protos/` target. See __version__.py.
+# 0.13.4 (2026-07-08): bug-fix — flatten the LangChain usage-
+# extraction elif-chain so every attribute source is read (not just
+# the first one with ``hasattr`` truthy). Pairs with PR #59.
+# 0.13.5 (2026-07-08): perf — make ``Transport._flush_loop`` sleep
+# cancellable (``threading.Event.wait`` instead of ``time.sleep``)
+# so ``runtime.shutdown()`` returns in ms instead of waiting out
+# the full ``flush_interval`` (5s default). Plus CI hygiene:
+# pip cache, ``fail-fast`` matrix, ``pytest-xdist -n auto``. No
+# on-wire change; backends on 1.0.0 keep working unchanged.
+version = "0.13.5"
 # Long form used by PyPI page meta-description and search snippets.
 # Kept under the 200-char preview threshold so the full line is visible
 # without an "expand" click. Keywords are matched against likely search
@@ -140,6 +151,14 @@ dev = [
     "ruff>=0.5",
     "coverage[toml]>=7.0",
     "httpx>=0.27.0,<1.0",
+    # xdist pins the parallel runner as a first-class dev dep so
+    # `pip install -e ".[dev]"` brings it in for local runs and
+    # CI both. The CI workflow also installs it explicitly to
+    # survive a future pyproject prune. ``-n auto`` is set in
+    # the workflow rather than ``addopts`` so single-CPU local
+    # runs (e.g. ``pytest tests/test_one.py``) don't accidentally
+    # spawn a worker pool.
+    "pytest-xdist>=3.6",
     # The SDK eagerly imports `nullrun.instrumentation.langgraph`
     # (from `nullrun.decorators`, imported by `nullrun.__init__` at
     # collection time), which itself does `from langchain_core.callbacks
@@ -477,7 +496,15 @@ ignore = [
 [tool.pytest.ini_options]
 asyncio_mode = "auto"
 testpaths = ["tests"]
-addopts = "--tb=short -q"
+# 2026-07-08: dropped the global ``-q`` so CI logs surface the
+# full PASSED line for each test (handy when scanning a red run).
+# Per-test verbosity stays low because ``--tb=short`` keeps the
+# tracebacks compact. ``-n auto`` lives in the workflow file, not
+# here, so a developer running ``pytest tests/test_x.py`` locally
+# gets a single process — the worker pool is only worth it on
+# the full suite, and some single-file debug sessions actively
+# want serial execution.
+addopts = "--tb=short"
 # Make the tests/ directory importable as a top-level package so
 # tests can use `from tests.conftest import BASE_URL`. Without this,
 # `from tests.conftest` raises ModuleNotFoundError on Python 3.10/3.11

docs/sdk-v3-migration-gaps.md

@@ -41,7 +41,7 @@ def my_agent(query):
 from nullrun.runtime import track_event, track_llm, track_tool
 
 
-def shutdown(timeout: float = 2.0) -> None:
+def shutdown(timeout: float = 2.0, flush: bool = True) -> None:
     """Gracefully shut down the NullRun runtime.
 
     Sends a clean WebSocket close frame, drains in-flight events, and
@@ -62,6 +62,13 @@ def shutdown(timeout: float = 2.0) -> None:
             ``NullRunRuntime.shutdown `` already caps WS join at
             0.5s and the WS close at 2.0s — this parameter is
             reserved for future expansion and is currently unused.
+        flush: when True (default) the transport drains any
+            buffered events to the backend on the way out. Pass
+            False to cancel the flush thread without a final
+            network call — used by the test conftest to teardown
+            between tests without racing the respx context exit
+            (see ``NullRunRuntime.shutdown(flush=False)`` for the
+            full rationale).
 
     Example::
 
@@ -75,7 +82,7 @@ def shutdown(timeout: float = 2.0) -> None:
     runtime = NullRunRuntime._instance  # type: ignore[attr-defined]
     if runtime is None:
         return
-    runtime.shutdown()
+    runtime.shutdown(flush=flush)
 
 
 def status():