Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,6 @@ jobs:
with:
python-version: ${{ matrix.python }}
- run: uv sync
- run: uv run pytest -q
- run: uv run pytest -q -m "slow or not slow"
- run: uv run mypy
- run: uv run python benchmarks/overhead.py --tasks 10 --rounds 10
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@ a test that deadlocks *every* time at seed 4217, in milliseconds, under a debugg

## Status

**Early development — nothing usable yet.** First goal: a from-scratch
deterministic event loop that runs a toy client/server pair with identical
trace hashes across repeated runs at the same seed, and different orderings
across seeds.
**Early development.** The deterministic loop core works: seeded scheduling,
virtual time, replay-proving trace hashes, deadlock detection, and seeded
`sim.random` / `sim.uuid4` / `sim.time` shims, with the stdlib coordination
primitives (`Queue`, `Event`, `Lock`, `gather`, `TaskGroup`, `timeout`)
running unchanged on top. No simulated network yet. See
[the supported subset](docs/supported-api.md) for the exact contract.

## Planned

Expand Down
60 changes: 60 additions & 0 deletions benchmarks/overhead.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Compare scheduling overhead: SimLoop against the stock asyncio loop.

The workload passes a token around a ring of queue-connected tasks, so the
measured cost is almost purely task switching and queue hand-off. Run with
``python benchmarks/overhead.py [--tasks N] [--rounds M]``. SimLoop's number
includes trace recording — that is the honest price of replayability.
"""

from __future__ import annotations

import argparse
import asyncio
import time

from simloop import SimLoop


async def _token_ring(n_tasks: int, rounds: int) -> None:
queues: list[asyncio.Queue[int]] = [asyncio.Queue() for _ in range(n_tasks)]

async def worker(index: int) -> None:
for _ in range(rounds):
token = await queues[index].get()
await queues[(index + 1) % n_tasks].put(token + 1)

workers = [asyncio.create_task(worker(i)) for i in range(n_tasks)]
await queues[0].put(0)
await asyncio.gather(*workers)


def _measure(loop: asyncio.AbstractEventLoop, n_tasks: int, rounds: int) -> float:
start = time.perf_counter()
try:
loop.run_until_complete(_token_ring(n_tasks, rounds))
finally:
loop.close()
return time.perf_counter() - start


def main() -> None:
parser = argparse.ArgumentParser(
description="Measure SimLoop scheduling overhead vs stock asyncio."
)
parser.add_argument("--tasks", type=int, default=100)
parser.add_argument("--rounds", type=int, default=200)
args = parser.parse_args()
hops = args.tasks * args.rounds

stock = _measure(asyncio.new_event_loop(), args.tasks, args.rounds)
simulated = _measure(SimLoop(seed=0), args.tasks, args.rounds)

print(f"{args.tasks} tasks x {args.rounds} rounds = {hops} hops")
print(f"{'loop':<10}{'total s':>10}{'us/hop':>10}")
for label, seconds in (("asyncio", stock), ("SimLoop", simulated)):
print(f"{label:<10}{seconds:>10.4f}{seconds / hops * 1e6:>10.2f}")
print(f"overhead: {simulated / stock:.2f}x")


if __name__ == "__main__":
main()
41 changes: 41 additions & 0 deletions docs/supported-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Supported asyncio subset

simloop runs real, unmodified asyncio code on a simulated event loop. This
page is the honest contract: what is simulated, what works unchanged on top,
and what is fenced off because it would escape the simulation and break
determinism. Fenced APIs raise `SimulationFenceError` (a subclass of
`NotImplementedError`) naming the offending call.

## Simulated by SimLoop

| API | Behavior under simulation |
|---|---|
| `loop.call_soon` / `call_later` / `call_at` | Seeded ready-queue ordering; `(deadline, seq)` timer tie-break |
| `loop.time()` / `asyncio.sleep` | Virtual clock starting at 0.0; never waits on wall time |
| `loop.create_task` / `asyncio.create_task` | Real stdlib `Task`s, including custom task factories |
| `loop.create_future` | Real stdlib `Future`s |
| `run_until_complete` / `run_forever` / `stop` / `close` | Deadlock detection: raises `SimulationDeadlockError` when nothing can run |
| Handle / timer cancellation | Honored and recorded in the scheduling trace |
| Exception handling | Unhandled failures fail the run at `run_until_complete`; `set_exception_handler` supported |
| `sim.random` / `sim.uuid4` / `sim.time` | Seed-derived streams inside a run; stdlib fallback outside |

## Works unchanged on top of the loop

`asyncio.Queue`, `asyncio.Event`, `asyncio.Lock`, `asyncio.Semaphore`,
`asyncio.gather`, `asyncio.TaskGroup`, `asyncio.timeout`, `asyncio.wait_for`
— everything built purely on futures, tasks and timers. Timeouts fire in
virtual time. Each of these claims is exercised by the test suite.

## Planned: simulated network

`create_connection`, `create_server`, `create_datagram_endpoint` and the
streams API, backed by in-memory transports with seeded fault injection
(latency, drops, duplication, reordering, partitions).

## Fenced

Anything that reaches outside the simulation raises `SimulationFenceError`:
executors and threads (`run_in_executor`, `call_soon_threadsafe`), signal
handlers, subprocesses, raw sockets (`sock_*`), file-descriptor callbacks
(`add_reader` / `add_writer`), name resolution (`getaddrinfo` /
`getnameinfo`), TLS upgrades, `sendfile`, and pipes.
4 changes: 3 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ build-backend = "hatchling.build"

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-m 'not slow'"
markers = ["slow: long-running replay-stability checks"]

[tool.mypy]
strict = true
files = ["src", "tests"]
files = ["src", "tests", "benchmarks"]
13 changes: 11 additions & 2 deletions src/simloop/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
"""simloop — deterministic simulation testing for Python asyncio."""

from simloop._loop import SimLoop, SimulationDeadlockError
from simloop._loop import SimLoop, SimulationDeadlockError, SimulationFenceError
from simloop._sim import Sim, sim
from simloop._trace import TraceEvent

__version__ = "0.0.1.dev0"

__all__ = ["SimLoop", "SimulationDeadlockError", "TraceEvent", "__version__"]
__all__ = [
"Sim",
"SimLoop",
"SimulationDeadlockError",
"SimulationFenceError",
"TraceEvent",
"__version__",
"sim",
]
Loading
Loading