|
| 1 | +.. currentmodule:: asyncio |
| 2 | + |
| 3 | +.. _asyncio-threading: |
| 4 | + |
| 5 | +asyncio and free-threaded Python |
| 6 | +================================ |
| 7 | + |
| 8 | +asyncio uses an event loop as a scheduler to enable highly efficient |
| 9 | +concurrency by switching between tasks to allow non-blocking I/O |
| 10 | +operations. This results in better performance for I/O-bound use |
| 11 | +cases. It also allows off-loading CPU-bound work to a thread or |
| 12 | +process pool, but that is still limited by the :term:`global |
| 13 | +interpreter lock` in CPython. |
| 14 | + |
| 15 | +However, in :ref:`free-threaded Python <freethreading-python-howto>`, |
| 16 | +the GIL is disabled and Python can run true multi-threaded code. This |
| 17 | +means that asyncio can now take advantage of multiple CPU cores without |
| 18 | +the limitations imposed by the GIL. |
| 19 | + |
| 20 | +Since Python 3.14, asyncio has first-class support for free-threaded |
| 21 | +Python, and the implementation of asyncio is safe to use in a |
| 22 | +multi-threaded environment. |
| 23 | + |
| 24 | +A single event loop on one core can handle many connections |
| 25 | +concurrently, but the Python code that runs to handle each one still |
| 26 | +executes serially. Once requests involve a non-trivial amount of |
| 27 | +per-request computation, that handling becomes the bottleneck, and a |
| 28 | +single core can no longer keep up. Combining asyncio with threads is |
| 29 | +most useful here: by running an event loop per thread, the handling of |
| 30 | +different requests can run in parallel across multiple CPU cores. It is |
| 31 | +also useful when you need to run blocking or CPU-bound code from an |
| 32 | +asyncio application. |
| 33 | + |
| 34 | + |
| 35 | +.. seealso:: |
| 36 | + |
| 37 | + `Scaling asyncio on Free-Threaded Python |
| 38 | + <https://labs.quansight.org/blog/scaling-asyncio-on-free-threaded-python>`__, |
| 39 | + a blog post by Kumar Aditya which explains the internal changes |
| 40 | + that make asyncio safe and efficient under free-threaded Python, |
| 41 | + together with benchmarks of the resulting improvements. |
| 42 | + |
| 43 | + |
| 44 | +Thread safety considerations |
| 45 | +---------------------------- |
| 46 | + |
| 47 | +While asyncio is designed to be thread-safe in a free-threaded Python |
| 48 | +environment, there are still some considerations to keep in mind when |
| 49 | +using asyncio with threads: |
| 50 | + |
| 51 | +1. **Event loop**: Each thread should have its own event loop which |
| 52 | + should not be shared across threads. This ensures that the event loop |
| 53 | + can manage its own tasks and callbacks without interference from |
| 54 | + other threads. |
| 55 | + |
| 56 | +2. **Task management**: Tasks and futures created in one thread should |
| 57 | + not be awaited or manipulated from another thread. |
| 58 | + |
| 59 | +3. **Thread-safe APIs**: When interacting with asyncio from multiple |
| 60 | + threads, it's important to use thread-safe APIs provided by asyncio, |
| 61 | + such as :func:`asyncio.run_coroutine_threadsafe` for submitting |
| 62 | + coroutines to an event loop from another thread. If you need to |
| 63 | + call a callback from a different thread, you can use |
| 64 | + :meth:`loop.call_soon_threadsafe` to schedule it safely. |
| 65 | + |
| 66 | +4. **Synchronization**: The synchronization primitives provided by |
| 67 | + asyncio (like :class:`asyncio.Lock` and :class:`asyncio.Event`) |
| 68 | + are not designed to be used across threads. If you need to |
| 69 | + synchronize between threads, you should use the synchronization |
| 70 | + primitives from the :mod:`threading` module instead. |
| 71 | + |
| 72 | + |
| 73 | +Using asyncio with threads |
| 74 | +-------------------------- |
| 75 | + |
| 76 | +asyncio supports running one event loop per thread, which allows you to |
| 77 | +take advantage of multiple CPU cores in a free-threaded Python |
| 78 | +environment. Each thread can run its own event loop, and tasks can be |
| 79 | +scheduled on those loops independently. |
| 80 | + |
| 81 | +Here's an example of how to use asyncio with threads:: |
| 82 | + |
| 83 | + import asyncio |
| 84 | + import threading |
| 85 | + |
| 86 | + async def worker(name: str) -> None: |
| 87 | + print(f"Worker {name} starting") |
| 88 | + await asyncio.sleep(1) |
| 89 | + print(f"Worker {name} done") |
| 90 | + |
| 91 | + def run_loop(name: str) -> None: |
| 92 | + asyncio.run(worker(name)) |
| 93 | + |
| 94 | + threads = [ |
| 95 | + threading.Thread(target=run_loop, args=(f"T{i}",)) |
| 96 | + for i in range(4) |
| 97 | + ] |
| 98 | + for t in threads: |
| 99 | + t.start() |
| 100 | + for t in threads: |
| 101 | + t.join() |
| 102 | + |
| 103 | +In this example, each thread creates its own event loop with |
| 104 | +:func:`asyncio.run` and runs a coroutine on it. The threads execute |
| 105 | +concurrently, and in a free-threaded build they can run on separate |
| 106 | +CPU cores in parallel. |
| 107 | + |
| 108 | + |
| 109 | +Producer/consumer across threads |
| 110 | +-------------------------------- |
| 111 | + |
| 112 | +When a regular (non-asyncio) thread needs to hand work to an asyncio |
| 113 | +event loop running in another thread, use a thread-safe primitive such |
| 114 | +as :class:`queue.Queue` rather than :class:`asyncio.Queue`, which is |
| 115 | +only safe within a single event loop.:: |
| 116 | + |
| 117 | + import asyncio |
| 118 | + import queue |
| 119 | + import threading |
| 120 | + |
| 121 | + def producer(q: queue.Queue[int]) -> None: |
| 122 | + for i in range(5): |
| 123 | + print(f"Producing {i}") |
| 124 | + q.put(i) |
| 125 | + q.shutdown() |
| 126 | + |
| 127 | + async def consumer(q: queue.Queue[int]) -> None: |
| 128 | + while True: |
| 129 | + try: |
| 130 | + item = q.get_nowait() |
| 131 | + except queue.Empty: |
| 132 | + await asyncio.sleep(0.1) |
| 133 | + continue |
| 134 | + except queue.ShutDown: |
| 135 | + break |
| 136 | + print(f"Consumed {item}") |
| 137 | + await asyncio.sleep(item) |
| 138 | + |
| 139 | + q: queue.Queue[int] = queue.Queue() |
| 140 | + consumer_thread = threading.Thread( |
| 141 | + target=lambda: asyncio.run(consumer(q)) |
| 142 | + ) |
| 143 | + consumer_thread.start() |
| 144 | + producer(q) |
| 145 | + consumer_thread.join() |
| 146 | + |
| 147 | +The producer runs on the main thread while the consumer runs inside an |
| 148 | +event loop on its own thread, yet they communicate safely through |
| 149 | +``queue.Queue``. When the queue is empty the consumer sleeps briefly |
| 150 | +and tries again. When the producer is done it calls |
| 151 | +:meth:`~queue.Queue.shutdown`, which causes subsequent |
| 152 | +:meth:`~queue.Queue.get_nowait` calls to raise :exc:`queue.ShutDown` |
| 153 | +so the consumer can exit cleanly. |
| 154 | + |
0 commit comments