Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion Doc/library/asyncio-eventloop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -838,7 +838,7 @@ Creating network servers
*, sock=None, backlog=100, ssl=None, \
ssl_handshake_timeout=None, \
ssl_shutdown_timeout=None, \
start_serving=True, cleanup_socket=True)
start_serving=True, cleanup_socket=True, mode=None)
:async:

Similar to :meth:`loop.create_server` but works with the
Expand All @@ -853,6 +853,13 @@ Creating network servers
be removed from the filesystem when the server is closed, unless the
socket has been replaced after the server has been created.

If *mode* is not ``None``, the permissions of the socket file created
for *path* are changed to *mode* (as accepted by :func:`os.chmod`)
right after binding, before the server starts accepting connections,
so a connection can never be accepted while the default,
umask-derived permissions are still in effect. *mode* cannot be
combined with *sock* and is not supported for abstract Unix sockets.

See the documentation of the :meth:`loop.create_server` method
for information about arguments to this method.

Expand All @@ -871,6 +878,10 @@ Creating network servers

Added the *cleanup_socket* parameter.

.. versionchanged:: 3.16

Added the *mode* parameter.


.. method:: loop.connect_accepted_socket(protocol_factory, \
sock, *, ssl=None, ssl_handshake_timeout=None, \
Expand Down
9 changes: 8 additions & 1 deletion Doc/library/asyncio-stream.rst
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,8 @@ and work with streams:
.. function:: start_unix_server(client_connected_cb, path=None, \
*, limit=None, sock=None, backlog=100, ssl=None, \
ssl_handshake_timeout=None, \
ssl_shutdown_timeout=None, start_serving=True, cleanup_socket=True)
ssl_shutdown_timeout=None, start_serving=True, \
cleanup_socket=True, mode=None)
:async:

Start a Unix socket server.
Expand All @@ -182,6 +183,9 @@ and work with streams:
be removed from the filesystem when the server is closed, unless the
socket has been replaced after the server has been created.

If *mode* is not ``None``, the permissions of the Unix socket file
are set to *mode* before the server starts accepting connections.

See also the documentation of :meth:`loop.create_unix_server`.

.. note::
Expand All @@ -205,6 +209,9 @@ and work with streams:
.. versionchanged:: 3.13
Added the *cleanup_socket* parameter.

.. versionchanged:: 3.16
Added the *mode* parameter.


StreamReader
============
Expand Down
9 changes: 9 additions & 0 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,15 @@ New modules
Improved modules
================

asyncio
-------

* Add the *mode* parameter to :meth:`asyncio.loop.create_unix_server` and
:func:`asyncio.start_unix_server` to set the permissions of the Unix
socket file created for *path*.
(Contributed by Sam Bull in :gh:`94984`.)


codecs
------

Expand Down
6 changes: 5 additions & 1 deletion Lib/asyncio/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ async def create_unix_server(
sock=None, backlog=100, ssl=None,
ssl_handshake_timeout=None,
ssl_shutdown_timeout=None,
start_serving=True):
start_serving=True, mode=None):
"""A coroutine which creates a UNIX Domain Socket server.

The return value is a Server object, which can be used to stop
Expand Down Expand Up @@ -480,6 +480,10 @@ async def create_unix_server(
the user should await Server.start_serving() or
Server.serve_forever() to make the server to start accepting
connections.

mode, if not None, is applied to the socket file created for
path with os.chmod() after binding and before the server
starts accepting connections.
"""
raise NotImplementedError

Expand Down
34 changes: 26 additions & 8 deletions Lib/asyncio/unix_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ async def create_unix_server(
sock=None, backlog=100, ssl=None,
ssl_handshake_timeout=None,
ssl_shutdown_timeout=None,
start_serving=True, cleanup_socket=True):
start_serving=True, cleanup_socket=True, mode=None):
if isinstance(ssl, bool):
raise TypeError('ssl argument must be an SSLContext or None')

Expand All @@ -294,6 +294,9 @@ async def create_unix_server(
'path and sock can not be specified at the same time')

path = os.fspath(path)
if mode is not None and path and path[0] in (0, '\x00'):
raise ValueError(
'mode is not supported for abstract sockets')
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

# Check for abstract socket. `str` and `bytes` paths are supported.
Expand Down Expand Up @@ -322,11 +325,26 @@ async def create_unix_server(
except:
sock.close()
raise

if mode is not None:
# The socket cannot accept connections until listen() is
# called, which happens later in Server._start_serving(),
# so no connection can be accepted while the socket still
# has the default permissions.
try:
os.chmod(path, mode)
except:
sock.close()
raise
else:
if sock is None:
raise ValueError(
'path was not specified, and no sock specified')

if mode is not None:
raise ValueError(
'mode is only meaningful with path')

if (sock.family != socket.AF_UNIX or
sock.type != socket.SOCK_STREAM):
raise ValueError(
Expand Down Expand Up @@ -658,19 +676,19 @@ def __init__(self, loop, pipe, protocol, waiter=None, extra=None):
# On AIX, the reader trick (to be notified when the read end of the
# socket is closed) only works for sockets. On other platforms it
# works for pipes and sockets. (Exception: OS X 10.4? Issue #19294.)
# On macOS, the trick misfires for named FIFOs (but not for pipes
# created with os.pipe(), which have st_nlink == 0): the write end
# polls as readable whenever unread data sits in the FIFO, and no
# On macOS and Solaris, the trick misfires for named FIFOs (but not for
# pipes created with os.pipe(), which have st_nlink == 0): the write
# end polls as readable whenever unread data sits in the FIFO, and no
# event is delivered when the read end is closed, so it can only
# ever report a false disconnection (gh-145030). The same xnu
# ever report a false disconnection (gh-145030). The same XNU
# behaviour applies on iOS/tvOS/watchOS (sys.platform is not
# "darwin" there).
is_named_fifo_on_apple = (
sys.platform in {"darwin", "ios", "tvos", "watchos"}
is_named_fifo_without_close_event = (
sys.platform in {"darwin", "ios", "tvos", "watchos", "sunos5"}
and is_fifo and pipe_stat.st_nlink > 0)
if is_socket or (is_fifo
and not sys.platform.startswith("aix")
and not is_named_fifo_on_apple):
and not is_named_fifo_without_close_event):
# only start reading when connection_made() has been called
self._loop.call_soon(self._loop._add_reader,
self._fileno, self._read_ready)
Expand Down
6 changes: 3 additions & 3 deletions Lib/test/test_asyncio/test_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -1729,9 +1729,9 @@ def reader(data):
"Don't support pipes for Windows")
@unittest.skipUnless(hasattr(os, 'mkfifo'), 'requires os.mkfifo()')
def test_write_named_fifo_unread_data(self):
# gh-145030: on macOS, the write end of a named FIFO polls as
# readable while unread data sits in the FIFO, which made the
# transport misinterpret the event as the reader hanging up
# gh-145030: on macOS and Solaris, the write end of a named FIFO
# polls as readable while unread data sits in the FIFO, which made
# the transport misinterpret the event as the reader hanging up
# and close itself.
path = os_helper.TESTFN
os.mkfifo(path)
Expand Down
51 changes: 51 additions & 0 deletions Lib/test/test_asyncio/test_unix_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,57 @@ def test_create_unix_server_bind_error(self, m_socket):
self.loop.run_until_complete(coro)
self.assertTrue(sock.close.called)

@socket_helper.skip_unless_bind_unix_socket
def test_create_unix_server_mode(self):
# Two distinct modes: whatever the umask, at most one of them
# can coincide with the default permissions, so a no-op chmod
# cannot pass both subtests.
for mode in (0o600, 0o644):
with self.subTest(mode=mode):
with test_utils.unix_socket_path() as path:
srv = self.loop.run_until_complete(
self.loop.create_unix_server(
lambda: None, path, mode=mode))
try:
self.assertEqual(
stat.S_IMODE(os.stat(path).st_mode), mode)
finally:
srv.close()
self.loop.run_until_complete(srv.wait_closed())

def test_create_unix_server_mode_sock(self):
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
with sock:
coro = self.loop.create_unix_server(lambda: None, path=None,
sock=sock, mode=0o600)
with self.assertRaisesRegex(ValueError,
'mode is only meaningful with path'):
self.loop.run_until_complete(coro)

def test_create_unix_server_mode_abstract(self):
# The check is a pure string test, so it runs on all platforms.
for path in ('\x00spam', b'\x00spam'):
with self.subTest(path=path):
coro = self.loop.create_unix_server(lambda: None, path,
mode=0o600)
with self.assertRaisesRegex(
ValueError, 'mode is not supported for abstract'):
self.loop.run_until_complete(coro)

@mock.patch('asyncio.unix_events.socket')
def test_create_unix_server_chmod_error(self, m_socket):
# Ensure that the socket is closed when os.chmod() fails
sock = mock.Mock()
m_socket.socket.return_value = sock

with mock.patch('asyncio.unix_events.os.chmod',
side_effect=PermissionError):
coro = self.loop.create_unix_server(lambda: None, path='/test',
mode=0o600)
with self.assertRaises(PermissionError):
self.loop.run_until_complete(coro)
self.assertTrue(sock.close.called)

def test_create_unix_connection_path_sock(self):
coro = self.loop.create_unix_connection(
lambda: None, os.devnull, sock=object())
Expand Down
13 changes: 13 additions & 0 deletions Lib/test/test_typing.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import annotationlib
import atexit
import contextlib
import collections
import collections.abc
Expand Down Expand Up @@ -6550,6 +6551,14 @@ class F:
class InternalsTests(BaseTestCase):
def test_collect_parameters(self):
typing = import_helper.import_fresh_module("typing")
# Importing typing registers an internal function named _clear_caches
# with atexit. The throwaway module created here installs its own
# handler, which holds the module alive and keeps references until
# interpreter shutdown even after the test finishes. Each repetition
# of this test under -R would therefore leak another module copy. To
# avoid this, we unregister the handler once the test is done.
self.addCleanup(atexit.unregister, typing._clear_caches)

with self.assertWarnsRegex(
DeprecationWarning,
"The private _collect_parameters function is deprecated"
Expand Down Expand Up @@ -7719,6 +7728,10 @@ def test_bytestring(self):

with self.assertWarns(DeprecationWarning):
from typing import ByteString
# Drop the exit handler of this throwaway copy, see the comment in
# InternalsTests.test_collect_parameters.
self.addCleanup(atexit.unregister, sys.modules["typing"]._clear_caches)

with self.assertWarns(DeprecationWarning):
self.assertIsInstance(b'', ByteString)
with self.assertWarns(DeprecationWarning):
Expand Down
6 changes: 6 additions & 0 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
"""

from abc import abstractmethod, ABCMeta
import atexit
import collections
from collections import defaultdict
import collections.abc
Expand Down Expand Up @@ -397,6 +398,11 @@ def _clear_caches():
cleanup()


# Release the LRU caches at shutdown, they otherwise redistribute reference
# leaks of one extension to types of unrelated ones. See GH-151728.
atexit.register(_clear_caches)


def _tp_cache(func=None, /, *, typed=False):
"""Internal wrapper caching __getitem__ of generic types.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Fix a data race on thread handle identifiers when ``_thread._shutdown()``
runs concurrently with the startup of non-daemon threads in the
:term:`free-threaded build`.
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Fix :mod:`asyncio` write pipe transports for named FIFOs on macOS. Unread
data sitting in the FIFO made the transport misinterpret a poll event as
the reader disconnecting, wrongly closing the transport.
Fix :mod:`asyncio` write pipe transports for named FIFOs on macOS and Solaris.
Unread data sitting in the FIFO made the transport misinterpret a poll event
as the reader disconnecting, wrongly closing the transport.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add the *mode* parameter to :meth:`asyncio.loop.create_unix_server` and
:func:`asyncio.start_unix_server` to set the permissions of the Unix
socket file created for *path*, applied before the server starts
accepting connections.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Clear the internal :mod:`typing` caches from an exit handler. Previously, an
extension module that leaked a reference to :mod:`typing` would also keep every
subscripted type alive past interpreter shutdown, including types owned by
unrelated extension modules.
2 changes: 1 addition & 1 deletion Modules/_threadmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -2413,7 +2413,7 @@ thread_shutdown(PyObject *self, PyObject *args)
struct llist_node *node;
llist_for_each_safe(node, &state->shutdown_handles) {
ThreadHandle *cur = llist_data(node, ThreadHandle, shutdown_node);
if (cur->ident != ident) {
if (ThreadHandle_ident(cur) != ident) {
ThreadHandle_incref(cur);
handle = cur;
break;
Expand Down
Loading