[pull] main from python:main

.github/workflows/posix-deps-apt.sh

@@ -26,9 +26,16 @@ apt-get -yq --no-install-recommends install \
     xvfb \
     zlib1g-dev
 
-# Workaround missing libmpdec-dev on ubuntu 24.04:
-# https://launchpad.net/~ondrej/+archive/ubuntu/php
-# https://deb.sury.org/
-sudo add-apt-repository ppa:ondrej/php
-apt-get update
-apt-get -yq --no-install-recommends install libmpdec-dev
+# Workaround missing libmpdec-dev on ubuntu 24.04 by building mpdecimal
+# from source. ppa:ondrej/php (launchpad.net) are unreliable 
+# (https://status.canonical.com) so fetch the tarball directly
+# from the upstream host.
+# https://www.bytereef.org/mpdecimal/
+MPDECIMAL_VERSION=4.0.1
+curl -fsSL "https://www.bytereef.org/software/mpdecimal/releases/mpdecimal-${MPDECIMAL_VERSION}.tar.gz" \
+    | tar -xz -C /tmp
+(cd "/tmp/mpdecimal-${MPDECIMAL_VERSION}" \
+    && ./configure --prefix=/usr/local \
+    && make -j"$(nproc)" \
+    && make install)
+ldconfig

Doc/faq/programming.rst

@@ -1924,7 +1924,7 @@ correctly using identity tests:
 
    .. code-block:: python
 
-      _sentinel = object()
+      _sentinel = sentinel('_sentinel')
 
       def pop(self, key, default=_sentinel):
           if key in self:

Doc/howto/descriptor.rst

@@ -594,7 +594,7 @@ a pure Python equivalent:
 
     def object_getattribute(obj, name):
         "Emulate PyObject_GenericGetAttr() in Objects/object.c"
-        null = object()
+        null = sentinel('null')
         objtype = type(obj)
         cls_var = find_name_in_mro(objtype, name, null)
         descr_get = getattr(type(cls_var), '__get__', null)
@@ -1635,12 +1635,12 @@ by member descriptors:
 
 .. testcode::
 
-    null = object()
+    null = sentinel('null')
 
     class Member:
 
         def __init__(self, name, clsname, offset):
-            'Emulate PyMemberDef in Include/structmember.h'
+            'Emulate PyMemberDef in Include/descrobject.h'
             # Also see descr_new() in Objects/descrobject.c
             self.name = name
             self.clsname = clsname

Doc/howto/perf_profiling.rst

@@ -217,8 +217,9 @@ Example, using the :mod:`sys` APIs in file :file:`example.py`:
 How to obtain the best results
 ------------------------------
 
-For best results, Python should be compiled with
-``CFLAGS="-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"`` as this allows
+For best results, keep frame pointers enabled. On supported GCC-compatible
+toolchains, CPython builds itself with ``-fno-omit-frame-pointer`` and, when
+available, ``-mno-omit-leaf-frame-pointer`` by default. These flags allow
 profilers to unwind using only the frame pointer and not on DWARF debug
 information. This is because as the code that is interposed to allow ``perf``
 support is dynamically generated it doesn't have any DWARF debugging information

Doc/library/compression.zstd.rst

@@ -331,10 +331,14 @@ Compressing and decompressing data in memory
 
       If *max_length* is non-negative, the method returns at most *max_length*
       bytes of decompressed data. If this limit is reached and further
-      output can be produced, the :attr:`~.needs_input` attribute will
-      be set to ``False``. In this case, the next call to
+      output can be produced (or EOF is reached), the :attr:`~.needs_input`
+      attribute will be set to ``False``. In this case, the next call to
       :meth:`~.decompress` may provide *data* as ``b''`` to obtain
-      more of the output.
+      more of the output. The full content can thus be read like::
+
+        process_output(d.decompress(data, max_length))
+        while not d.eof and not d.needs_input:
+            process_output(d.decompress(b"", max_length))
 
       If all of the input data was decompressed and returned (either
       because this was less than *max_length* bytes, or because

Doc/library/email.policy.rst

@@ -403,11 +403,26 @@ added matters.  To illustrate::
    .. attribute:: utf8
 
       If ``False``, follow :rfc:`5322`, supporting non-ASCII characters in
-      headers by encoding them as "encoded words".  If ``True``, follow
-      :rfc:`6532` and use ``utf-8`` encoding for headers.  Messages
+      headers by encoding them as :rfc:`2047` "encoded words".  If ``True``,
+      follow :rfc:`6532` and use ``utf-8`` encoding for headers.  Messages
       formatted in this way may be passed to SMTP servers that support
       the ``SMTPUTF8`` extension (:rfc:`6531`).
 
+      When ``False``, the generator will raise
+      :exc:`~email.errors.HeaderWriteError` if any header includes non-ASCII
+      characters in a context where :rfc:`2047` does not permit encoded words.
+      This particularly applies to mailboxes ("addr-spec") with non-ASCII
+      characters, which can be created via
+      :class:`~email.headerregistry.Address`. To use a mailbox with a non-ASCII
+      domain name with ``utf8=False``, first encode the domain using the
+      third-party :pypi:`idna` or :pypi:`uts46` module or with
+      :mod:`encodings.idna`. It is not possible to use a non-ASCII username
+      ("local-part") in a mailbox when ``utf8=False``.
+
+      .. versionchanged:: 3.15
+         Can trigger the raising of :exc:`~email.errors.HeaderWriteError`.
+         (Earlier versions incorrectly applied :rfc:`2047` in certain contexts,
+         mostly notably in addr-specs.)
 
    .. attribute:: refold_source
 

Doc/library/tarfile.rst

@@ -142,6 +142,10 @@ Some facts and figures:
    a Zstandard dictionary used to improve compression of smaller amounts of
    data.
 
+   For modes ``'w:gz'`` and ``'w|gz'``, :func:`tarfile.open` accepts the
+   keyword argument *mtime* to create a gzip archive header with that mtime. By
+   default, the mtime is set to the time of creation of the archive.
+
    For special purposes, there is a second format for *mode*:
    ``'filemode|[compression]'``.  :func:`tarfile.open` will return a :class:`TarFile`
    object that processes its data as a stream of blocks.  No random seeking will

Doc/library/threading.rst

@@ -1436,3 +1436,159 @@ is equivalent to::
 Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`,
 :class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as
 :keyword:`with` statement context managers.
+
+
+Iterator synchronization
+------------------------
+
+By default, Python iterators do not support concurrent access. Most iterators make
+no guarantees when accessed simultaneously from multiple threads. Generator
+iterators, for example, raise :exc:`ValueError` if one of their iterator methods
+is called while the generator is already executing. The tools in this section
+allow reliable concurrency support to be added to ordinary iterators and
+iterator-producing callables.
+
+The :class:`serialize_iterator` wrapper lets multiple threads share a single iterator and
+take turns consuming from it. While one thread is running ``__next__()``, the
+others block until the iterator becomes available. Each value produced by the
+underlying iterator is delivered to exactly one caller.
+
+The :func:`concurrent_tee` function lets multiple threads each receive the full
+stream of values from one underlying iterator. It creates independent iterators
+that all draw from the same source. Values are buffered until consumed by all
+of the derived iterators.
+
+.. class:: serialize_iterator(iterable)
+
+   Return an iterator wrapper that serializes concurrent calls to
+   :meth:`~iterator.__next__` using a lock.
+
+   If the wrapped iterator also defines :meth:`~generator.send`,
+   :meth:`~generator.throw`, or :meth:`~generator.close`, those calls
+   are serialized as well.
+
+   This makes it possible to share a single iterator, including a generator
+   iterator, between multiple threads. A lock ensures that calls are handled
+   one at a time. No values are duplicated or skipped by the wrapper itself.
+   Each item from the underlying iterator is given to exactly one caller.
+
+   This wrapper does not copy or buffer values. Threads that call
+   :func:`next` while another thread is already advancing the iterator will
+   block until the active call completes.
+
+   Example:
+
+   .. code-block:: python
+
+      import threading
+
+      def squares(n):
+          for x in range(n):
+              yield x * x
+
+      def consume(name, iterable):
+          for item in iterable:
+              print(name, item)
+
+      source = threading.serialize_iterator(squares(5))
+
+      t1 = threading.Thread(target=consume, args=("left", source))
+      t2 = threading.Thread(target=consume, args=("right", source))
+      t1.start()
+      t2.start()
+      t1.join()
+      t2.join()
+
+   In this example, each number is printed exactly once, but the work is shared
+   between the two threads.
+
+   .. versionadded:: next
+
+
+.. function:: synchronized_iterator(func)
+
+   Wrap an iterator-producing callable so that each iterator it returns is
+   automatically passed through :class:`serialize_iterator`.
+
+   This is especially useful as a :term:`decorator` for generator functions,
+   allowing their generator-iterators to be consumed from multiple threads.
+
+   Example:
+
+   .. code-block:: python
+
+      import threading
+
+      @threading.synchronized_iterator
+      def squares(n):
+          for x in range(n):
+              yield x * x
+
+      def consume(name, iterable):
+          for item in iterable:
+              print(name, item)
+
+      source = squares(5)
+
+      t1 = threading.Thread(target=consume, args=("left", source))
+      t2 = threading.Thread(target=consume, args=("right", source))
+      t1.start()
+      t2.start()
+      t1.join()
+      t2.join()
+
+   The returned wrapper preserves the metadata of *func*, such as its name and
+   wrapped function reference.
+
+   .. versionadded:: next
+
+
+.. function:: concurrent_tee(iterable, n=2)
+
+   Return *n* independent iterators from a single input *iterable*, with
+   guaranteed behavior when the derived iterators are consumed concurrently.
+
+   This function is similar to :func:`itertools.tee`, but is intended for cases
+   where the source iterator may feed consumers running in different threads.
+   Each returned iterator yields every value from the underlying iterable, in
+   the same order.
+
+   Internally, values are buffered until every derived iterator has consumed
+   them.
+
+   The returned iterators share the same underlying synchronization lock. Each
+   individual derived iterator is intended to be consumed by one thread at a
+   time. If a single derived iterator must itself be shared by multiple
+   threads, wrap it with :class:`serialize_iterator`.
+
+   If *n* is ``0``, return an empty tuple. If *n* is negative, raise
+   :exc:`ValueError`.
+
+   Example:
+
+   .. code-block:: python
+
+      import threading
+
+      def squares(n):
+          for x in range(n):
+              yield x * x
+
+      def consume(name, iterable):
+          for item in iterable:
+              print(name, item)
+
+      source = squares(5)
+      left, right = threading.concurrent_tee(source)
+
+      t1 = threading.Thread(target=consume, args=("left", left))
+      t2 = threading.Thread(target=consume, args=("right", right))
+      t1.start()
+      t2.start()
+      t1.join()
+      t2.join()
+
+   In this example, both consumer threads see the full sequence of squares
+   from a single generator expression.
+
+   .. versionadded:: next

Doc/library/zlib.rst

@@ -308,6 +308,11 @@ Decompression objects support the following methods and attributes:
    :attr:`unconsumed_tail`. This bytestring must be passed to a subsequent call to
    :meth:`decompress` if decompression is to continue.  If *max_length* is zero
    then the whole input is decompressed, and :attr:`unconsumed_tail` is empty.
+   For example, the full content could be read like::
+
+     process_output(d.decompress(data, max_length))
+     while chunk := d.decompress(d.unconsumed_tail, max_length):
+         process_output(chunk)
 
    .. versionchanged:: 3.6
       *max_length* can be used as a keyword argument.

Doc/using/configure.rst

@@ -780,6 +780,24 @@ also be used to improve performance.
 
    .. versionadded:: 3.14
 
+.. option:: --without-frame-pointers
+
+   Disable frame pointers, which are enabled by default (see :pep:`831`).
+
+   By default, the build appends ``-fno-omit-frame-pointer`` (and
+   ``-mno-omit-leaf-frame-pointer`` when the compiler supports it) to
+   ``BASECFLAGS`` so profilers, debuggers, and system tracing tools
+   (``perf``, ``eBPF``, ``dtrace``, ``gdb``) can walk the C call stack
+   without DWARF metadata. The flags propagate to third-party C
+   extensions through :mod:`sysconfig`. On compilers that do not
+   understand them, the build silently skips them.
+
+   Downstream packagers and authors of native libraries built with
+   custom build systems should set the same flags so the unwind chain
+   stays unbroken across all native frames.
+
+   .. versionadded:: 3.15
+
 .. option:: --without-mimalloc
 
    Disable the fast :ref:`mimalloc <mimalloc>` allocator

Doc/whatsnew/3.14.rst

@@ -953,10 +953,24 @@ when a module is imported) will still emit the syntax warning.
 (Contributed by Irit Katriel in :gh:`130080`.)
 
 
+.. _incremental-garbage-collection:
 .. _whatsnew314-incremental-gc:
 
-Incremental garbage collection
-------------------------------
+Garbage collection
+------------------
+
+**From Python 3.14.5 onwards:**
+
+The garbage collector (GC) has changed in Python 3.14.5.
+
+Python 3.14.0-3.14.4 shipped with a new incremental GC.
+However, due to a number of `reports
+<https://github.com/python/cpython/issues/142516>`__
+of significant memory pressure in production environments,
+it has been reverted back to the generational GC from 3.13.
+This is the GC now used in Python 3.14.5 and later.
+
+**Previously in Python 3.14.0-3.14.4:**
 
 The cycle garbage collector is now incremental.
 This means that maximum pause times are reduced
@@ -2203,7 +2217,18 @@ difflib
 gc
 --
 
-* The new :ref:`incremental garbage collector <whatsnew314-incremental-gc>`
+* **From Python 3.14.5 onwards:**
+
+  Python 3.14.0-3.14.4 shipped with a new incremental garbage collector.
+  However, due to a number of `reports
+  <https://github.com/python/cpython/issues/142516>`__
+  of significant memory pressure in production environments,
+  it has been reverted back to the generational GC from 3.13.
+  This is the GC now used in Python 3.14.5 and later.
+
+* **Previously in Python 3.14.0-3.14.4:**
+
+  The new :ref:`incremental garbage collector <whatsnew314-incremental-gc>`
   means that maximum pause times are reduced
   by an order of magnitude or more for larger heaps.
 
@@ -3447,3 +3472,17 @@ Changes in the C API
   functions on Python 3.13 and older.
 
 .. _pythoncapi-compat project: https://github.com/python/pythoncapi-compat/
+
+
+Notable changes in 3.14.5
+=========================
+
+gc
+--
+
+* The incremental garbage collector shipped in Python 3.14.0-3.14.4 has been
+  reverted back to the generational garbage collector from 3.13,
+  due to a number of `reports
+  <https://github.com/python/cpython/issues/142516>`__
+  of significant memory pressure in production environments.
+  See :ref:`whatsnew314-incremental-gc` for details.