Forward-port generational GC to 3.15

Author: zanieb

Doc/library/gc.rst

@@ -45,9 +45,9 @@ The :mod:`!gc` module provides the following functions:
    Calling ``gc.collect(0)`` will perform a GC collection on the young generation.
 
    Calling ``gc.collect(1)`` will perform a GC collection on the young generation
-   and an increment of the old generation.
+   and generation 1.
 
-   Calling ``gc.collect(2)`` or ``gc.collect()`` performs a full collection
+   Calling ``gc.collect(2)`` or ``gc.collect()`` performs a full collection.
 
    The free lists maintained for a number of built-in types are cleared
    whenever a full collection or collection of the highest generation (2)
@@ -57,9 +57,6 @@ The :mod:`!gc` module provides the following functions:
    The effect of calling ``gc.collect()`` while the interpreter is already
    performing a collection is undefined.
 
-   .. versionchanged:: 3.14
-      ``generation=1`` performs an increment of collection.
-
 
 .. function:: set_debug(flags)
 
@@ -80,15 +77,12 @@ The :mod:`!gc` module provides the following functions:
    returned. If *generation* is not ``None``, return only the objects as follows:
 
    * 0: All objects in the young generation
-   * 1: No objects, as there is no generation 1 (as of Python 3.14)
+   * 1: All objects in generation 1
    * 2: All objects in the old generation
 
    .. versionchanged:: 3.8
       New *generation* parameter.
 
-   .. versionchanged:: 3.14
-      Generation 1 is removed
-
    .. audit-event:: gc.get_objects generation gc.get_objects
 
 .. function:: get_stats()
@@ -124,33 +118,27 @@ The :mod:`!gc` module provides the following functions:
    Set the garbage collection thresholds (the collection frequency). Setting
    *threshold0* to zero disables collection.
 
-   The GC classifies objects into two generations depending on whether they have
-   survived a collection. New objects are placed in the young generation. If an
-   object survives a collection it is moved into the old generation.
-
-   In order to decide when to run, the collector keeps track of the number of object
-   allocations and deallocations since the last collection.  When the number of
-   allocations minus the number of deallocations exceeds *threshold0*, collection
-   starts. For each collection, all the objects in the young generation and some
-   fraction of the old generation is collected.
+   The GC classifies objects into three generations depending on how many
+   collection sweeps they have survived. New objects are placed in the young
+   generation (generation ``0``). If an object survives a collection it is moved
+   into the next older generation. Since generation ``2`` is the oldest
+   generation, objects in that generation remain there after a collection.
+
+   In order to decide when to run, the collector keeps track of the number of
+   object allocations and deallocations since the last collection. When the
+   number of allocations minus the number of deallocations exceeds *threshold0*,
+   collection starts. Initially only generation ``0`` is examined. If
+   generation ``0`` has been examined more than *threshold1* times since
+   generation ``1`` has been examined, then generation ``1`` is examined as
+   well. With generation ``2``, things are a bit more complicated; see
+   `Garbage collector design <https://devguide.python.org/garbage_collector>`_
+   for more information.
 
    In the free-threaded build, the increase in process memory usage is also
-   checked before running the collector.  If the memory usage has not increased
+   checked before running the collector. If the memory usage has not increased
    by 10% since the last collection and the net number of object allocations
    has not exceeded 40 times *threshold0*, the collection is not run.
 
-   The fraction of the old generation that is collected is **inversely** proportional
-   to *threshold1*. The larger *threshold1* is, the slower objects in the old generation
-   are collected.
-   For the default value of 10, 1% of the old generation is scanned during each collection.
-
-   *threshold2* is ignored.
-
-   See `Garbage collector design <https://devguide.python.org/garbage_collector>`_ for more information.
-
-   .. versionchanged:: 3.14
-      *threshold2* is ignored
-
 
 .. function:: get_count()
 

Doc/whatsnew/3.14.rst

@@ -108,7 +108,7 @@ Interpreter improvements:
 * :ref:`A new type of interpreter <whatsnew314-tail-call-interpreter>`
 * :ref:`Free-threaded mode improvements <whatsnew314-free-threaded-cpython>`
 * :ref:`Improved error messages <whatsnew314-improved-error-messages>`
-* :ref:`Incremental garbage collection <whatsnew314-incremental-gc>`
+* :ref:`Generational garbage collection <whatsnew314-incremental-gc>`
 
 Significant improvements in the standard library:
 
@@ -955,26 +955,22 @@ when a module is imported) will still emit the syntax warning.
 
 .. _whatsnew314-incremental-gc:
 
-Incremental garbage collection
-------------------------------
-
-The cycle garbage collector is now incremental.
-This means that maximum pause times are reduced
-by an order of magnitude or more for larger heaps.
+Generational garbage collection
+-------------------------------
 
-There are now only two generations: young and old.
-When :func:`gc.collect` is not called directly, the
-GC is invoked a little less frequently. When invoked, it
-collects the young generation and an increment of the
-old generation, instead of collecting one or more generations.
+The cycle garbage collector uses three generations in the default build:
+generation 0, generation 1, and generation 2.
 
-The behavior of :func:`!gc.collect` changes slightly:
+The behavior of :func:`!gc.collect` matches the traditional generational
+collector:
 
-* ``gc.collect(1)``: Performs an increment of garbage collection,
-  rather than collecting generation 1.
-* Other calls to :func:`!gc.collect` are unchanged.
+* ``gc.collect(0)``: Collect generation 0.
+* ``gc.collect(1)``: Collect generations 0 and 1.
+* ``gc.collect(2)`` or ``gc.collect()``: Perform a full collection.
 
-(Contributed by Mark Shannon in :gh:`108362`.)
+Likewise, :func:`!gc.get_objects(1)` returns objects in generation 1, and
+:func:`!gc.set_threshold` and :func:`!gc.get_threshold` continue to use all
+three threshold values.
 
 
 Default interactive shell
@@ -2203,36 +2199,11 @@ difflib
 gc
 --
 
-* The new :ref:`incremental garbage collector <whatsnew314-incremental-gc>`
-  means that maximum pause times are reduced
-  by an order of magnitude or more for larger heaps.
-
-  Because of this optimization, the meaning of the results of
-  :meth:`~gc.get_threshold` and :meth:`~gc.set_threshold` have changed,
-  along with :meth:`~gc.get_count` and :meth:`~gc.get_stats`.
-
-  - For backwards compatibility, :meth:`~gc.get_threshold` continues to return
-    a three-item tuple.
-    The first value is the threshold for young collections, as before;
-    the second value determines the rate at which the old collection is scanned
-    (the default is 10, and higher values mean that the old collection
-    is scanned more slowly).
-    The third value is now meaningless and is always zero.
-
-  - :meth:`~gc.set_threshold` now ignores any items after the second.
-
-  - :meth:`~gc.get_count` and :meth:`~gc.get_stats` continue to return
-    the same format of results.
-    The only difference is that instead of the results referring to
-    the young, aging and old generations,
-    the results refer to the young generation
-    and the aging and collecting spaces of the old generation.
-
-  In summary, code that attempted to manipulate the behavior of the cycle GC
-  may not work exactly as intended, but it is very unlikely to be harmful.
-  All other code will work just fine.
-
-  (Contributed by Mark Shannon in :gh:`108362`.)
+* The :ref:`garbage collector uses three generations <whatsnew314-incremental-gc>`.
+  In particular, :func:`gc.collect(1)` collects generations 0 and 1,
+  :func:`gc.get_objects(1)` returns objects in generation 1, and
+  :func:`gc.set_threshold` and :func:`gc.get_threshold` continue to use all
+  three threshold values.
 
 
 io
@@ -3273,12 +3244,11 @@ Changes in the Python API
   Wrap it in :func:`staticmethod` if you want to preserve the old behavior.
   (Contributed by Serhiy Storchaka and Dominykas Grigonis in :gh:`121027`.)
 
-* The :ref:`garbage collector is now incremental <whatsnew314-incremental-gc>`,
-  which means that the behavior of :func:`gc.collect` changes slightly:
-
-  * ``gc.collect(1)``: Performs an increment of garbage collection,
-    rather than collecting generation 1.
-  * Other calls to :func:`!gc.collect` are unchanged.
+* The :ref:`garbage collector uses three generations <whatsnew314-incremental-gc>`.
+  In particular, :func:`gc.collect(1)` collects generations 0 and 1,
+  :func:`gc.get_objects(1)` returns objects in generation 1, and
+  :func:`gc.set_threshold` and :func:`gc.get_threshold` continue to use all
+  three threshold values.
 
 * The :func:`locale.nl_langinfo` function now temporarily sets the ``LC_CTYPE``
   locale in some cases.

Include/internal/pycore_gc.h

@@ -207,10 +207,6 @@ static inline void _PyGC_CLEAR_FINALIZED(PyObject *op) {
 
 extern void _Py_ScheduleGC(PyThreadState *tstate);
 
-#ifndef Py_GIL_DISABLED
-extern void _Py_TriggerGC(struct _gc_runtime_state *gcstate);
-#endif
-
 
 /* Tell the GC to track this object.
  *
@@ -220,7 +216,7 @@ extern void _Py_TriggerGC(struct _gc_runtime_state *gcstate);
  * ob_traverse method.
  *
  * Internal note: interp->gc.generation0->_gc_prev doesn't have any bit flags
- * because it's not object header.  So we don't use _PyGCHead_PREV() and
+ * because it's not an object header. So we don't use _PyGCHead_PREV() and
  * _PyGCHead_SET_PREV() for it to avoid unnecessary bitwise operations.
  *
  * See also the public PyObject_GC_Track() function.
@@ -245,18 +241,12 @@ static inline void _PyObject_GC_TRACK(
                           filename, lineno, __func__);
 
     struct _gc_runtime_state *gcstate = &_PyInterpreterState_GET()->gc;
-    PyGC_Head *generation0 = &gcstate->young.head;
-    PyGC_Head *last = (PyGC_Head*)(generation0->_gc_prev);
+    PyGC_Head *generation0 = gcstate->generation0;
+    PyGC_Head *last = (PyGC_Head *)(generation0->_gc_prev);
     _PyGCHead_SET_NEXT(last, gc);
     _PyGCHead_SET_PREV(gc, last);
-    uintptr_t not_visited = 1 ^ gcstate->visited_space;
-    gc->_gc_next = ((uintptr_t)generation0) | not_visited;
+    _PyGCHead_SET_NEXT(gc, generation0);
     generation0->_gc_prev = (uintptr_t)gc;
-    gcstate->young.count++; /* number of tracked GC objects */
-    gcstate->heap_size++;
-    if (gcstate->young.count > gcstate->young.threshold) {
-        _Py_TriggerGC(gcstate);
-    }
 #endif
 }
 
@@ -291,11 +281,6 @@ static inline void _PyObject_GC_UNTRACK(
     _PyGCHead_SET_PREV(next, prev);
     gc->_gc_next = 0;
     gc->_gc_prev &= _PyGC_PREV_MASK_FINALIZED;
-    struct _gc_runtime_state *gcstate = &_PyInterpreterState_GET()->gc;
-    if (gcstate->young.count > 0) {
-        gcstate->young.count--;
-    }
-    gcstate->heap_size--;
 #endif
 }
 

Include/internal/pycore_interp_structs.h

@@ -243,9 +243,16 @@ struct _gc_runtime_state {
     /* Is automatic collection enabled? */
     int enabled;
     int debug;
-    /* linked lists of container objects */
+
+    /* Generational GC state used in GIL builds. */
+    struct gc_generation generations[NUM_GENERATIONS];
+    PyGC_Head *generation0;
+    struct gc_generation_stats generation_stats_gen[NUM_GENERATIONS];
+
+    /* Incremental/free-threaded GC state. */
     struct gc_generation young;
     struct gc_generation old[2];
+
     /* a permanent generation which won't be collected */
     struct gc_generation permanent_generation;
     struct gc_stats *generation_stats;
@@ -265,7 +272,6 @@ struct _gc_runtime_state {
     int visited_space;
     int phase;
 
-#ifdef Py_GIL_DISABLED
     /* This is the number of objects that survived the last full
        collection. It approximates the number of long lived objects
        tracked by the GC.
@@ -278,6 +284,7 @@ struct _gc_runtime_state {
        the first time. */
     Py_ssize_t long_lived_pending;
 
+#ifdef Py_GIL_DISABLED
     /* True if gc.freeze() has been used. */
     int freeze_active;
 

Include/internal/pycore_runtime_init.h

@@ -130,6 +130,11 @@ extern PyTypeObject _PyExc_MemoryError;
         }, \
         .gc = { \
             .enabled = 1, \
+            .generations = { \
+                { .threshold = 2000, }, \
+                { .threshold = 10, }, \
+                { .threshold = 10, }, \
+            }, \
             .young = { .threshold = 2000, }, \
             .old = { \
                 { .threshold = 10, }, \