Merge branch 'main' into use-attributes-to-guide-inline-values-size

Doc/c-api/init.rst

@@ -1904,6 +1904,58 @@ Python-level trace functions in previous versions.
 
 .. versionadded:: 3.12
 
+Reference tracing
+=================
+
+.. versionadded:: 3.13
+
+.. c:type:: int (*PyRefTracer)(PyObject *, int event, void* data)
+
+ The type of the trace function registered using :c:func:`PyRefTracer_SetTracer`.
+ The first parameter is a Python object that has been just created (when **event**
+ is set to :c:data:`PyRefTracer_CREATE`) or about to be destroyed (when **event**
+ is set to :c:data:`PyRefTracer_DESTROY`). The **data** argument is the opaque pointer
+ that was provided when :c:func:`PyRefTracer_SetTracer` was called.
+
+.. versionadded:: 3.13
+
+.. c:var:: int PyRefTracer_CREATE
+
+ The value for the *event* parameter to :c:type:`PyRefTracer` functions when a Python
+ object has been created.
+
+.. c:var:: int PyRefTracer_DESTROY
+
+ The value for the *event* parameter to :c:type:`PyRefTracer` functions when a Python
+ object has been destroyed.
+
+.. c:function:: int PyRefTracer_SetTracer(PyRefTracer tracer, void *data)
+
+ Register a reference tracer function. The function will be called when a new
+ Python has been created or when an object is going to be destroyed. If
+ **data** is provided it must be an opaque pointer that will be provided when
+ the tracer function is called. Return ``0`` on success. Set an exception and
+ return ``-1`` on error.
+
+ Not that tracer functions **must not** create Python objects inside or
+ otherwise the call will be re-entrant. The tracer also **must not** clear
+ any existing exception or set an exception. The GIL will be held every time
+ the tracer function is called.
+
+ The GIL must be held when calling this function.
+
+.. versionadded:: 3.13
+
+.. c:function:: PyRefTracer PyRefTracer_GetTracer(void** data)
+
+ Get the registered reference tracer function and the value of the opaque data
+ pointer that was registered when :c:func:`PyRefTracer_SetTracer` was called.
+ If no tracer was registered this function will return NULL and will set the
+ **data** pointer to NULL.
+
+ The GIL must be held when calling this function.
+
+.. versionadded:: 3.13
 
 .. _advanced-debugging:
 

Doc/library/functions.rst

@@ -524,11 +524,11 @@ are always available. They are listed here in alphabetical order.
 
 .. _func-eval:
 
-.. function:: eval(expression, globals=None, locals=None)
+.. function:: eval(source, /, globals=None, locals=None)
 
- :param expression:
+ :param source:
  A Python expression.
- :type expression: :class:`str` | :ref:`code object <code-objects>`
+ :type source: :class:`str` | :ref:`code object <code-objects>`
 
  :param globals:
  The global namespace (default: ``None``).
@@ -583,11 +583,15 @@ are always available. They are listed here in alphabetical order.
  Raises an :ref:`auditing event <auditing>` ``exec`` with the code object
  as the argument. Code compilation events may also be raised.
 
+ .. versionchanged:: 3.13
+
+ The *globals* and *locals* arguments can now be passed as keywords.
+
 .. index:: pair: built-in function; exec
 
-.. function:: exec(object, globals=None, locals=None, /, *, closure=None)
+.. function:: exec(source, /, globals=None, locals=None, *, closure=None)
 
- This function supports dynamic execution of Python code. *object* must be
+ This function supports dynamic execution of Python code. *source* must be
  either a string or a code object. If it is a string, the string is parsed as
  a suite of Python statements which is then executed (unless a syntax error
  occurs). [#]_ If it is a code object, it is simply executed. In all cases,
@@ -640,6 +644,10 @@ are always available. They are listed here in alphabetical order.
  .. versionchanged:: 3.11
  Added the *closure* parameter.
 
+ .. versionchanged:: 3.13
+
+ The *globals* and *locals* arguments can now be passed as keywords.
+
 
 .. function:: filter(function, iterable)
 
@@ -1733,8 +1741,9 @@ are always available. They are listed here in alphabetical order.
  :ref:`function` for details.
 
  A static method can be called either on the class (such as ``C.f()``) or on
- an instance (such as ``C().f()``). Moreover, they can be called as regular
- functions (such as ``f()``).
+ an instance (such as ``C().f()``).
+ Moreover, the static method :term:`descriptor` is also callable, so it can
+ be used in the class definition (such as ``f()``).
 
  Static methods in Python are similar to those found in Java or C++. Also, see
  :func:`classmethod` for a variant that is useful for creating alternate class

Doc/library/os.rst

@@ -2430,6 +2430,10 @@ features:
  platform-dependent. On some platforms, they are ignored and you should call
  :func:`chmod` explicitly to set them.
 
+ On Windows, a *mode* of ``0o700`` is specifically handled to apply access
+ control to the new directory such that only the current user and
+ administrators have access. Other values of *mode* are ignored.
+
  This function can also support :ref:`paths relative to directory descriptors
  <dir_fd>`.
 
@@ -2444,6 +2448,9 @@ features:
  .. versionchanged:: 3.6
  Accepts a :term:`path-like object`.
 
+ .. versionchanged:: 3.13
+ Windows now handles a *mode* of ``0o700``.
+
 
 .. function:: makedirs(name, mode=0o777, exist_ok=False)
 

Doc/whatsnew/3.13.rst

@@ -288,6 +288,10 @@ Other Language Changes
  class scopes are not inlined into their parent scope. (Contributed by
  Jelle Zijlstra in :gh:`109118` and :gh:`118160`.)
 
+* ``from __future__ import ...`` statements are now just normal
+ relative imports if dots are present before the module name.
+ (Contributed by Jeremiah Gabriel Pascual in :gh:`118216`.)
+
 
 New Modules
 ===========
@@ -701,6 +705,9 @@ pdb
  command line option or :envvar:`PYTHONSAFEPATH` environment variable).
  (Contributed by Tian Gao and Christian Walther in :gh:`111762`.)
 
+* :mod:`zipapp` is supported as a debugging target.
+ (Contributed by Tian Gao in :gh:`118501`.)
+
 queue
 -----
 
@@ -1957,6 +1964,11 @@ New Features
 * Add :c:func:`PyType_GetModuleByDef` to the limited C API
  (Contributed by Victor Stinner in :gh:`116936`.)
 
+* Add two new functions to the C-API, :c:func:`PyRefTracer_SetTracer` and
+ :c:func:`PyRefTracer_GetTracer`, that allows to track object creation and
+ destruction the same way the :mod:`tracemalloc` module does. (Contributed
+ by Pablo Galindo in :gh:`93502`.)
+
 
 Porting to Python 3.13
 ----------------------

Include/cpython/object.h

@@ -510,3 +510,13 @@ PyAPI_FUNC(int) PyType_Unwatch(int watcher_id, PyObject *type);
  * assigned, or 0 if a new tag could not be assigned.
  */
 PyAPI_FUNC(int) PyUnstable_Type_AssignVersionTag(PyTypeObject *type);
+
+
+typedef enum {
+ PyRefTracer_CREATE = 0,
+ PyRefTracer_DESTROY = 1,
+} PyRefTracerEvent;
+
+typedef int (*PyRefTracer)(PyObject *, PyRefTracerEvent event, void *);
+PyAPI_FUNC(int) PyRefTracer_SetTracer(PyRefTracer tracer, void *data);
+PyAPI_FUNC(PyRefTracer) PyRefTracer_GetTracer(void**);

Include/cpython/optimizer.h

@@ -142,7 +142,7 @@ void _Py_BloomFilter_Init(_PyBloomFilter *);
 void _Py_BloomFilter_Add(_PyBloomFilter *bloom, void *obj);
 PyAPI_FUNC(void) _Py_Executor_DependsOn(_PyExecutorObject *executor, void *obj);
 PyAPI_FUNC(void) _Py_Executors_InvalidateDependency(PyInterpreterState *interp, void *obj, int is_invalidation);
-extern void _Py_Executors_InvalidateAll(PyInterpreterState *interp, int is_invalidation);
+PyAPI_FUNC(void) _Py_Executors_InvalidateAll(PyInterpreterState *interp, int is_invalidation);
 
 /* For testing */
 PyAPI_FUNC(PyObject *)PyUnstable_Optimizer_NewCounter(void);

Include/internal/pycore_codecs.h

@@ -8,6 +8,17 @@ extern "C" {
 # error "this header requires Py_BUILD_CORE define"
 #endif
 
+#include "pycore_lock.h" // PyMutex
+
+/* Initialize codecs-related state for the given interpreter, including
+ registering the first codec search function. Must be called before any other
+ PyCodec-related functions, and while only one thread is active. */
+extern PyStatus _PyCodec_InitRegistry(PyInterpreterState *interp);
+
+/* Finalize codecs-related state for the given interpreter. No PyCodec-related
+ functions other than PyCodec_Unregister() may be called after this. */
+extern void _PyCodec_Fini(PyInterpreterState *interp);
+
 extern PyObject* _PyCodec_Lookup(const char *encoding);
 
 /* Text codec specific encoding and decoding API.
@@ -48,6 +59,26 @@ extern PyObject* _PyCodecInfo_GetIncrementalEncoder(
  PyObject *codec_info,
  const char *errors);
 
+// Per-interpreter state used by codecs.c.
+struct codecs_state {
+ // A list of callable objects used to search for codecs.
+ PyObject *search_path;
+
+ // A dict mapping codec names to codecs returned from a callable in
+ // search_path.
+ PyObject *search_cache;
+
+ // A dict mapping error handling strategies to functions to implement them.
+ PyObject *error_registry;
+
+#ifdef Py_GIL_DISABLED
+ // Used to safely delete a specific item from search_path.
+ PyMutex search_path_mutex;
+#endif
+
+ // Whether or not the rest of the state is initialized.
+ int initialized;
+};
 
 #ifdef __cplusplus
 }

Include/internal/pycore_interp.h

@@ -14,6 +14,7 @@ extern "C" {
 #include "pycore_atexit.h" // struct atexit_state
 #include "pycore_ceval_state.h" // struct _ceval_state
 #include "pycore_code.h" // struct callable_cache
+#include "pycore_codecs.h" // struct codecs_state
 #include "pycore_context.h" // struct _Py_context_state
 #include "pycore_crossinterp.h" // struct _xidregistry
 #include "pycore_dict_state.h" // struct _Py_dict_state
@@ -182,10 +183,7 @@ struct _is {
  possible to facilitate out-of-process observability
  tools. */
 
- PyObject *codec_search_path;
- PyObject *codec_search_cache;
- PyObject *codec_error_registry;
- int codecs_initialized;
+ struct codecs_state codecs;
 
  PyConfig config;
  unsigned long feature_flags;

Include/internal/pycore_list.h

@@ -28,7 +28,11 @@ _PyList_AppendTakeRef(PyListObject *self, PyObject *newitem)
  Py_ssize_t allocated = self->allocated;
  assert((size_t)len + 1 < PY_SSIZE_T_MAX);
  if (allocated > len) {
+#ifdef Py_GIL_DISABLED
+ _Py_atomic_store_ptr_release(&self->ob_item[len], newitem);
+#else
  PyList_SET_ITEM(self, len, newitem);
+#endif
  Py_SET_SIZE(self, len + 1);
  return 0;
  }

Include/internal/pycore_object.h

@@ -257,7 +257,7 @@ extern int _PyDict_CheckConsistency(PyObject *mp, int check_content);
  when a memory block is reused from a free list.
 
  Internal function called by _Py_NewReference(). */
-extern int _PyTraceMalloc_NewReference(PyObject *op);
+extern int _PyTraceMalloc_TraceRef(PyObject *op, PyRefTracerEvent event, void*);
 
 // Fast inlined version of PyType_HasFeature()
 static inline int

Include/internal/pycore_runtime.h

@@ -132,6 +132,12 @@ typedef struct _Py_DebugOffsets {
  } unicode_object;
 } _Py_DebugOffsets;
 
+/* Reference tracer state */
+struct _reftracer_runtime_state {
+ PyRefTracer tracer_func;
+ void* tracer_data;
+};
+
 /* Full Python runtime state */
 
 /* _PyRuntimeState holds the global state for the CPython runtime.
@@ -236,6 +242,7 @@ typedef struct pyruntimestate {
  struct _fileutils_state fileutils;
  struct _faulthandler_runtime_state faulthandler;
  struct _tracemalloc_runtime_state tracemalloc;
+ struct _reftracer_runtime_state ref_tracer;
 
  // The rwmutex is used to prevent overlapping global and per-interpreter
  // stop-the-world events. Global stop-the-world events lock the mutex

Include/internal/pycore_runtime_init.h

@@ -128,6 +128,10 @@ extern PyTypeObject _PyExc_MemoryError;
  }, \
  .faulthandler = _faulthandler_runtime_state_INIT, \
  .tracemalloc = _tracemalloc_runtime_state_INIT, \
+ .ref_tracer = { \
+ .tracer_func = NULL, \
+ .tracer_data = NULL, \
+ }, \
  .stoptheworld = { \
  .is_global = 1, \
  }, \