Skip to content

experimental-inspect: mark disjoint base classes with typing_extensions.disjoint_base - #6362

Open
jonasdedden wants to merge 6 commits into
PyO3:mainfrom
jonasdedden:introspection-disjoint-base
Open

experimental-inspect: mark disjoint base classes with typing_extensions.disjoint_base#6362
jonasdedden wants to merge 6 commits into
PyO3:mainfrom
jonasdedden:introspection-disjoint-base

Conversation

@jonasdedden

Copy link
Copy Markdown
Contributor

What was wrong

#[pyclass] stubs never emitted PEP 800 @disjoint_base, so stubtest reported 6 errors for the 6 non-final pyclasses in pytests.

The obvious rule does not hold

"Not final implies disjoint base" is wrong. stubtest's check (mypy _is_disjoint_base) is purely __basicsize__ / __itemsize__ differing from __base__. Measured on a scratch extension:

class disjoint at runtime
#[pyclass(subclass)] struct Base {} yes (24 vs 16)
#[pyclass(subclass, frozen)] struct FrozenEmpty {} no (16 vs 16)
#[pyclass(extends = Base, subclass)] struct MidEmpty {} no (24 vs 24)
#[pyclass(extends = PyDict, subclass)] struct DictSub {} yes (56 vs 48)
#[pyclass(extends = PyDict, subclass, frozen)] struct DictSubFrozen {} no (48 vs 48)

A blanket rule would decorate three of these wrongly, producing the inverse stubtest error and telling type checkers that legal multiple inheritance is impossible.

The fix

The exact criterion is whether the class adds to its base's instance layout, which is size_of::<PyClassObjectContents<T>>() > 0. That is the same quantity PyO3 already feeds into tp_basicsize / Py_tp_extra_basicsize (see src/pycell/impl_.rs, BASIC_SIZE).

@Tpt Tpt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Two questions

// Being a disjoint base depends on the instance layout, so the decorator list is
// picked by the compiler.
let disjoint_base = IntrospectionNode::List(vec![PyExpr::module_attr(
"typing_extensions",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Genuine question: are stubs going to be considered valid by all tools even if typing_extensions if not installed?

@jonasdedden jonasdedden Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checker verdict on @disjoint_base
mypy 1.0.1 – 1.17.1 error: Module "typing_extensions" has no attribute "disjoint_base" [attr-defined]
mypy 1.18.1 (released 2025-09-11) and up (through 2.3.1) clean
pyright ≤ 1.1.405 error: "disjoint_base" is unknown import symbol
pyright 1.1.406 (released 2025-10-02) clean

=> Seems to be actually typechecker dependent, since they ship their own typeshed copies AFAIK.

So this PR would lead at least to a minimum typechecker version dependency, but the versions required are ~10-ish months old. Dunno whether this is okay or not.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about other type checkers (ty, pyrefly etc?)

I think it'd be better IMO emit typing.disjoint_base here from the macro rather than assume they all support typing_extensions unconditionally

We can then have some options in pyo3-introspection:

  • we could replace typing with typing_extensions for this import
  • we could have some kind of version-based import
  • we could even just drop this decorator if targeting codebases older than 3.15

We could potentially give users some kind of control over which happens with config.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Dropping the decorator looks like a good safe first step to me

@jonasdedden jonasdedden Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm strongly suggesting not to do this (as it actually wouldn't be spec-compliant), but keep typing_extensions.disjoint_base as-is. Specifically not replace it with typing (heavily reduced Python version compat and not spec-compliant), not a version-based import (not required, and also not done in typeshed, and wouldn't increase compatibility), and not drop the decorator for code bases older than 3.15 (same; i.e. not required, but actually the type checker version instead is the version gate).

But a config flag where one could turn the feature off all together sounds reasonable, if we really want to support almost ~1 year old type checkers in an upcoming PyO3 release (do we?).

Explanation of this is here.

To give full signal on minimum version requirements:

  • All of this hinges on the first release of typeshed where disjoint_base was introduced, as this is what actually resolves typing_extensions.disjoint_base in downstream typechekers. PR and release of typing_extensions 4.15.0 was 2025-08-24/25 respectively
  • mypy introduced support in 1.18.1, released 2025-09-11
  • ty introduced support in this PR, 0.0.1-alpha.20 release was 2025-09-3
  • pyrefly introduced support in 0.29.0, released 2025-10-27
  • pyright introduced support in 1.1.406, released 2025-10-01

Comment thread pyo3-macros-backend/src/introspection.rs Outdated
Comment thread src/impl_/introspection.rs Outdated
// Being a disjoint base depends on the instance layout, so the decorator list is
// picked by the compiler.
let disjoint_base = IntrospectionNode::List(vec![PyExpr::module_attr(
"typing_extensions",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about other type checkers (ty, pyrefly etc?)

I think it'd be better IMO emit typing.disjoint_base here from the macro rather than assume they all support typing_extensions unconditionally

We can then have some options in pyo3-introspection:

  • we could replace typing with typing_extensions for this import
  • we could have some kind of version-based import
  • we could even just drop this decorator if targeting codebases older than 3.15

We could potentially give users some kind of control over which happens with config.

@jonasdedden

jonasdedden commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

There were a few suggestions spread across different places about typing_extensions.disjoint_base vs. typing.disjoint_base. I believe these are potentially fueled because of misunderstandings of how typing, typing_extensions, Python versions and typecheckers interact. Let me explain why the current status of this PR currently is actually spec compliant and the more correct solution.

TL;DR

# mod_te.pyi`
from typing_extensions import disjoint_base

@disjoint_base
class Foo: ...
# use_te.py
from mod_te import Foo
$ mypy --no-site-packages --python-version 3.10 use_te.py
Success: no issues found in 1 source file

=> Even if targetting Python version 3.10 in the type checker, this checks just fine.

If I use typing.disjoint_base instead, it does not work and mypy even suggests exactly what this PR is already doing:

$ mypy --no-site-packages --python-version 3.10 use_ty.py
mod_ty.pyi:1: error: Module "typing" has no attribute "disjoint_base"  [attr-defined]
mod_ty.pyi:1: note: Use `from typing_extensions import disjoint_base` instead
mod_ty.pyi:1: note: See https://mypy.readthedocs.io/en/stable/runtime_troubles.html#using-new-additions-to-the-typing-module
Found 1 error in 1 file (checked 1 source file)

Some facts

typeshed currently has 17 imports of disjoint_base, all of them from typing_extensions: https://github.com/search?q=repo%3Apython%2Ftypeshed+%22import+disjoint_base%22&type=code
If you search for a bit, you will even find INVERSE Python version checks to the one (I believe) you're suggesting: https://github.com/python/typeshed/blob/f40e0da70e10e818692c3a771d5d5ac7010042f9/stdlib/crypt.pyi#L16 (in this case of course because of some semantic changes, similar but opposite to one like this, but not becaue of Python version typing compatibility)

Every Python version >= 3.0 shall support everything in typing_extensions: https://github.com/python/typeshed/blob/f40e0da70e10e818692c3a771d5d5ac7010042f9/stdlib/VERSIONS#L322

And this explicit rule about features not present in every Python version having to be imported from typing_extensions:

Features from the typing module that are not present in all supported Python versions must be imported from typing_extensions instead in typeshed stubs.

Why is that?

Every type checker ships its own copy of typeshed internally as a fallback (making imports from typing_extensions not a runtime dependency), which includes typing.pyi but also typing_extensions.pyi. pyright for example does it here, mypy here. As stub files are not read at runtime but exclusively by the type checker, just the version of the type checker and its bundled typing_extensions define what can be used and what not. Even the "Python version" flags don't have any influence on this, since (as stated above) every Python version >=3.0 has to support everything from typing_extensions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants