experimental-inspect: mark disjoint base classes with typing_extensions.disjoint_base - #6362
experimental-inspect: mark disjoint base classes with typing_extensions.disjoint_base#6362jonasdedden wants to merge 6 commits into
experimental-inspect: mark disjoint base classes with typing_extensions.disjoint_base#6362Conversation
| // Being a disjoint base depends on the instance layout, so the decorator list is | ||
| // picked by the compiler. | ||
| let disjoint_base = IntrospectionNode::List(vec![PyExpr::module_attr( | ||
| "typing_extensions", |
There was a problem hiding this comment.
Genuine question: are stubs going to be considered valid by all tools even if typing_extensions if not installed?
There was a problem hiding this comment.
| checker | verdict on @disjoint_base |
|---|---|
| mypy 1.0.1 – 1.17.1 | error: Module "typing_extensions" has no attribute "disjoint_base" [attr-defined] |
| mypy 1.18.1 (released 2025-09-11) and up (through 2.3.1) | clean |
| pyright ≤ 1.1.405 | error: "disjoint_base" is unknown import symbol |
| pyright 1.1.406 (released 2025-10-02) | clean |
=> Seems to be actually typechecker dependent, since they ship their own typeshed copies AFAIK.
So this PR would lead at least to a minimum typechecker version dependency, but the versions required are ~10-ish months old. Dunno whether this is okay or not.
There was a problem hiding this comment.
What about other type checkers (ty, pyrefly etc?)
I think it'd be better IMO emit typing.disjoint_base here from the macro rather than assume they all support typing_extensions unconditionally
We can then have some options in pyo3-introspection:
- we could replace
typingwithtyping_extensionsfor this import - we could have some kind of version-based import
- we could even just drop this decorator if targeting codebases older than 3.15
We could potentially give users some kind of control over which happens with config.
There was a problem hiding this comment.
+1
Dropping the decorator looks like a good safe first step to me
There was a problem hiding this comment.
I'm strongly suggesting not to do this (as it actually wouldn't be spec-compliant), but keep typing_extensions.disjoint_base as-is. Specifically not replace it with typing (heavily reduced Python version compat and not spec-compliant), not a version-based import (not required, and also not done in typeshed, and wouldn't increase compatibility), and not drop the decorator for code bases older than 3.15 (same; i.e. not required, but actually the type checker version instead is the version gate).
But a config flag where one could turn the feature off all together sounds reasonable, if we really want to support almost ~1 year old type checkers in an upcoming PyO3 release (do we?).
Explanation of this is here.
To give full signal on minimum version requirements:
- All of this hinges on the first release of
typeshedwheredisjoint_basewas introduced, as this is what actually resolvestyping_extensions.disjoint_basein downstream typechekers. PR and release oftyping_extensions4.15.0 was 2025-08-24/25 respectively mypyintroduced support in 1.18.1, released 2025-09-11tyintroduced support in this PR, 0.0.1-alpha.20 release was 2025-09-3pyreflyintroduced support in 0.29.0, released 2025-10-27pyrightintroduced support in 1.1.406, released 2025-10-01
| // Being a disjoint base depends on the instance layout, so the decorator list is | ||
| // picked by the compiler. | ||
| let disjoint_base = IntrospectionNode::List(vec![PyExpr::module_attr( | ||
| "typing_extensions", |
There was a problem hiding this comment.
What about other type checkers (ty, pyrefly etc?)
I think it'd be better IMO emit typing.disjoint_base here from the macro rather than assume they all support typing_extensions unconditionally
We can then have some options in pyo3-introspection:
- we could replace
typingwithtyping_extensionsfor this import - we could have some kind of version-based import
- we could even just drop this decorator if targeting codebases older than 3.15
We could potentially give users some kind of control over which happens with config.
|
There were a few suggestions spread across different places about TL;DR # mod_te.pyi`
from typing_extensions import disjoint_base
@disjoint_base
class Foo: ...# use_te.py
from mod_te import Foo=> Even if targetting Python version 3.10 in the type checker, this checks just fine. If I use Some facts
Every Python version >= 3.0 shall support everything in And this explicit rule about features not present in every Python version having to be imported from
Why is that? Every type checker ships its own copy of |
What was wrong
#[pyclass]stubs never emitted PEP 800@disjoint_base, so stubtest reported 6 errors for the 6 non-final pyclasses in pytests.The obvious rule does not hold
"Not final implies disjoint base" is wrong. stubtest's check (mypy
_is_disjoint_base) is purely__basicsize__/__itemsize__differing from__base__. Measured on a scratch extension:#[pyclass(subclass)] struct Base {}#[pyclass(subclass, frozen)] struct FrozenEmpty {}#[pyclass(extends = Base, subclass)] struct MidEmpty {}#[pyclass(extends = PyDict, subclass)] struct DictSub {}#[pyclass(extends = PyDict, subclass, frozen)] struct DictSubFrozen {}A blanket rule would decorate three of these wrongly, producing the inverse stubtest error and telling type checkers that legal multiple inheritance is impossible.
The fix
The exact criterion is whether the class adds to its base's instance layout, which is
size_of::<PyClassObjectContents<T>>() > 0. That is the same quantity PyO3 already feeds intotp_basicsize/Py_tp_extra_basicsize(seesrc/pycell/impl_.rs,BASIC_SIZE).