Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 131 additions & 13 deletions docs/spec/directives.rst
Original file line number Diff line number Diff line change
Expand Up @@ -154,23 +154,141 @@ left undefined by the typing spec at this time.
Version and platform checking
-----------------------------

Type checkers are expected to understand simple version and platform
checks, e.g.::
Type checkers should support narrowing based on:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think "narrowing" is the right term here.

Suggested change
Type checkers should support narrowing based on:
Type checkers should understand code paths as definitely reachable or not reachable due to comparison tests against these symbols:

* ``sys.version_info``
* ``sys.platform``
* ``sys.implementation.version``
* ``sys.implementation.name``

import sys
Type checkers should support combining these checks with:
* A ``not`` unary operator
* An ``and`` or ``or`` binary operator

if sys.version_info >= (3, 12):
# Python 3.12+
else:
# Python 3.11 and lower
Type checkers are only required to support the fully-qualified form (e.g., ``sys.platform``).
Support for aliases or import variants (e.g., ``from sys import platform``) is not required, though type checkers may choose to support them.

if sys.platform == 'win32':
# Windows specific definitions
else:
# Posix specific definitions
The comparison patterns for these variables are described in more detail in the following paragraphs.

Don't expect a checker to understand obfuscations like
``"".join(reversed(sys.platform)) == "xunil"``.
sys.version_info checks
^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers should support the following comparison patterns:
* ``sys.version_info >= <2-tuple>``
* ``sys.version_info < <2-tuple>``

Comparisons checks are only supported against the first two elements of the version tuple.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned a while back in https://discuss.python.org/t/proposal-to-improve-support-for-other-python-platforms-in-the-typing-specification/91877/13, there are genuine use cases for also comparing against the patch version, e.g. in typeshed: https://github.com/python/typeshed/blob/4f84ac178fb23541475f9038840c5b149834374f/stdlib/heapq.pyi#L9-L11

So I propose we loosen this restriction to also allow for 3-tuples.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As also discussed in that thread, I think mandating support for 3-tuple comparisons opens up a can of worms regarding what "support" for 3-tuple comparisons even means. We cannot require type checkers to always precisely support such comparisons, because type checkers may not have access to precise micro Python versions: type checker configurations (and pyproject.toml etc) typically do not specify micro versions, and I don't think we want to require users to always start requiring them. (Whereas I think in practice all type checkers already do require users to always provide major/minor version in some way, explicitly or implicitly.) So what is the expected behavior if micro Python version is not available to the type checker?

For example, ty currently "supports" if sys.version_info >= (3, 14, 1) (when Python version is configured to 3.14) in the "sound" way, by treating it as "unknown truthiness" and considering that either path may be taken (since actual Python version could be 3.14.0 or 3.14.1+). Would that behavior be considered to meet the bar required of type checkers here, or not? In some sense that behavior is a "regression" from writing if sys.version_info >= (3, 14), where ty would always infer a definite truthiness.

("Get the micro version from the runtime Python environment" is not a reliable fallback. It requires running the Python executable, which is avoided by at least ty since it can ~double the overall runtime of checking a small project using a fast type checker. And type checking -- e.g. for a small project without dependencies -- doesn't even necessarily require a Python runtime environment to begin with.)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My instinct in case of sys.version_info >= (x, y, z) and sys.version_info < (x, y, z) when statically only x and y are known is to use the latest available z, because that's what I expect to be most common in practice. I'm sure there will be edge-cases that this could be problematic for, but on the macro I this this'll solve more problems that it'll cause.

But having just written this, I now see how the complexity of this solution might indeed not be worth it for a problem as niche as rare as this one.

So how about we don't require type-checkers to support 3-tuples, but explicitly state that type-checkers may choose to also support 3-tuples?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use the latest available z

Type checkers don't really have a way to know the latest available z at any point in time, either, so I think this rule would have to be something more like "assume z is ", or something like that. Which will lead to weird behavior in some edge cases (imagine a new feature is introduced in 3.15 beta and someone introduces an if sys.version_info <= (3, 14) in typeshed when it is introduced, and immediately type checkers all start assuming every user is running 3.15 beta or later). But you may be right that in practice it does the right thing most of the time.

So how about we don't require type-checkers to support 3-tuples, but explicitly state that type-checkers may choose to also support 3-tuples?

I think in general the assumption of this entire PR should be that type checkers can always choose to support more than what is required here, and the wording in all sections should be such as to leave that possibility open. If this is the route we take, I don't know that we need to specially emphasize it for 3-tuples, but we can.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Comparisons checks are only supported against the first two elements of the version tuple.
Comparison checks are only supported against the first two elements of the version tuple.

Use of named attributes is not mandated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use of named attributes is not mandated.
Type checkers are not expected to support comparisons with named attributes of `sys.version_info`.


.. code-block:: python
:caption: Example `sys.version_info`
:emphasize-lines: 2

import sys
if sys.version_info >= (3, 12):
# Python 3.12+
else:
# Python 3.11 and lower
Comment on lines +189 to +190

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to include elif in this example, so that it's clear that elif is also expected to be supported, and not just if and else. For example as:

Suggested change
else:
# Python 3.11 and lower
elif sys.version_info >= (3, 11):
# Python 3.11.*
else:
# Python 3.10 and lower


sys.platform checks
^^^^^^^^^^^^^^^^^^^

Type checkers should support the following comparison patterns:
* ``sys.platform == <string literal>``
* ``sys.platform != <string literal>``
* ``sys.platform in <tuple of string literals>``
* ``sys.platform not in <tuple of string literals>``
Comment on lines +198 to +199

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruff (PLR6201) will report an error for this tuple membership check, and I tend to agree with ruff that a set literal would be more idiomatic here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the wild I've seen things like sys.platform.startswith("freebsd") a couple of times, because in this case there's also a version number in the platform string (e.g. "freebsd8"). So how about we also allow sys.platform.startswith(<string literal>)?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty already supports sys.platform.startswith; I don't have any objection there.

Supporting set literals will be a little tricky in ty, but should be doable, and I'm not opposed to requiring support for it. I don't think the performance motivation of PLR6201 typically applies much to sys.version_info comparisons, but it is awkward if this rule is generally being applied in a codebase and has to be specifically ignored for sys.version_info checks.


Common values: ``"linux"``, ``"darwin"``, ``"win32"``, ``"emscripten"``, ``"wasi"``

The membership checks ``in`` and ``not in`` only support simple containment testing with a tuple of literal strings.

.. code-block:: python
:caption: Example `sys.platform`
:emphasize-lines: 2,4

import sys
if sys.platform == 'win32':
# Windows specific definitions
if sys.platform in ("linux", "darwin"):
# Platform-specific stubs for Linux and macOS
...


sys.implementation.name checks
Comment thread
Josverl marked this conversation as resolved.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers should support comparison patterns:
* ``sys.implementation.name == <string literal>``
* ``sys.implementation.name != <string literal>``
* ``sys.implementation.name in <tuple of string literals>``
* ``sys.implementation.name not in <tuple of string literals>``

Common values: ``"cpython"``, ``"pypy"``, ``"micropython"``, ``"graalpy"``, ``"jython"``, ``"ironpython"``

.. code-block:: python
:caption: Example `sys.implementation.name`
:emphasize-lines: 2,4

import sys
if sys.implementation.name == "cpython":
# CPython-specific stub
if sys.implementation.name == "micropython":
# MicroPython-specific stub
Comment on lines +220 to +236

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This raises similar questions as the three-tuple sys.version_info feature request above. Type checkers may commonly not have any source of information as to the Python implementation the checked code is expected to run under. What should be their behavior in this case?

I think for sys.implementation.name a reasonable choice would be "assume cpython unless known otherwise". Is that just a choice that is up to the individual type checker, or should it be standardized? It seems like if libraries begin to use these comparisons under the assumption type checkers will understand it, it can become a compatibility concern how type checkers behave here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess an alternative behavior to specify here would be that Python type checkers should not assume any particular implemention -- that is, should not treat these checks in any special way -- unless a particular known implementation has been configured.



sys.implementation.version checks

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, I think we should probably be explicit here that when a type checker has no "implementation" information, it should assume "CPython, and implementation version matches sys.version_info".

It's less clear to me what should happen if a type-checker is told that the implementation is not CPython, but is not given any specific version information. I guess this could be an error? Otherwise I'm not sure how type-checkers should guess at the implementation version.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers should support the following comparison patterns:
* ``sys.implementation.version >= <2-tuple>``
* ``sys.implementation.version < <2-tuple>``

Comparisons checks are only supported against the first two elements of the implementation version tuple.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Comparisons checks are only supported against the first two elements of the implementation version tuple.
Comparison checks are only supported against the first two elements of the implementation version tuple.

Use of named attributes is not mandated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use of named attributes is not mandated.
Type checkers are not required to support comparisons against named attributes of `sys.implementation.version`.


.. code-block:: python
:caption: Example `sys.implementation.version`
:emphasize-lines: 2,4

import sys
if sys.implementation.name == "pypy" and sys.implementation.version >= (7, 3):
# PyPy version 7.3 and above
if sys.implementation.name == "micropython" and sys.implementation.version >= (1, 24):
# MicroPython version 1.24 and above

.. note::

``sys.implementation.version`` is a tuple, in the same format as sys.version_info. However it represents the version of the Python implementation rather than the version of the Python language.
This has a distinct meaning from the specific version of the Python language to which the currently running interpreter conforms. For CPython this is the same as `sys.version_info`.
Comment on lines +261 to +262

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it belongs more as introductory text in this section, rather than as a note.



No support for complex expressions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Type checkers are only required to support the above patterns, and are not required to evaluate complex expressions involving these variables.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the expectation is that and and or boolean expressions are supported, that should be explicitly discussed here. Rather than saying "are not required to evaluate complex expressions" (which is vague), we should simply specify the forms that must be supported, and that no other form is required.

For example, the pattern ``sys.platform == "linux"`` is supported but other syntax variants such as ``platform == "linux"`` and ``"win" not in sys.platform`` are not mandated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we need to give exactly the same examples both in prose and then below in a code snippet.


Therefore checkers are **not required** to understand obfuscations such as:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "reversed" example is an obfuscation, but "win" not in sys.platform is not an obfuscation, and from sys import platform; platform == "linux" is definitely not an obfuscation.

Let's avoid value judgments and just give a clear presentation of the forms that are supported, and some example forms that type checkers are not required to support.


.. code-block:: python
:caption: Examples of unsupported or overly complex version/platform checks
:emphasize-lines: 3,5,7

import sys
from sys import platform
if "".join(reversed(sys.platform)) == "xunil":
# Linux specific code
if platform == "linux":
# Linux specific code
if "win" not in sys.platform:
# Non-Windows specific code


Configuration
^^^^^^^^^^^^^

Type checkers should provide configuration or CLI options to specify target sys.version, sys.platform, sys.implementation.name and sys.implementation.version.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Type checkers should provide configuration or CLI options to specify target sys.version, sys.platform, sys.implementation.name and sys.implementation.version.
Type checkers should provide configuration or CLI options to specify target ``sys.version``, ``sys.platform``, ``sys.implementation.name`` and ``sys.implementation.version``.

The exact mechanism for this is implementation-defined by the type checker.

.. _`deprecated`:

Expand Down