Some thoughts on Union[X, NoReturn]

[tusharsadhwani]

I stumbled upon code that looked like this recently:

def f(x: Union[int, str]) -> Union[int, NoReturn]:
    if isinstance(x, str):
        sys.exit(1)

   return x

The function's signature seems to say that it "sometimes returns int, and sometimes it never returns". Mypy is happy with this, but I am not.

There's a few reasons I think this is wrong:

1. Replacing Union[int, NoReturn] here with just int is also perfectly valid.
   Mypy seems fine with this code:

   def f(x: Union[int, str]) -> int:
       if isinstance(x, str):
           sys.exit(1)
   
      return x

   I believe that every instance of NoReturn | X | Y | ... can be written without the NoReturn, i.e. X | Y | ...

2. Having NoReturn in a union defeats its entire purpose.

   What's the use case of NoReturn? It is to avoid cases like these:

   def f() -> NoReturn:
       while True: do_stuff()
   
   value = f()  # Error: Function never returns

   If instead of the function never returning, it returns even some times, mypy won't be able to give you this error, making the use of int | NoReturn not really useful.

3. If Union[X, NoReturn] is correct here, all your code is wrong.

   If the type hint for "The function sometimes doesn't return" is actually Union[X, NoReturn], then any piece of code that raises an exception needs to be refactored.

   This, for example:

   def parse_int(x: str) -> int:
       try:
           value = int(x)
       except:
           raise ValueError('Not a valid integer.')
   
       return value

   This function as well, doesn't return sometimes. Does that mean we should use Union[int, NoReturn] here too? Even though here:

   num = parse_int("5")

   The type of num is guaranteed to be int? For this reason, I'd say int as a return type is much better, and Union[X, NoReturn] doesn't make sense.

---

This is why I wanted to know why mypy allows people to use Union[X, NoReturn]. I think it should not be allowed. I'd really like some discussion on this.

[erictraut]

I agree that X | NoReturn is generally not useful as a return type annotation and is effectively equivalent to X. There is at least one case where it is useful: in an overload implementation.

@overload
def func(x: int) -> NoReturn:
    ...

@overload
def func(x: str) -> str:
    ...

def func(x: int | str) -> str | NoReturn:
    ...

Without the ability to use str | NoReturn in the overload implementation, mypy (and pyright) will complain that the implementation isn't consistent with the overload signatures.

So there is at least one legitimate use of a return type of the form X | NoReturn. When used, it should be treated as equivalent to X. Pyright properly elides the NoReturn from the union in this case, but mypy does not. I think this is a bug in mypy.

def func() -> str | NoReturn:
    ...
reveal_type(func()) # pyright: str, mypy: Union[str, <nothing>]
func().lower() # pyright: no error, # mypy: NoReturn" of "Union[str, NoReturn]" has no attribute "lower"

[tushar-deepsource]

Hey!

This is actually where the entire discussion started from: mypy doesn't complain about that overload, while pyright does.

Sample code:

import sys
from typing import Union, NoReturn, overload

@overload
def f(x: int) -> int: ...

@overload
def f(x: str) -> NoReturn: ...

def f(x: Union[int, str]) -> int:
    if isinstance(x, str):
        sys.exit()

    return x

$ mypy mytest.py
Success: no issues found in 1 source file

$ pyright mytest.py
/Users/tusharsadhwani/mytest.py
  /Users/tusharsadhwani/mytest.py:11:5 - error: Overloaded function implementation is not consistent with signature of overload 2
    Function return type "NoReturn" is incompatible with type "int"
      "NoReturn" is incompatible with "int" (reportGeneralTypeIssues)

I'd say that mypy is doing the right thing here, not requiring NoReturn to be in the return type.

[tushar-deepsource]

The latter case of func().lower() erroring out on mypy is probably a mypy bug, I agree.

[erictraut]

I'd say that mypy is doing the right thing here

Yeah, on further thought, I agree. Pyright shouldn't require the NoReturn to be included in the implementation return type. I've filed a bug to remind myself to fix this.

[sobolevn]

NoReturn serves as a bottom-ish type in our type system. It helps us with the Halting problem.

So, technically speaking all X return types are X | NoReturn. And Union[int, NoReturn, NoReturn] should be flattened out as Union[int, NoReturn] or just int.

I see some benefit in allowing explicit NoReturn return type as in -> Union[int, NoReturn]:. In my POV it serves a documentation purpose. It means that "this function is more likely to fail than other ones, so be extra careful".

Probably you can use some flake8 (or Deepsource 😆) plugin to disallow code like this in your codebase if you don't like it.

[tushar-deepsource]

I can see the documentation point of view, just like how some re functions return AnyStr | Any instead of just Any.

But, I feel like this slightly disagrees with PEP 484, which says "Static type checkers will ensure that functions annotated as returning NoReturn truly never return, either implicitly or explicitly", as well as this comment.

Having the ambiguity of "you can either use int or int | NoReturn" leaves room for confusion and (imo) unnecessary discussions about personal preference. I feel there's more benefit in not doing that with NoReturn.

[Akuli]

I can see the documentation point of view, just like how some re functions return AnyStr | Any instead of just Any.

This is not actually for documentation. It means "code that handles the return value must work if it is a string, but it could be something else too". This is different from AnyStr which would be "return value is a string, end of story".

Technically, the correct type would be AnyStr | None, but then everyone would have to check for None, and that's just too annoying in practice. And plain AnyStr doesn't work, because then type checkers would think that code under if the_return_value is None: can never run, and give unreachable code warnings.

[tushar-deepsource]

@Akuli I meant to say AnyStr | Any is actually functionally the same as Any, but it lets the developer know that usually you should expect AnyStr.

[Akuli]

That isn't quite right either. foo + 2 will work if foo has type Any, but fail if it has type AnyStr | Any.

[tushar-deepsource]

Oh interesting. Didn't think about that :)