I think we can rename the new typevar to _ExcReturnT because that's basically what it means: on exception, return either None or Never (every return type includes Never).

This PR so far has a significant drawback, as it assumes all @contextmanager-derived context decorators can suppress exceptions.

Therefore for

from collections.abc import Generator

@contextmanager
def not_suppressing() -> Generator[None]:
    yield

@not_suppressing()
def foo(x: int, y: int) -> int:
    return x // y

it causes foo() to be typed as (x: int, y: int) -> int | None, which is incorrect, as foo() always returns int.

Ideally, whether suppression happens or not could be inferred by the type checker.

jax error

It's because Fn (source) can be bound to a consistent subtype of Callable[P, R] (the type that replaced the _F/_AF type variables in this PR) that other consistent subtypes of Callable[P, R] aren't assignable to, e.g. T = TypeVar("T", bound=Callable[[], object]) can be bound to type[object] and def () -> object: isn't assignable to it. Assignability to Fn on the return type was previously guaranteed by _F.

I believe that in this sense ContextDecorator doesn't have to be truly stable, therefore these annotations may have to change downstream if we accept this PR at some point.

prefect error

silence_docker_warnings incorrectly extended decoratees's return type with None because of this PR's incorrect assumptions.

openlibrary error

Similarly for openlibrary, elapsed_time incorrectly extended decoratee's return type with None because of this PR's incorrect assumptions.

This PR is blocked until I figure out a way to make this edge case an opt-in.

That might either be via type checker special casing:

def example() -> int:
    with suppress():
        return 1

for both mypy and pyright, type checking this code errors out (although type checkers could be smarter in this exact case :-)):

from contextlib import suppress

def example() -> int:
    # [pyright] error: Function with declared return type "int" must return value on all code paths
    # [mypy] error: Missing return statement  [return]
    with suppress():
        return 1

and this doesn't:

from contextlib import nullcontext

def example() -> int:
    with nullcontext():
        return 1

So it definitely is possible to infer what _ExcReturnT for ContextDecorator should be, but the info has to be inferred from generators. To make this possible, _GeneratorContextManager and _AsyncGeneratorContextManager could adopt the new type variable to parametrize the ContextDecorator / AsyncContextDecorator base.

@contextmanager
def example() -> Generator[None]:
    # type checker should flag this as suppressing context manager
    # therefore `example` becomes a `Callable[_P, _GeneratorContextManager[..., None]]`
    with suppress():
        yield

@contextmanager
def example() -> Generator[None]:
    # normal path, this becomes a `Callable[_P, _GeneratorContextManager[..., Never]]`
    with nullcontext():
        yield

Oh, although that doesn't solve the problem with jax.

That shouldn't really be a huge issue if the annotation can be corrected downstream, question is how.
I think that Callable[P, R] should suffice. It better depicts what happens, because what is happening is the callable gets replaced with a possibly different assignable callable that can have different state or behavior (it does).

I exposed _ExcReturnT through _[Async]GeneratorContextManager with a default of Never. This should be fully backwards-compatible (except for the jax error) and reproduce the reported false positive. Now I'll need to research possibilities of getting type checkers to support this.

Diff from mypy_primer, showing the effect of this PR on open source code:

cki-lib (https://gitlab.com/cki-project/cki-lib)
- tests/test_metrics.py:75: error: Call to untyped function "dummy_function" in typed context  [no-untyped-call]

jax (https://github.com/google/jax)
+ jax/_src/api.py:2882: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], Any]", expected "F")  [return-value]

OK, this looks good.

This would not solve a related problem though:

from contextlib import contextmanager, suppress
from collections.abc import Generator

@contextmanager
def ignore() -> Generator[None]:
    with suppress(ZeroDivisionError):
        yield

def div(a: int, b: int) -> float:
    with ignore():
        return a / b

That should cause in the same error as in

from contextlib import suppress

def div(a: int, b: int) -> float:
    with suppress(ZeroDivisionError):
        return a / b

Which signalizes that _ExitT_co of Abstract[Async]ContextManager should be inferred from generators as well, but that's a separate problem to solve.

This led me to seeing a more general problem consisting of at least these 3 issues with how type checkers (don't) handle generator-based context managers:

1. Inferring _ExitT_co argument to AbstractContextManager in generator-based context managers by checking try-except blocks (that sometimes may only be reachable after recursively solving generator delegation) + same for async

2. Inferring _ExitT_co argument to AbstractContextManager in generator-based context managers by checking CM.__exit__ return type from with CM: yield blocks (that sometimes may only be reachable after recursively solving generator delegation) + same for async

3. Inferring _ExcReturnT argument to ContextDecorator that I added in this PR (possibly through AbstractContextManager) in generator-based context managers + same for async
   the _ExcReturnT is added to decorated callable's return type and is None if the exception can be suppressed by the decorating context manager or Never otherwise to retain the original return type

In my understanding, there are only two correct bindings of _ExitT_co and _ExcReturnT together

• Literal[True] + None
• Literal[False] | None + Never

Their individual upper bounds would remain the same, but there should be some way to enforce that logic.

This sheds new light on this issue as a much more general problem that needs discussion and new sections in the typing specification.