Confusing error: "Never" does not derive from BaseException

[pepoluan]

Complete error message is:

/home/pepoluan/Projects/aiosmtpd/aiosmtpd/controller.py
  166:19 - error: Invalid exception class or object
    "Never" does not derive from BaseException (reportGeneralTypeIssues)

The referenced line -- and some preceding ones -- is:

        try:
            self._testconn()
        except Exception:
            # We totally don't care of exceptions experienced by _testconn,
            # which _will_ happen if factory() experienced problems.
            pass
        if self._thread_exception is not None:
            raise self._thread_exception

Now, _thread_exception had been annotated as such:

    _thread_exception: Optional[Exception] = None

I am totally puzzled by this error message:

1. What is "Never" ? I've searched and found no occurrence of Never in the code.
2. Since _thread_exception is marked as Union[None, Exception], the if ... is not None line immediately before the raise line should've strongly indicated that _thread_exception is an Exception.
   a. Why doesn't pyright pick this up?
   b. Should I file a bug?

[erictraut]

The Never type is borrowed from the TypeScript compiler. For details, refer to this documentation. It means that the type checker has eliminated all possible types for that symbol at that point in the code flow — because the type has been narrowed down to nothing or no code flow path can be found to that location.

I'd need to see more of the function in question to say whether it's a bug. Yes, please file a bug and post a minimal, self-contained example. Thanks!

[pepoluan]

Full code is here: https://github.com/pepoluan/aiosmtpd/tree/pytest_based

The impacted code is aiosmtpd/controller.py, notably the Controller.start() method.

Do note that the code passed 100% coverage test, so that particular line is indeed reachable.

A bit difficult to provide minimal, self-contained example; will sharing the instructions to clone the branch do?

[erictraut]

Thanks, I should be able to reproduce the error from this code.

[pepoluan]

Sorry, it was wrong branch ... should be this one: https://github.com/pepoluan/aiosmtpd/tree/authenticator_see_session

And the impacted code is then this: https://github.com/pepoluan/aiosmtpd/blob/authenticator_see_session/aiosmtpd/controller.py#L144-L168

Snippet of shell session:

% git status
On branch authenticator_see_session
Your branch is up to date with 'win/authenticator_see_session'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        pyrightconfig.json

nothing added to commit but untracked files present (use "git add" to track)

% pyright
Loading configuration file at /home/pepoluan/Projects/aiosmtpd/pyrightconfig.json
Assuming Python version 3.8
Assuming Python platform Linux
Auto-excluding **/node_modules
Auto-excluding **/__pycache__
Auto-excluding .git
stubPath /home/pepoluan/Projects/aiosmtpd/typings is not a valid directory.
Searching for source files
Found 20 source files
/home/pepoluan/Projects/aiosmtpd/aiosmtpd/controller.py
--- 8< --- snipped --- >8 ---
  165:19 - error: Invalid exception class or object
    "Never" does not derive from BaseException (reportGeneralTypeIssues)

[erictraut]

OK, I looked at the code. The issue is that Pyright (like most type checkers) assumes that an instance variable's type won't change across method calls. This assumption allows type narrowing for expressions of the form self.x, which is both useful and important. But the assumption breaks if the code modifies the value of self.x "behind the back" of the type checker within another method.

The code contains the following check:

        if self._thread_exception is not None:
            raise self._thread_exception

Pyright narrows the type of self._thread_exception to None if the exception is not raised. The code later performs the same check:

        if self._thread_exception is not None:
            raise self._thread_exception

But here the type of expression self._thread_exception was already narrowed to None.

You can easily work around this by introducing another local variable like this:

        exc = self._thread_exception
        if exc is not None:
            raise exc

[pepoluan]

Ahhh, gotcha. Yeah, that variable may be changed in-between by an async in another thread.

Okay I'll try your workaround.