Mypy plugin reports false positives and not existing lines

[febus982]

Bug Report

mypy identifies false positive X | Y errors on not existing lines if pydantic.mypy plugin is enabled.

Note that I cannot find any instance of X | Y in the whole repository.

To Reproduce

This is the file: https://github.com/febus982/bootstrap-python-fastapi/blob/main/src/bootstrap/config.py

from pathlib import Path
from typing import Dict, Literal, Optional

from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict
from sqlalchemy_bind_manager import SQLAlchemyConfig

TYPE_ENVIRONMENT = Literal["local", "test", "staging", "production"]


class CeleryConfig(BaseModel):
    # https://docs.celeryq.dev/en/stable/userguide/configuration.html#configuration

    timezone: str = "UTC"

    # Broker config
    broker_url: Optional[str] = None
    broker_connection_retry_on_startup: bool = True

    # Results backend config
    result_backend: Optional[str] = None
    redis_socket_keepalive: bool = True

    # Enable to ignore the results by default and not produce tombstones
    task_ignore_result: bool = False

    # We want to use the default python logger configured using structlog
    worker_hijack_root_logger: bool = False

    # Events enabled for monitoring
    worker_send_task_events: bool = True
    task_send_sent_event: bool = True

    # Recurring tasks triggered directly by Celery
    beat_schedule: dict = {}
    # beat_schedule: dict = {
    #     "recurrent_example": {
    #         "task": "domains.books._tasks.book_cpu_intensive_task",
    #         "schedule": 5.0,
    #         "args": ("a-random-book-id",),
    #     },
    # }


class AppConfig(BaseSettings):
    model_config = SettingsConfigDict(env_nested_delimiter="__")

    APP_NAME: str = "bootstrap"
    CELERY: CeleryConfig = CeleryConfig()
    DEBUG: bool = False
    ENVIRONMENT: TYPE_ENVIRONMENT = "local"
    SQLALCHEMY_CONFIG: Dict[str, SQLAlchemyConfig] = dict(
        default=SQLAlchemyConfig(
            engine_url=f"sqlite+aiosqlite:///{Path(__file__).parent.parent.joinpath('sqlite.db')}",
            engine_options=dict(
                connect_args={
                    "check_same_thread": False,
                },
                echo=False,
                future=True,
            ),
            async_engine=True,
        ),
    )

Command used: mypy --python-version 3.9 --no-incremental from the root repository directory.

Expected Behavior

No errors reported

Actual Behavior

Note that the src/bootstrap/config.py file is only 64 lines long

src/bootstrap/config.py:128: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:129: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:130: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:131: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:132: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:133: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:134: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:135: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:136: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:137: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:138: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:139: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:140: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:141: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:142: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:143: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:144: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:145: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:146: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:147: error: X | Y syntax for unions requires Python 3.10  [syntax]
src/bootstrap/config.py:148: error: X | Y syntax for unions requires Python 3.10  [syntax]
Found 21 errors in 1 file (checked 63 source files)

Your Environment

• Mypy version used: mypy 1.11.2 (compiled: yes)
• Mypy command-line flags: --python-version 3.9 (also --no-incremental to show line numbers but the error persists without)
• Mypy configuration options from mypy.ini (and other config files): from pyproject.toml

[tool.mypy]
files = ["src", "tests"]
exclude = ["alembic"]
plugins = "pydantic.mypy,strawberry.ext.mypy_plugin"

# We can remove celery by installing `celery-types` but needs
# at least python 3.10. For now we maintain the override.
[[tool.mypy.overrides]]
module = [
    "celery.*"
]
ignore_missing_imports = true

• Python version used: Python 3.11.9 (but experienced also directly on 3.9 and 3.12)
• Pydantic and Pydantic-settings versions used: pydantic==2.9.1 and pydantic-settings==2.5.3 - the issue didn't occur on pydantic==2.8.2 and pydantic-settings==2.3.4

[hramezani]

Thanks @febus982 for reporting this issue. we will try to fix the problem.

[user1584]

I get the same 'error: X | Y syntax for unions requires Python 3.10 [syntax]' for the following file:

import pydantic_settings


class Temp(pydantic_settings.BaseSettings):
    pass

I'm pretty sure there's no '|' in there. 😃
Some versions:
mypy=1.11.1
pydantic=2.8.2
pydantic-core=2.20.1
pydantic-settings=2.5.2

What's really strange is that I do not get the error in the CI-pipeline, which uses the exact same versions. I deleted the .mypy_cache folder multiple times but the error persists.

[quantori-pokidovea]

The error is related Python < 3.10, because the signature of BaseSettings.__init__ looks like this:

def __init__(
        __pydantic_self__,
        _case_sensitive: bool | None = None,
        _nested_model_default_partial_update: bool | None = None,
        _env_prefix: str | None = None,
        _env_file: DotenvType | None = ENV_FILE_SENTINEL,
        _env_file_encoding: str | None = None,
        _env_ignore_empty: bool | None = None,
        _env_nested_delimiter: str | None = None,
        _env_parse_none_str: str | None = None,
        _env_parse_enums: bool | None = None,
        _cli_prog_name: str | None = None,
        _cli_parse_args: bool | list[str] | tuple[str, ...] | None = None,
        _cli_settings_source: CliSettingsSource[Any] | None = None,
        _cli_parse_none_str: str | None = None,
        _cli_hide_none_type: bool | None = None,
        _cli_avoid_json: bool | None = None,
        _cli_enforce_required: bool | None = None,
        _cli_use_class_docs_for_groups: bool | None = None,
        _cli_exit_on_error: bool | None = None,
        _cli_prefix: str | None = None,
        _cli_implicit_flags: bool | None = None,
        _secrets_dir: PathType | None = None,
        **values: Any,
    ) -> None:

You need to maintain backward compatibility or make separate releases for each Python version

[alessandroberlati]

I temporarily patched by specifying:

[tool.mypy]
plugins = "pydantic.mypy"
# Suppress errors since pydantic_settings uses X | Y typing 
python_version = "3.10"

In the configuration.

[febus982]

I temporarily patched by specifying:

[tool.mypy]
plugins = "pydantic.mypy"
# Suppress errors since pydantic_settings uses X | Y typing 
python_version = "3.10"

In the configuration.

@alessandroberlati Note that this workaround cannot be applied to a python 3.9 codebase, or it would allow writing unsupported annotations in you code.

[redb0]

Are there any solutions other than installing version 3.10 or ignoring this error?

[febus982]

Are there any solutions other than installing version 3.10 or ignoring this error?

I don't think there is. IMHO if the package is distributed as compatible with python 3.9, then all the code should be python 3.9 compatible (annotations included) 🤷‍♂

[davnat]

The error is related Python < 3.10, because the signature of BaseSettings.__init__ looks like this:

def init(
pydantic_self,
_case_sensitive: bool | None = None,
_nested_model_default_partial_update: bool | None = None,
_env_prefix: str | None = None,
_env_file: DotenvType | None = ENV_FILE_SENTINEL,
_env_file_encoding: str | None = None,
_env_ignore_empty: bool | None = None,
_env_nested_delimiter: str | None = None,
_env_parse_none_str: str | None = None,
_env_parse_enums: bool | None = None,
_cli_prog_name: str | None = None,
_cli_parse_args: bool | list[str] | tuple[str, ...] | None = None,
_cli_settings_source: CliSettingsSource[Any] | None = None,
_cli_parse_none_str: str | None = None,
_cli_hide_none_type: bool | None = None,
_cli_avoid_json: bool | None = None,
_cli_enforce_required: bool | None = None,
_cli_use_class_docs_for_groups: bool | None = None,
_cli_exit_on_error: bool | None = None,
_cli_prefix: str | None = None,
_cli_implicit_flags: bool | None = None,
_secrets_dir: PathType | None = None,
**values: Any,
) -> None:

You need to maintain backward compatibility or make separate releases for each Python version

Indeed the error is related to Python < 3.10, but I'm on Python 3.8 and in my packages I have tens of dependencies that use this syntax: no one causes this problem: mypy usually only checks my files, which do not contain unsupported syntax, and no problems are reported except this one.
The mypy plugin must be doing something that triggers the issue.
By the way, I tried installing pydantic==2.8.2 and pydantic-settings==2.3.4 but the problem persists, so this is pretty blocking here.

Edited to add:
I also have a package that uses both Pydantic and the mypy plugin, and no errors are reported: maybe the culprit is pydantic-settings related?
Note that also BaseModel uses the | syntax, at least from v2.8.2, as I can see.

[davnat]

I seems that adding

from __future__ import annotations

at the top of the file solves the issue.