Skip to content

Add collections._tuplegetter support #19479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

Gatsik
Copy link

@Gatsik Gatsik commented Jul 19, 2025

Fixes #19414

Add TupleGetterType to describe NamedTuple's class variables

I could get away with some NotImplementedError's, and this is most likely because tests are not exhaustive, but I have no clue which use cases will trigger those methods

This comment has been minimized.

Copy link
Member

@brianschubert brianschubert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

There should be no need to add a new type construct; descriptors like this can already be modeled by Instance.

I think it would be sufficient to make the field types instances of a collections._tuplegetter descriptor. Then mypy's existing descriptor handling will take care of the rest. The caveat is that collections._tuplegetter doesn't exist in the typeshed stubs, so someone would need to add it first. If you'd be interested in doing that, the functools.cached_property stubs work pretty similarly and can be used as a reference. Once collections._tuplegetter exists, the mypy implementation should be as simple as using self.api.named_type("collections._tuplegetter", [typ]) for the type.

in favour of `api.named_type("collections._tuplegetter")`

This comment has been minimized.

This comment has been minimized.

[builtins fixtures/dict.pyi]
[builtins fixtures/tuple.pyi]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of replacing these fixtures and expanding fixtures/tuple.pyi, I'd suggest leaving them as-is and adding import collections to the fixtures that are used in named tuple tests. That should help reduce the diff

Comment on lines 27 to 37
class _tuplegetter(Generic[KT]):
if sys.version_info >= (3, 10):
@overload
def __get__(self, instance: None, owner: type[Any] | None = None) -> Self: ...
@overload
def __get__(self, instance: object, owner: type[Any] | None = None) -> KT: ...
else:
@overload
def __get__(self, instance: None, owner: type[Any] | None) -> Self: ...
@overload
def __get__(self, instance: object, owner: type[Any] | None) -> KT: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to be too detailed for the fixture. This should suffice, and will avoid the issues with introducing sys and Self:

class _tuplegetter(Generic[T]):
    @overload
    def __get__(self, instance: None, owner: type[Any]) -> _tuplegetter[T]: ...
    @overload
    def __get__(self, instance: object, owner: type[Any]) -> T: ...

@@ -2097,7 +2097,7 @@ class Y(TypedDict):
@collect_cases
def test_named_tuple(self) -> Iterator[Case]:
yield Case(
stub="from typing import NamedTuple",
stub="from typing import NamedTuple; import collections",
Copy link
Member

@brianschubert brianschubert Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead, let's add import collections to stubtest_typing_stub

@Gatsik
Copy link
Author

Gatsik commented Jul 22, 2025

There are some fixtures which break typegen tests having collections imported, that's why I initially decided to use tuple.pyi everywhere -- to stick to 1 approach.

But as you want to reduce diff 4 different solutions appear:

  1. Import collections to some fixtures
  2. Replace some fixtures from whatever to tuple.pyi and expand tuple.pyi
  3. Create additional fixtures for some tests -- this is the case of list.pyi -- importing collections broke less typegen cases than not importing breaks namedtuple cases, so I decided to create a simpler version of list.pyi for typegen tests and replace less fixtures from list.pyi to list-simple.pyi rather than use the second approach for more cases
  4. Modify broken typegen tests and add EllipsisExpr: ... (I don't remember actual output fully) to them, but it still modifies the same amount of typegen tests as in 3 but with bigger and meaningless diff, because EllipsisExpr are probably not what supposed to be tested in them

I hope this would be an acceptable compromise

This comment has been minimized.

@brianschubert
Copy link
Member

brianschubert commented Jul 22, 2025

Thanks, that's a much smaller diff! Your approach for the fixtures looks good. The important thing is that it's easy for a maintainer to see how the tests have changed. A smaller diff and less changes in the actual .test files helps with that :-)

To get the CI to run, you could try applying python/typeshed#14436 to mypy's vendored copy of typeshed at mypy/typeshed. We may or may not have to revert that before this PR can be merged, but for now it would be helpful to see if the tests run green and if there are any affected primer projects.

(also, can you update the PR title?)

@Gatsik Gatsik changed the title Add TupleGetterType type Add collections._tuplegetter support Jul 22, 2025
@brianschubert
Copy link
Member

For the mypyc test failures, try adding import collections to mypyc/test-data/fixtures/typing-full.pyi, and add the fixture [typing fixtures/typing-full.pyi] to the failing cases that don't already have it.

This comment has been minimized.

Copy link
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NamedTuple's _tuplegetter is not recognized
2 participants