Skip to content

Depency group implementation does not correctly normalize group names #3539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
robsdedude opened this issue May 28, 2025 · 1 comment · Fixed by #3540
Closed

Depency group implementation does not correctly normalize group names #3539

robsdedude opened this issue May 28, 2025 · 1 comment · Fixed by #3540
Labels
bug:minor does not affect many people or has no big impact

Comments

@robsdedude
Copy link
Contributor

Issue

Consider this pyproject.toml

[project]
name = "foo"
version = "0.0.1"

[build-system]
requires = ["setuptools >= 77.0.3"]
build-backend = "setuptools.build_meta"

[dependency-groups]
test = [{include-group = "_pytest"}]
_pytest = ["pytest"]

[tool.tox]
requires = ["tox>=4.26"]
env_list = ["3.13"]

[tool.tox.env.test]
dependency_groups = ["test"]
commands = [
    ["unit", "python -m pytest -V"]
]

When running pip install --group test, pytest is successfully installed.

However, when trying to run tox -f test, tox errors out saying

test: failed with dependency group '-pytest' not found
test: FAIL code 1 (0.04 seconds)
evaluation failed :( (0.11 seconds)

This leads me to believe that group names are not always normalized in tox. However, looking at PEP-735 which states "These keys must be valid non-normalized names, and must be normalized before comparisons." and also looking at the reference implementation, it seems they really should be.

Environment

Provide at least:

  • OS: Linux Mint 21
Output of pip list of the host Python, where tox is installed 
Package       Version
------------- -------
cachetools    6.0.0
chardet       5.2.0
colorama      0.4.6
distlib       0.3.9
filelock      3.18.0
packaging     25.0
pip           24.3.1
platformdirs  4.3.8
pluggy        1.6.0
pyproject-api 1.9.1
tox           4.26.0
virtualenv    20.31.2

Output of running tox

Output of tox -rvv 
test: 229 W remove tox env folder /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test [tox/tox_env/api.py:333]
.pkg: 251 W remove tox env folder /home/xxxxxx/tmp/tox_dep_group_repro/.tox/.pkg [tox/tox_env/api.py:333]
test: 257 I find interpreter for spec PythonSpec(path=/home/xxxxxx/tmp/tox_dep_group_repro/.venv/bin/python) [virtualenv/discovery/builtin.py:76]
test: 258 D filesystem is case-sensitive [virtualenv/info.py:27]
test: 259 D got python info of /home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13 from /home/xxxxxx/.local/share/virtualenv/py_info/2/3d3183e8132c1820530140783e1de1b790bfeadf628f4e809c913da0ec231843.json [virtualenv/app_data/via_disk_folder.py:132]
test: 259 I proposed PythonInfo(spec=CPython3.13.1.final.0-64, system=/home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13, exe=/home/xxxxxx/tmp/tox_dep_group_repro/.venv/bin/python, platform=linux, version='3.13.1 (main, Jan 28 2025, 15:47:29) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
test: 259 D accepted PythonInfo(spec=CPython3.13.1.final.0-64, system=/home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13, exe=/home/xxxxxx/tmp/tox_dep_group_repro/.venv/bin/python, platform=linux, version='3.13.1 (main, Jan 28 2025, 15:47:29) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:85]
test: 287 I create virtual environment via CPython3Posix(dest=/home/xxxxxx/tmp/tox_dep_group_repro/.tox/test, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:52]
test: 287 D create folder /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/bin [virtualenv/util/path/_sync.py:14]
test: 287 D create folder /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages [virtualenv/util/path/_sync.py:14]
test: 287 D write /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
test: 288 D 	home = /home/xxxxxx/.pyenv/versions/3.13.1/bin [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	version_info = 3.13.1.final.0 [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	base-prefix = /home/xxxxxx/.pyenv/versions/3.13.1 [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	base-exec-prefix = /home/xxxxxx/.pyenv/versions/3.13.1 [virtualenv/create/pyenv_cfg.py:40]
test: 288 D 	base-executable = /home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13 [virtualenv/create/pyenv_cfg.py:40]
test: 288 D symlink /home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13 to /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/bin/python [virtualenv/util/path/_sync.py:34]
test: 289 D create virtualenv import hook file /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:93]
test: 289 D create /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:96]
test: 290 D ============================== target debug ============================== [virtualenv/run/session.py:54]
test: 290 D debug via /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/bin/python /home/xxxxxx/tmp/tox_dep_group_repro/.venv/lib/python3.13/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:215]
test: 290 D {
  "sys": {
    "executable": "/home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/bin/python",
    "_base_executable": "/home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13",
    "prefix": "/home/xxxxxx/tmp/tox_dep_group_repro/.tox/test",
    "base_prefix": "/home/xxxxxx/.pyenv/versions/3.13.1",
    "real_prefix": null,
    "exec_prefix": "/home/xxxxxx/tmp/tox_dep_group_repro/.tox/test",
    "base_exec_prefix": "/home/xxxxxx/.pyenv/versions/3.13.1",
    "path": [
      "/home/xxxxxx/.pyenv/versions/3.13.1/lib/python313.zip",
      "/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13",
      "/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13/lib-dynload",
      "/home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.13.1 (main, Jan 28 2025, 15:47:29) [GCC 11.4.0]",
  "makefile_filename": "/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13/config-3.13-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' (frozen)>",
  "site": "<module 'site' (frozen)>",
  "datetime": "<module 'datetime' from '/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13/datetime.py'>",
  "math": "<module 'math' from '/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13/lib-dynload/math.cpython-313-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/home/xxxxxx/.pyenv/versions/3.13.1/lib/python3.13/json/__init__.py'>"
} [virtualenv/run/session.py:55]
test: 324 I add seed packages via FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/home/xxxxxx/.local/share/virtualenv) [virtualenv/run/session.py:59]
test: 325 D got embed update of distribution pip from /home/xxxxxx/.local/share/virtualenv/wheel/3.13/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:132]
test: 328 D install pip from wheel /home/xxxxxx/tmp/tox_dep_group_repro/.venv/lib/python3.13/site-packages/virtualenv/seed/wheels/embed/pip-25.1.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:51]
test: 329 D copy directory /home/xxxxxx/.local/share/virtualenv/wheel/3.13/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip to /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages/pip [virtualenv/util/path/_sync.py:42]
test: 419 D copy /home/xxxxxx/.local/share/virtualenv/wheel/3.13/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.virtualenv to /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages/pip-25.1.1.virtualenv [virtualenv/util/path/_sync.py:42]
test: 419 D copy directory /home/xxxxxx/.local/share/virtualenv/wheel/3.13/image/1/CopyPipInstall/pip-25.1.1-py3-none-any/pip-25.1.1.dist-info to /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/lib/python3.13/site-packages/pip-25.1.1.dist-info [virtualenv/util/path/_sync.py:42]
test: 423 D generated console scripts pip-3.13 pip3.13 pip3 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:45]
test: 423 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:65]
test: 427 D write /home/xxxxxx/tmp/tox_dep_group_repro/.tox/test/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:35]
test: 427 D 	home = /home/xxxxxx/.pyenv/versions/3.13.1/bin [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	version_info = 3.13.1.final.0 [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	virtualenv = 20.31.2 [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	base-prefix = /home/xxxxxx/.pyenv/versions/3.13.1 [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	base-exec-prefix = /home/xxxxxx/.pyenv/versions/3.13.1 [virtualenv/create/pyenv_cfg.py:40]
test: 427 D 	base-executable = /home/xxxxxx/.pyenv/versions/3.13.1/bin/python3.13 [virtualenv/create/pyenv_cfg.py:40]
test: 428 E failed with dependency group '-pytest' not found [tox/session/cmd/run/single.py:57]
  test: FAIL code 1 (0.20 seconds)
  evaluation failed :( (0.28 seconds)
@robsdedude
Copy link
Contributor Author

I just realized, that the group _pytest is not actually a valid name

https://packaging.python.org/en/latest/specifications/name-normalization/#name-format states

A valid name consists only of ASCII letters and numbers, period, underscore and hyphen. It must start and end with a letter or number. [...] with re.IGNORECASE
^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$

But that reproducer also fails with valid names such as dep_pytest.

@robsdedude robsdedude mentioned this issue Jun 1, 2025
Merged
5 tasks
@ssbarnea ssbarnea added the bug:minor does not affect many people or has no big impact label Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug:minor does not affect many people or has no big impact
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix dependency-group name normalization robsdedude/tox
2 participants
@ssbarnea @robsdedude