[bug] Plugins can only inject envs as default but not additional

[webknjaz]

Over the course of having discoveries related to #3507, I realized that all the examples with injecting MemoryLoader(env_list=env_list) add envs as default.

And so typing in tox would attempt running them:

$ tox l
default environments:
pip-compile              -> [tox-lock] Invoke pip-compile of pip-tools
pip-compile-build-lock   -> [tox-lock] Produce a PEP 517/660 build deps lock
pip-compile-tox-env-lock -> [tox-lock] Produce a lock file for the passed tox env using current python

additional environments:
build-dists              -> Build dists and put them into the dist/ folder
pre-commit               -> Run the quality checks under python3; run as `SKIP=check-id1,check-id2 tox r -e pre-commit` to instruct the underlying `pre-commit` invocation avoid running said checks; Use `tox r -e pre-commit -- check-id1 --all-files` to select checks matching IDs aliases: `tox r -e pre-commit -- mypy --all-files` will run 3 MyPy invocations, but `tox r -e pre-commit -- mypy-py313 --all-files` runs one.
build-docs               -> Build The Docs

I don't want to influence whatever the end-user configures as env_list in their config, only providing additional envs that should be requested via tox r -e <env> (or through the env var, or config explicitly).

It seems to me like a gap in the plugin API. Having stared long enough at the source code, it seems that the discovery of additional envs is tightly coupled with the source objects. It's effectively happening @

tox/src/tox/session/env_select.py

Line 180 in a985171

yield self._state.conf, everything_active

.
And if we unwrap a few layers of indirection, that effectively does state.conf._src._discover_tox_envs(state.conf.core).

It looks like this currently means that adding a memory loader into the core config adds default envs that are processed separately when discovering all envs:

tox/src/tox/session/env_select.py

Line 172 in a985171

env_list, everything_active = self._state.conf.core["env_list"], False

. And this is what makes it work.
But adding “inactive” / “additional” envs would only be possible by monkey-patching the private state.conf._src object or state.conf.__iter__().

@gaborbernat should the plugin API just implement a tox_discover_envs() hook or a method on the state or config objects for declaring additional environments?

[webknjaz]

@gaborbernat any idea what could be done here?

[gaborbernat]

Don't know. This would require more in depth thought, which I did not get to yet. In an ideal world we'd have on loaders something like extend, but then we'd need to come up with merge strategies that's not simple. So don't know what's a good design here.

[webknjaz]

My feeling is that if only there was a way to extend the lists of env names in the core config hook, then those envs could be populated in the env config hooks as it's possible already.

[gaborbernat]

That seems odly specific to cover just the env_list and not all the other core setttings. I'm trying to stir away from monkey patching individual configurations and instead come up with solutions that work the same way for all configurations.

[webknjaz]

But this is specifically about adding envs. What are the other settings that influence this?

[gaborbernat]

I don't think it is. This is about extending a core configuration and not overriding it. env_list is not special is just one of the core configurations. You're proposing it to make it special.

[webknjaz]

Not really, I'm looking for a way to avoid using/referencing env_list. And the internal logic in tox emits envs based on the combination of env list and config file sections. It seems reasonable to add another source for extending that list.

[webknjaz]

Actually, looking at

tox/src/tox/session/env_select.py

Lines 181 to 183 in a985171

	label_envs = dict.fromkeys(chain.from_iterable(self._state.conf.core["labels"].values()))
	if label_envs:
	yield label_envs.keys(), False

, there may be a way to hack it to inject extra env names in there, it seems. I'll need to play with it...

[webknjaz]

Ugh… Nope. It's failing on the further validation, then.

[gaborbernat]

Not really, I'm looking for a way to avoid using/referencing env_list. And the internal logic in tox emits envs based on the combination of env list and config file sections. It seems reasonable to add another source for extending that list.

Yes, but these new environments are a side effect of the core configuration. I am not saying it's not reasonable to extend it. I'm saying that we shouldn't come up with a solution for just this, but rather a generic solution for all core configurations.

[webknjaz]

Could you expand what you mean by all core config, then? Perhaps, I'm not scoping the terminology the same way...

[gaborbernat]

self._state.conf.core["env_list"] showing that that's a core configuration, not some special field 🤔

[webknjaz]

@gaborbernat is that basically what's in the [tox] section of tox.ini? Are we talking about ways of extending it? My understanding was that what I'm looking for would be set somewhere outside that section.. Likely, after parsing it and the list of env sections. This way, a plugin author could verify that they aren't declaring existing env names again.

[webknjaz]

Possible signature:

@impl
def tox_extend_envs() -> Iterable[str]:

(options: Options could be an arg but maybe not)

Hook point: https://github.com/tox-dev/tox/blob/b244a59/src/tox/session/state.py#L21.

This must be happening after https://github.com/tox-dev/tox/blob/b244a59/src/tox/config/cli/parse.py#L62.