feat: freethreaded support for the builder API #3063
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
from
ce4dd59 to
594f026
Compare
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
4 times, most recently
from
bc8424b to
5e66b5b
Compare
rickeylev
reviewed
Jul 6, 2025
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Jul 21, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Jul 21, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Jul 21, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
3 times, most recently
from
b1720c1 to
57a4372
Compare
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Aug 3, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
from
57a4372 to
5fd3c86
Compare
Before this PR we would be constructing slightly different environments when the `env_marker_setting` is doing it in the analysis phase and when we are doing it in the repo phase due to how the defaults are handled. In this change we simply reuse the same select statements and add an extra helper that is allowing us to process that. Work towards bazel-contrib#2949 Prep for bazel-contrib#3058
This will be needed when we start selecting wheels entirely in the bzlmod extension evaluation phase (bazel-contrib#3058). This adds a few unit tests to just ensure that we conform to the spec even though the code is very simple. Work towards bazel-contrib#2747 Work towards bazel-contrib#2759 Work towards bazel-contrib#2849
DO NOT MERGE: stacked on bazel-contrib#3110 This PR only implements the selection algorithm where instead of selecting all wheels that are compatible with the set of target platforms, we select a single wheel that is most specialized for a particular *single* target platform. What is more, compared to the existing algorithm it does not assume a particular list of supported platforms and just fully implements the spec. Work towards bazel-contrib#2747 Work towards bazel-contrib#2759 Work towards bazel-contrib#2849
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Aug 9, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
from
5fd3c86 to
66c133d
Compare
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Aug 10, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
Before this PR the configuration for platforms would be built non-incrementally and was somewhat special, making it harder to override particular attributes of the already configured ones.
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
DO NOT MERGE: stacked on bazel-contrib#3058 This is a continuation of bazel-contrib#3058 where we define freethreaded platforms. They need to be used only for particular python versions so I included an extra marker configuration attribute where we are using pipstar marker evaluation before using the platform. I think this in general will be a useful tool to configure only particular platforms for particular python versions Work towards bazel-contrib#2548, since this shows how we can define custom platforms Work towards bazel-contrib#2747
aignas
force-pushed
the
exp/pypi-simplify-freethreaded
branch
from
66c133d to
040f77e
Compare
rickeylev
reviewed
Aug 16, 2025
| ), | ||
| "marker": attr.string( | ||
| doc = """\ | ||
| A marker which will be evaluated to disable the target platform for certain python versions. This |
There was a problem hiding this comment.
It wasn't clear to me what "marker" was. It was only after reading the PR description that it was clear it was an environment marker.
Can this be any expression? Is it evaluated as part of the bzlmod phase or analysis (pipstar) phase? Why would I set this instead of env or vice versa?
The docs say "to disable", so if this expression is true, then the platform will not match? That logic seems inverted from regular marker expressions. e.g normally foo; python_version=3.14 means "if the python version is 3.14, then include/use/accept foo". However, here it means the opposite: if the expression is true, then don't include/use/accept foo
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Aug 16, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Aug 16, 2025
Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support is done by defining extra freethreaded platforms using the new builder API (bazel-contrib#3063). This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the roll out. Work towards bazel-contrib#2747 Fixes bazel-contrib#2759 Fixes bazel-contrib#2849
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE: stacked on #3058
This is a continuation of #3058 where we define freethreaded platforms. They
need to be used only for particular python versions so I included an extra
marker configuration attribute where we are using pipstar marker evaluation
before using the platform.
I think this in general will be a useful tool to configure only particular
platforms for particular python versions
Work towards #2548, since this shows how we can define custom platforms
Work towards #2747