-
Notifications
You must be signed in to change notification settings - Fork 65
Add parallel front end goal in 25h2 #330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SparrowLii
wants to merge
2
commits into
rust-lang:main
Choose a base branch
from
SparrowLii:parallel-25h2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+76
−0
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
# Promoting Parallel Front End | ||
|
||
| Metadata | | | ||
| :--------------- | ------------------------------------------------- | | ||
| Point of contact | @SparrowLii | | ||
| Teams | <!-- TEAMS WITH ASKS --> | | ||
| Task owners | <!-- TASK OWNERS --> | | ||
| Status | Proposed | | ||
| Tracking issue | [https://github.com/rust-lang/rust/issues/113349] | | ||
| Zulip channel | [#t-compiler/wg-parallel-rustc][channel] | | ||
|
||
[channel]: https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fwg-parallel-rustc/ | ||
|
||
|
||
## Summary | ||
|
||
Continue with stabilization and performance improvements to parallel front-end, continuing from the [2025h1 goal](https://rust-lang.github.io/rust-project-goals/2025h1/parallel-front-end.html). | ||
|
||
## Motivation | ||
|
||
Verify and resolve the few remaining deadlock and other issues, stabilize the feature, and try to further improve parallel compilation performance through various means. | ||
|
||
### The status quo | ||
|
||
Parallel front end progressed well in the first half of the year. We resolved the deadlock issue caused by the deadlock handler, added the rustc-rayon dependency of the parallel front end to the rustc working tree, and changed the work-stealing scheduling algorithm to avoid deadlocks in parallel execution of rustc queries. | ||
|
||
Next we need to resolve and verify the remaining issue list, improve the parallel front end test suite, and include all known issues into the ui tests to ensure stability of the feature. | ||
|
||
Then we will start the feature stabilization process to push parallel front end into the stable version. | ||
|
||
In addition, there are already contributors working on improving the performance of the parallel front end, such as trying to parallelize the macro expansion process, analysing data contention in query calls, etc. (thank you very much). We will conduct more detailed analysis and implementation of these directions in the second half of the year. | ||
|
||
In addition, we need to document and enhance the support of surrounding tools for the parallel frontend, such as bootstrap, rustc-perf, and Cargo. | ||
|
||
### The next 6 months | ||
|
||
- Resolve the remaining issues in the issue [list](https://github.com/rust-lang/rust/issues?q=state%3Aopen%20label%3AA-parallel-compiler). Most of them have been solved, but are missing tests ensuring that the code won't regress again. | ||
- Land the parallel front end [test suite](https://github.com/rust-lang/rust/pull/132051) to ensure the robustness and prevent various issues from occurring again. | ||
- Enable parallel front end in bootstrap. | ||
- Continue to improve parallel compilation performance, by parallelizing macro expansion and reducing data contention. | ||
- Enable parallel front end in Cargo. | ||
- Add more benchmarks for the parallel front end to rustc-perf | ||
- Write stabilization report for the feature and submit it for compiler FCP | ||
|
||
### The "shiny future" we are working towards | ||
|
||
We will ensure robustness of the parallel front end and push it to stabilization. | ||
|
||
The current parallelization front end can already reduce the overall compilation time by 20~30+ percent, but we will continue to optimize it so that this number continues to grow. | ||
|
||
## Design axioms | ||
|
||
The parallel front end should be: | ||
- safe: Ensure the safe and correct execution of the compilation process | ||
- consistent: The compilation result should be consistent with that in single thread by default | ||
- maintainable: The implementation should be easy to maintain and extend, and not cause confusion to developers who are not familiar with it. | ||
|
||
[da]: ../about/design_axioms.md | ||
|
||
## Ownership and team asks | ||
|
||
**Owner:** @SparrowLii and Parallel Rustc WG own this goal | ||
|
||
| Task | Owner(s) or team(s) | Notes | | ||
| ---------------------------- | -------------------- | ----- | | ||
| Implementation | @SparrowLii | | | ||
| Author tests | @SparrowLii | | | ||
| Discussion and moral support | ![Team][] [compiler] | | | ||
|
||
## Frequently asked questions | ||
|
||
|
||
[ICE]: https://github.com/rust-lang/rust/issues?q=is%3Aopen+label%3AWG-compiler-parallel+ice | ||
[deadlock]: https://github.com/rust-lang/rust/issues?q=is%3Aopen+label%3AWG-compiler-parallel+deadlock | ||
[test]: https://github.com/rust-lang/rust/issues/118698 | ||
[issues]: https://github.com/rust-lang/rust/labels/WG-compiler-parallel |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably something like this