Add support for passing config and predefined configs to VM re-execution tests #4180
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…o work w/ github action benchmark
Base automatically changed from
aaronbuchwald/archive-reexecute-post-state
to
master
August 19, 2025 17:41
RodrigoVillar
approved these changes
Aug 20, 2025
maru-ava
reviewed
Aug 26, 2025
maru-ava
reviewed
Aug 26, 2025
Comment on lines
87
to
91
| run: | | ||
| # xz-utils might be present on some containers. Install if not present. | ||
| if ! command -v xz &> /dev/null; then | ||
| sudo apt-get update | ||
| sudo apt-get install -y xz-utils |
There was a problem hiding this comment.
Please strip trailing whitespace on these lines too.
There was a problem hiding this comment.
Addressed, sorry didn't realize the previous comment was intended multi-line
maru-ava
approved these changes
Aug 26, 2025
This was referenced Aug 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why this should be merged
This PR supports passing config bytes in to the VM for re-execution tests. This is one of the features outlined in #4108.
Users specify a predefined config to use when running the benchmark. The currently supported options are "defaultt"/"", "archive", and "firewood". This does not support passing arbitrary JSON atm because the name of the config is used as part of the Benchmark run name and supplied as a Prometheus label, which means we need a simple string and don't want to convert from JSON. If desired, we could add flags to provide both a full arbitrary JSON config and a name to use as the corresponding parameter.
You can see an example of how to add a new pre-defined config here: 99e9d48.
Note: this PR also migrates from using ARC to Blacksmith because running on limited infrastructure on ARC with a larger volume of jobs results in a much higher queueing delay with our current configuration.
This PR also reduces the execution range of the scheduled job because it is currently failing at the hour mark seemingly hitting some timeout at this point.
The README has been updated to include instructions on using the new
CONFIGparameter, a section on adding a new entry to the schedule/PR matrix of runs, and using the GH CLI to trigger a run via manual dispatch:https://github.com/ava-labs/avalanchego/blob/aaronbuchwald/reexecute-benchmark-configs/tests/reexecute/c/README.md
How this works
This adds a new flag to the benchmark execution tests that will be parsed as either the name of a pre-defined config or a string to interpret as the JSON config.
How this was tested
This was tested by adding each of the config options to run the range [101, 250k] on every PR and the schedule job here: https://github.com/ava-labs/avalanchego/actions/runs/17104240451/job/48508879094?pr=4180
Manual workflows as documented in the README to run a single job using GH CLI:
Tested PR triggers via CI, which are a mirror of the schedule run:
Need to be documented in RELEASES.md?
No