Try to avoid fuzz disk fullness
#3916
Conversation
|
👋 Thanks for assigning @joostjager as a reviewer! |
tnull
force-pushed
the
2025-07-avoid-fuzz-disk-fullness
branch
from
193e1bf to
cd91843
Compare
tnull
changed the title
fuzz disk fullnessfuzz disk fullness
tnull
force-pushed
the
2025-07-avoid-fuzz-disk-fullness
branch
from
cd91843 to
9fe30c8
Compare
| # Because we're fuzzing relatively few iterations, the maximum possible | ||
| # compiler optimizations aren't necessary, so switch to defaults. | ||
| sed -i 's/lto = true//' Cargo.toml | ||
| sed -i 's/codegen-units = 1//' Cargo.toml |
There was a problem hiding this comment.
Removing this is likely to slow the build step down substantially.
There was a problem hiding this comment.
Removing this is likely to slow the build step down substantially.
Which part do you mean exactly. As mentioned below, lto was never used as it conflicts with codegen-units = 1.
With the sed commands above, we would fallback to defaults in CI, meaning lto = false (see https://doc.rust-lang.org/cargo/reference/profiles.html#dev), and codegen-units = 16. Do we expect the latter to make that much of an impact?
There was a problem hiding this comment.
Yes, the build step takes a minute, and making it single-threaded is unlikely to help :). It may be okay because we can parallelize across binaries, but no reason to slow it down more.
There was a problem hiding this comment.
Hmm, I guess we'll have the benchmark for comparison in a bit. Let's see how long it will take.
There was a problem hiding this comment.
The fuzz job on this branch finished in ~4h 20m. Looking at recent examples of PRs where the CI used to pass, it did so in ~4h 15m. I think 5 min/2% difference is acceptable?
There was a problem hiding this comment.
Ah, 4h 13m this time around, so we even have some speedup compared to some recent PRs I looked at. Seems the variance in CI is larger than the impact of the actual change.
There was a problem hiding this comment.
Alright, yea, then the cross-binary parallelization dominates. Cool. Its less clear to me if this makes a difference on disk space usage, but it doesn't really matter.
| @@ -18,13 +18,10 @@ export RUSTFLAGS="--cfg=secp256k1_fuzz --cfg=hashes_fuzz" | |||
| mkdir -p hfuzz_workspace/full_stack_target/input | |||
| pushd write-seeds | |||
| RUSTFLAGS="$RUSTFLAGS --cfg=fuzzing" cargo run ../hfuzz_workspace/full_stack_target/input | |||
| cargo clean | |||
There was a problem hiding this comment.
Shouldn't we really not add a du in here to see what needs cleaning and what doesn't? Also to not unnecessarily increase build times.
And can the disk space in github be increased?
There was a problem hiding this comment.
Shouldn't we really not add a
duin here to see what needs cleaning and what doesn't? Also to not unnecessarily increase build times.
Huh, we only clean up here and below once were done? So should not have ~any overhead to clean target dirs that you aren't using anymore?
There was a problem hiding this comment.
I thought it would lead to re-downloads of packages. Not that would matter much even if it was the case with a 4h job...
.. to hopefully clear up some disk space in the process.
.. which marginally (~2%) increases the runtime while decreasing the disk usage.
tnull
force-pushed
the
2025-07-avoid-fuzz-disk-fullness
branch
from
9fe30c8 to
f6238fc
Compare
No description provided.