-
Notifications
You must be signed in to change notification settings - Fork 11
use sync.OnceValue for various regular expressions, require go1.21 #15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
27e92a1
to
4ca1403
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #15 +/- ##
==========================================
+ Coverage 83.71% 84.19% +0.48%
==========================================
Files 5 5
Lines 393 405 +12
==========================================
+ Hits 329 341 +12
Misses 54 54
Partials 10 10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
/cc @tonistiigi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is interesting. Out of curiosity. what triggered this change? Who has discovered/highlighted the memory pressure issue?
Yes; regexes being "hungry" on resources is a known issue in general, which is also why Kir made various pull requests some time ago to try to reduce their use (e.g. docker/go-units#40), but this one cam through @tonistiigi who found that For a long-lived process / daemon, it's probably less problematic, but in the CLI, this resulted in nearly 2MB to be used just by importing those packages; here's from a Slack thread on that;
So this is an attempt at reducing such cases, and making the regexes compiled on first use. |
@thaJeztah now that #16 has been merged, wanna open this for review? |
ping @thaJeztah |
.github/workflows/test.yml
Outdated
@@ -17,15 +17,15 @@ jobs: | |||
build: | |||
strategy: | |||
matrix: | |||
go-version: [1.20.x, 1.21.x] | |||
go-version: [1.21.x, 1.22.x] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should change this to 1.22.x
and 1.23.x
We need to bump Go on this repo 🙈 |
@milosgajdos failure was because an invalid |
Using regex.MustCompile consumes a significant amount of memory when importing the package, even if those regular expressions are not used. This changes compiling the regular expressions to use a sync.OnceValue so that they're only compiled the first time they're used. There are various regular expressions remaining that are still compiled on import, but these are exported, so changing them to a sync.OnceValue would be a breaking change; we can still decide to do so, but leaving that for a follow-up. It's worth noting that sync.OnceValue requires go1.21 or up, so raising the minimum version accordingly. Signed-off-by: Sebastiaan van Stijn <[email protected]>
use sync.OnceValue for various regular expressions, require go1.21
Using regex.MustCompile consumes a significant amount of memory when
importing the package, even if those regular expressions are not used.
This changes compiling the regular expressions to use a sync.OnceValue
so that they're only compiled the first time they're used.
There are various regular expressions remaining that are still compiled
on import, but these are exported, so changing them to a sync.OnceValue
would be a breaking change; we can still decide to do so, but leaving
that for a follow-up.
It's worth noting that sync.OnceValue requires go1.21 or up, so raising
the minimum version accordingly.
Before / After (on the docker CLI (
GODEBUG=inittrace=1 ./build/docker
)):