Skip to content

Conversation

bobby-stripe
Copy link

One of the great things you can do with netlink is query for detailed stats on all open TCP connections, including e.g. retransmits, how much data is in socket buffers, the computed BBR bandwidth estimates, etc. Unfortunately, if you ask the kernel for this info on a host with a lot of connections, the []netlink.Message slice returned by netlink/Conn.Execute can take 50+ MB of space, when the slice will be (likely) be thrown away after its iterated over once.

This PR adds a new method ExecuteFunc where a user can pass in a callback function that is called once per message. Becuase of how Conn.receive/the netlink recvmsg API works in batches/streaming, this dramatically reduces the peak RSS/memory overhead required to process a large stream of netlink message responses.

To try to minimize duplication, I refactored receive into receiveEach which operates on a callback, changed lockedReceive to call receiveEach, and then added a new lockedReceiveEach which is callled by ExecuteFunc (I started with the Each suffix, but then saw that a bunch of the callback variants in the Go stdlib have a Func suffix. LMK if you prefer one or the other or something else and I can standardize the naming).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant