Explore: Alter handling of free batches. #728
Description
In #724, we added a limit to how many remote frees could be processed in one go. This should improve general performance of batched frees.
As part of this PR, there was a discussion about taking this further. This issue is raised to not lose that discussion.
The key ideas were
- Process varying batch sizes based on how slow a path we are on.
- Split the remote message queues by sizeclass
- This would allow us to quickly find messages containing the sizes we are using at the moment.
- When we are about to use the backend, process everything to see if we can consolidate existing memory to serve the request.
I am sure there is much more to discuss here.