Skip to content

Moves balancing code out of Manager class #5537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

keith-turner
Copy link
Contributor

@keith-turner keith-turner commented May 6, 2025

The balancing code needs some improvements. Currently the code is entangled in the manager code in odd places (like some of the balancing code is un the Manager.StatusThread class). This change moves the balancing code to its own class w/o making any changes to the code (except fixing two small existing bugs in shouldCleanupMigration()).

Making this change as a first step in making the balancer more independent of other manager code and threads.

The balancing code needs some improvements.  Currently the code is
entangled in the manager code in odd places (like under some of the
balancing code is un the Manager.StatusThread class).  This change moves
the balancing code to its own class w/o making any changes to the code
(except fixing two small existing bugs in shouldCleanupMigration()).

Making this change as a first step in making the balancer more
independent of other manager code and threads.
Copy link
Contributor

@dlmarion dlmarion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should run ScanServerUpgrade11to12TestIT.testScanRefTableCreation with this change locally. It was hanging trying to scan the root and metadata tables because they were not hosted.

@keith-turner
Copy link
Contributor Author

You should run ScanServerUpgrade11to12TestIT.testScanRefTableCreation with this change locally. It was hanging trying to scan the root and metadata tables because they were not hosted.

I think that is the problem that needs the balancer to run in a separate thread. With this change was just attempting to clean up some tech debt in the way the balancer code is organized before making the threading changes. Hoping we can make a change to spin up three balancer threads and remove balancing from the status thread after this change.

@ctubbsii
Copy link
Member

You should run ScanServerUpgrade11to12TestIT.testScanRefTableCreation with this change locally. It was hanging trying to scan the root and metadata tables because they were not hosted.

I think that is the problem that needs the balancer to run in a separate thread. With this change was just attempting to clean up some tech debt in the way the balancer code is organized before making the threading changes. Hoping we can make a change to spin up three balancer threads and remove balancing from the status thread after this change.

I created #5577 to create a new IT to test this, because ScanServerUpgrade11to12TestIT.testScanRefTableCreation should be removed. I think it should be possible to create a reproducer test for that, which does not rely on removing a built-in table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants