Online graph partitioning using Scotch #1339
Open
+256
−57
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
abishekg7
changed the title
mgduda
reviewed
Jul 2, 2025
|
|
||
| if (dminfo % my_proc_id == IO_NODE) then | ||
| useScotch = .false. | ||
| if ( trim(blockFilePrefix) == '' ) then | ||
| call mpas_log_write('Namelist option config_block_decomp_file_prefix is set to \'\' ', MPAS_LOG_ERR) |
There was a problem hiding this comment.
This line gives a compilation error with the ifx compiler:
mpas_block_decomp.F(119): error #5078: Unrecognized token '\' skipped
call mpas_log_write('Namelist option config_block_decomp_file_prefix is set to \'\' ', MPAS_LOG_ERR)
---------------------------------------------------------------------------------------------^
mgduda
reviewed
Jul 2, 2025
| @@ -759,6 +759,15 @@ endif | |||
| LIBS += $(NCLIB) | |||
| endif | |||
|
|
|||
| ifneq "$(SCOTCH)" "" | |||
| SCOTCH_FCINCLUDES += -I$(SCOTCH)/src/include | |||
| SCOTCH_LIBS += -L$(SCOTCH)/lib -lscotch -lscotcherr | |||
There was a problem hiding this comment.
It may depend on the version on Scotch, but when I installed v7.0.7, the header files are in .../include (rather than in .../src/include, and the libraries are in .../lib64 rather than in .../lib.
mgduda
reviewed
Jul 2, 2025
| @# | ||
| @# Create a Fortran test program that will link against the SCOTCH library | ||
| @# | ||
| $(info Checking for a working MUSICA-Fortran library...) |
There was a problem hiding this comment.
Should "MUSICA-Fortran" be "SCOTCH"?
mgduda
reviewed
Jul 2, 2025
| end do | ||
|
|
||
| if (istatus /= 0) then | ||
| call mpas_log_write('Writing out Scotch Graph paritioning data to '//trim(filename)) |
There was a problem hiding this comment.
Is filename uninitialized here? Or an empty string?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enables the use of MPAS atmosphere core without requiring any graph decomposition files specified as input, if the atmosphere core has been built with the Scotch graph partitioning library (built separately).
Building MPAS with Scotch
Instructions to build Scotch are provided later in the description. Once Scotch has been installed, the MPAS atmosphere core can be leveraged to use online graph partitioning by setting the path to Scotch prior to building MPAS.
If MPAS is built/linked with Scotch successfully, you should see a message stating that at the end of the build output.
Usage and run-time behavior
After building MPAS with Scotch, you can still choose whether or not to use online graph partitioning by setting appropriate values for the
config_block_decomp_file_prefixnamelist option.If
config_block_decomp_file_prefix+mpi_taskspoints to a valid graph decomp file that already exists in the run directory, then it is used to proceed with the model run without invoking Scotch.If
config_block_decomp_file_prefix==''orconfig_block_decomp_file_prefix+mpi_tasksdoes not match any valid graph decomp file in the run directory, then Scotch graph partitioning is invoked. During this process, the generated partition is saved as a graph decomp file to the run directory so that it may be reused in the following runs without needing to invoke Scotch again.If MPAS has not been built with scotch, any code paths relating to Scotch will not be taken. And an incorrect specification of
config_block_decomp_file_prefix+mpi_tasksshould lead to the model halting.Downloading and building Scotch
Scotch may be obtained from its gitlab repository or as tarballs from this link
Build instructions are on this page
More details to follow..