[AUTOGENERATED] [release/2.6] [rocm6.4_internal_testing][SWDEV-535305] Fixed test_extra_cuda_context in test_c10d_nccl.py and refactored is_navi3_arch function
#2381
+14
−5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…xt` in `test_c10d_nccl.py` and refactored is_navi3_arch function (#2341) In this PR, I have added a sleep statement before collectives. We need this extra sleep for NAVI_ARCH because rccl_init inside init_process_group is happening in a separate process and it is taking longer to finish on NAVI_ARCH. Sleeping here ensures that the init is competed successfully and mem_get_info can get stable numbers. Note that in the test the sleep statement was already there after collectives. Also, refactored is_navi3_arch function to is_arch that takes arch list as an argument and compares with existing arch. Tested with docker image- compute-artifactory.amd.com:5000/rocm-plus-docker/framework/compute-rocm-rel-6.4:108_ubuntu22.04_py3.10_pytorch_rocm6.4_internal_testing_aeb5d79
|
Jenkins build for 6f7c92e6bed5d46e7f536096e2ffa96777d7103c commit finished as FAILURE |
pragupta
requested changes
Jul 21, 2025
There was a problem hiding this comment.
@akashveramd - I am seeing this UT fail on MI350 as well. Lets remove the check for navi arch.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #2341