-
Notifications
You must be signed in to change notification settings - Fork 72
[release/2.6] NAVI32 specific fixes #2450
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Jenkins build for 21df250d84ef981cee62f1b519184c7c48ab700e commit finished as FAILURE |
Jenkins build for fe52793e8230c26e19cdc80b6bd78abab3138753 commit finished as FAILURE |
!cherry-pick --onto release/2.7 |
Hello @jithunnair-amd I found out that we run out of memory on navi32 on some tests with bigger tensors during benchmarking. No amount of tricks or hacks were able to fix this since the tests were inherently designed for bigger GPUs. We decided to skip the tests not based on arch, but based on available memory. This should avoid any issues with other smaller cards later down the line, we will probably need to upstream this when we have navi CI. Thus I sadly decided to add another ROCm decorator to avoid hitting Nvidia cards and make the logic more readable. cc @jataylo |
@iupaikov-amd @jataylo Can we please file an upstream PR for this anyway? We are still in process of getting Navi CI in upstream, but that shouldn't prevent us from filing the PR at least. And you can use that PR to consolidate all Navi-related inductor fixes. |
!cherry-pick --onto release/2.7 release/2.8 |
Created branch autogenerated/release/2.7_cherry-pick_pr-2450 and #2465. It contains a merge conflict. Please resolve it |
Created branch autogenerated/release/2.7_cherry-pick_pr-2450 and #2466. It contains a merge conflict. Please resolve it Created branch autogenerated/release/2.8_cherry-pick_pr-2450 and #2467. It contains a merge conflict. Please resolve it |
!cherry-pick --onto rocm7.1_internal_testing |
Created branch autogenerated/rocm7.1_internal_testing_cherry-pick_pr-2450 and #2473. It contains a merge conflict. Please resolve it |
Cherry-pick of #2450 --------- Co-authored-by: iupaikov-amd <[email protected]>
Fixes https://github.com/ROCm/frameworks-internal/issues/12096
Cherry-picked to release/2.7 branch via #2465
Cherry-picked to release/2.7 branch via #2466
Cherry-picked to release/2.8 branch via #2467
Cherry-picked to rocm7.1_internal_testing branch via #2473