ZLUDA notes

[druggedhippo]

Just wanted to share my ZLUDA notes on my messing with this today.

My video card is an AMD 6700XT 12GB and I'm installing on Windows 11. I'm using SD1.5 models and EulerA sampler.

Installation

• Make sure you are using the correct Python version. There are so many, but automatic needs particular versions (because torch does)
• If you do an install with DirectML the installer.py will not be able to switch to zluda even with --reinstall. This is because function is_rocm_available returns early if it detects DirectML installed and refuses to even detect rocm./hip. You need to uninstall torch-directml FIRST before trying reinstall with ZLUDA enabled.
• If you get an error about caffe2_nvrtc.dll make sure you installed HIP 5.7
• Reinstalling takes a long time, try to avoid it
• The installer and run will fail if you are attempt to run it from a symlinked directory as it detects the path difference and errors out.

Running

• Enable DeepCache, GPU generator - about 5it/s. Damn, that's nice.
• By contrast, stable-diffusion-webui-directml in DirectML mode gets around 1.5it/s
• You NEED to shutdown the server if you change any of the above otherwise you WILL get errors. Don't just restart, it has to be a full shutdown or you will get all sorts of errors. In fact, I've found that in general you should do this after any system config change, it's just too volatile otherwise.
• With those settings above, you can't change model without needing a full shutdown or you get errors like KeyError: 'unet_forward'. or AttributeError: 'DeepCacheSDHelper' object has no attribute 'function_dict'
• Curiously Windows Task Manager shows 1-4% GPU usage during diffusion.... But AMD Catalyst performance monitor shows usage up at 99% during diffusion..

[brknsoul]

By contrast, stable-diffusion-webui-directml in DirectML mode gets around 1.5it/s

You really should be testing SD.Next DirectML versus SD.Next ZLUDA, not A1111.

will not be able to switch to zluda even with --reinstall

That's interesting. Looks like I might have to add that section back to the wiki. (You'll also need to uninstall torch, since Zluda uses a newer version).

Curiously Windows Task Manager shows 1-4% GPU usage during diffusion.... But AMD Catalyst performance monitor shows usage up at 99% during diffusion..

If you switch one of the GPU graphs to Compute 1, it'll show GPU usage;

[freggggg]

I am getting the affe2_nvrtc.dll error you described, despite having HIP 5.7 installed.

OSError: [WinError 126] The specified module could not be found. Error loading "C:\Opera Downloads\zluvlad\automatic\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" or one of its dependencies.

[druggedhippo]

If you have HIP 5.7 installed, make sure you restart your computer. The installer should have put in the path to the bin directory into your environment, if not you might need to add it yourself.

If it's still failing it's usually because there is some DLL dependency chain windows failed to determine. Now is the part where a random internet person tells you to download and run some program:

1. Download and extract this: https://www.dependencywalker.com/ (make sure it's the x64 version) ( virustotal results - https://www.virustotal.com/gui/search/57c483dc985a9757501993e969c2a7043c26517f97fd49a42b33d2d6a4193d8b )
2. Run it and click File-> Open and load the "C:\Opera Downloads\zluvlad\automatic\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" file into it (this might take a while)
3. Look at the files on the left, the program has figured out all the "dependent" dlls. You want to try to find the one it's failing to load, it should have a big red icon next to it (ignore the yellow ones). Ignore anything past a few levels in the tree. Ignore anything complaining about API-MS-WIN
4. Work out which DLL is failing to load.

You want to see something like this where the first few levels of DLLs are fine:

You do NOT want to see this where something like hiprtc0507.dll has failed:

[freggggg]

Thanks for pointing me the right direction! I fixed the PATH issue of HIP SDK with setx PATH "%PATH%;C:\Program Files\AMD\ROCm\5.7\bin". Then verified it with 'echo %PATH%'.

Webui.bat now correctly finds and uses HIP SDK and gives no errors.

Ive also installed your spyware (its only polite after you've helped me), and it indicates missing NVCUDA.DLL. Despite that i can render images and i can get significantly higher res than directml.

now im off to find some optimizations and see how much i can get out of ZLUDA.

Edit: 512x512 is roughly 15x faster on ZLUDA compared to directml on my RX 7600, wtf

[grantterra]

If you have HIP 5.7 installed, make sure you restart your computer. The installer should have put in the path to the bin directory into your environment, if not you might need to add it yourself.

If it's still failing it's usually because there is some DLL dependency chain windows failed to determine. Now is the part where a random internet person tells you to download and run some program:

1. Download and extract this: https://www.dependencywalker.com/ (make sure it's the x64 version) ( virustotal results - https://www.virustotal.com/gui/search/57c483dc985a9757501993e969c2a7043c26517f97fd49a42b33d2d6a4193d8b )

2. Run it and click File-> Open and load the  "C:\Opera Downloads\zluvlad\automatic\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" file into it  (this might take a while)

3. Look at the files on the left, the program has figured out all the "dependent" dlls. You want to try to find the one it's failing to load, it should have a big red icon next to it (ignore the yellow ones). Ignore anything past a few levels in the tree. Ignore anything complaining about API-MS-WIN

4. Work out which DLL is failing to load.

You want to see something like this where the first few levels of DLLs are fine:

You do NOT want to see this where something like hiprtc0507.dll has failed:

you saved my day!

[ArthurZhou]

Thanks for pointing me the right direction! I fixed the PATH issue of HIP SDK with setx PATH "%PATH%;C:\Program Files\AMD\ROCm\5.7\bin". Then verified it with 'echo %PATH%'.

Webui.bat now correctly finds and uses HIP SDK and gives no errors.

Ive also installed your spyware (its only polite after you've helped me), and it indicates missing NVCUDA.DLL. Despite that i can render images and i can get significantly higher res than directml.

now im off to find some optimizations and see how much i can get out of ZLUDA.

Edit: 512x512 is roughly 15x faster on ZLUDA compared to directml on my RX 7600, wtf

For nvcuda.dll errors, try this:

navigate to zluda_installer.py, and locate the following code: (in def load() near line 120)

core = Core(ctypes.windll.LoadLibrary(os.path.join(path, 'nvcuda.dll')))
ml = ZLUDALibrary(ctypes.windll.LoadLibrary(os.path.join(path, 'nvml.dll')))

replace them with the following:

 nvcuda_path = os.path.join(path, 'nvcuda.dll')
    nvml_path = os.path.join(path, 'nvml.dll')

    # winmode=0 is the fix
    core = Core(ctypes.CDLL(nvcuda_path, winmode=0))
    ml = ZLUDALibrary(ctypes.CDLL(nvml_path, winmode=0))

this should fix this problem

[pinea00]

Although the A1111 version of zluda is not as fast as deep-cache, it is relatively stable and does not frequently crash or produce low-quality images.
I think about that i should comeback use A1111

[vladmandic]

there may be ppl that got it working, but a1111 on its own does not have zluda support at all.

[Epyon01P]

[brknsoul]

[Soulreaver90]

Your results match my results and I also have a 6700xt.

Windows DirectML - 1.5-2 it/s
Windows ZLUDA - ~5 it/s
Linux ROCM - ~6.5 it/s
Linux Deep-cache - ~15 it/s

[Dmytriy-hub]

what model was used?