Didn't work

[aghyad-alghazawi]

I tried following this and i installed the flash attention but then when i run comfy it says torch not compiled with cuda

[dicksondickson]

This seems to be a pytorch issue. Its something the pytorch devs needs to fix. If you import flash attention into python it works.

from flash_attn import flash_attn_qkpacked_func, flash_attn_func

[aghyad-alghazawi]

yeah but comfy doesnt work at all when it says that

i reinstall torch and then it says torch not compiled with flash atten

[dicksondickson]

Dude, like I said, this seems to be a pytorch issue. Its something the pytorch devs needs to fix. Its something with how pytorch calls flash-attn / sdp attention.

[dicksondickson]

Comfy doesn't require flash attention. If your comfy is not working, then there is something wrong with your installation.

[dicksondickson]

torch not compiled with cuda

Do you have an Nvidia GPU?

[aghyad-alghazawi]

i think u didnt get the point.. im saying that if i install flash attention comfy doesnt start and says torch not compiled with cuda and if i install torch normally it says torch not compiled with flash attention.. rn i switched back to that so that at least it works

[dicksondickson]

How are you installing torch? Do you have a Nvidia GPU?

[aghyad-alghazawi]

yes i have it

[Donzox]

I had plenty of issues with flash attention. Finally got it to work but on linux not windows.

[dicksondickson]

@Donzox I've been encountering the same issues as you and from what I found, others have as well. It seems flash-attn is absolutely a no-go on windows.

[Donzox]

@Donzox I've been encountering the same issues as you and from what I found, others have as well. It seems flash-attn is absolutely a no-go on windows.

Seeing people posts about how comfyui works better and faster on linux made me try linux for the first time. I just tested the speeds of linux with flash attention vs windows and found that they were identical. I believe there has been a lot of improvements made for windows in the last month plus someone created a fork of triton to have support on windows https://github.com/woct0rdho/triton. From what I read on reddit, triton not having support on windows decreases performance.