Please release the code to support Wan 2.1

[deepbeepmeep]

Hello

I would very happy to add RifleX to a new WAN tool I about to release. I would be grateful if you could release the code.

[gracezhao1997]

Thank you for your attention! We are currently reorganizing the code and will release it soon. By the way, how do users typically use these models to generate videos, and which repositories should we support to facilitate this process?

[deepbeepmeep]

These models are very recent so it is hard to give you a feedback, but Wan, Hunyuan and CogvideoX are the most popular at the moment

I understand it is not easy to produce a plug and play RifleX demo given its code structure. In fact we just need the RIFLEx parameters (k, N_k) for each Wan model. It would help if you could please share them.

[deepbeepmeep]

I have used you identify tool with multiples values of N (the first observed repetition frame in latent space), I keep getting 4 as a value of k. I have then modified the code equivalent to get_1d_rotary_pos_embed in Wan.
The result is not as good as with Hunyuan (repetition is still there, visual artifacts). Is 4 wrong for k ? Any recommendation ? Many thanks in advance

[zhuhz22]

Hi @deepbeepmeep ， we found k=6 in Wan2.1. We observed repetition at about 6.5s, where the latent frame index is about 21, and for k=6, its N_k is 26, so we set k=6 as it's the nearest frequency.

Also，there's a tip that the meaning of freqs in Wan's code is different from that of Hunyuan. So the code shall be

def rope_params_riflex(max_seq_len, dim, theta=10000, L_test=30, k=6):
    assert dim % 2 == 0
    exponents = torch.arange(0, dim, 2, dtype=torch.float64).div(dim)
    inv_theta_pow = 1.0 / torch.pow(theta, exponents)
    
    inv_theta_pow[k-1] = 0.9 * 2 * torch.pi / L_test
        
    freqs = torch.outer(torch.arange(max_seq_len), inv_theta_pow)
    freqs = torch.polar(torch.ones_like(freqs), freqs)
    return freqs

rather than freqs[k-1]= 0.9 * 2 * torch.pi / L_test.

We're currently reorganizing the code, and will release support for Wan soon.

[deepbeepmeep]

Thank you. I did integrate the code my application Wan2.1GP.
https://github.com/deepbeepmeep/Wan2GP
It is nice but unless I did something wrong it doesn't seem to be as efficient as with Hunyuanvideo: this time the max is 8s and it seems there is a higher chance to get some repetition

[markrmiller]

@gracezhao1997

By the way, how do users typically use these models to generate videos

I think via ComfyUI has got to be the most popular way...

[IntellectzProductions]

Yes, please make sure the comfy is supported. Kijai's Wrapper and the Native set nodes.

[4ever-AI]

I also agree about ComfyUI. It would be great if you could support it.

[gracezhao1997]

Hi, @markrmiller @IntellectzProductions @4ever-AI. Thank you for your suggestions! I'd like to clarify whether the ComfyUI you are referring to is the one found at comfyanonymous/ComfyUI or if it's Kijai's Wrapper.

[markrmiller]

People will often use both, but I have to assume the built in nodes end up with the widest use.

It's fairly common for someone to build some custom ComfyUI nodes for a new model before ComfyUi supports it out of the box. For example, like that Wrapper. Then ComfyUI will add native support for the model, but the custom nodes that someone first built as a plugin will often have more customizability or advanced features, so some people will use the native nodes and others will install and use the custom nodes.

If support were added to either, I wouldn't be surprised if the other one just picked it up as well as its likely very similar work in both cases.

[kijai]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.

For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

[4ever-AI]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.

For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

@kijai Thank you so much for all of your efforts and all the nodes you have done for ComfyUI!
RIFLEx nodes are not native supported in ComfyUI yet. Is it possible to write a short description how to manually instal RIFLEx and then your nodes as well? I have many of your nodes running in my ComfyUI but I am not sure which new to install or to update.

Thank you in advance and wish you all the best for your next projects!
4ever-AI

[kijai]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.
For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

@kijai Thank you so much for all of your efforts and all the nodes you have done for ComfyUI! RIFLEx nodes are not native supported in ComfyUI yet. Is it possible to write a short description how to manually instal RIFLEx and then your nodes as well? I have many of your nodes running in my ComfyUI but I am not sure which new to install or to update.

Thank you in advance and wish you all the best for your next projects! 4ever-AI

The nodes I showed are for native workflows, they are simply part of KJNodes and all you'd need to do is install that, then add that node to any existing WanVideo workflow.

[4ever-AI]

Hi, @markrmiller @IntellectzProductions @4ever-AI. Thank you for your suggestions! I'd like to clarify whether the ComfyUI you are referring to is the one found at comfyanonymous/ComfyUI or if it's Kijai's Wrapper.

@gracezhao1997 Thank you for your kind support and congratulations for your beautiful project! I just checked and your node / model is not yet officially supported by ComfyUI. Maybe yes, it would be nice if you could come in communication with comfyanonymous to officially support it. Till then maybe you could offer some instructions to manually install it in ComfyUI.
Thank you once more!

[4ever-AI]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.
For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

@kijai Thank you so much for all of your efforts and all the nodes you have done for ComfyUI! RIFLEx nodes are not native supported in ComfyUI yet. Is it possible to write a short description how to manually instal RIFLEx and then your nodes as well? I have many of your nodes running in my ComfyUI but I am not sure which new to install or to update.
Thank you in advance and wish you all the best for your next projects! 4ever-AI

The nodes I showed are for native workflows, they are simply part of KJNodes and all you'd need to do is install that, then add that node to any existing WanVideo workflow.

@kijai Thank you so much for your quick reply and kind support! I am not very skilled in installing manually something. I think I can do it, by creating a new directory in the nodes folder inside comfyui, and then run these commands inside the directory ?
conda create -n riflex python=3.10
pip install -r requirements.txt
pip install -U bitsandbytes

or just install the requirements and the bitsandbytes without the python install since it's already installed in the comfyui ?

then just update your nodes ComfyUI-KJNodes ? I have currently your node at the version 1.0.6 nightly .

Thank you again!

[kijai]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.
For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

@kijai Thank you so much for all of your efforts and all the nodes you have done for ComfyUI! RIFLEx nodes are not native supported in ComfyUI yet. Is it possible to write a short description how to manually instal RIFLEx and then your nodes as well? I have many of your nodes running in my ComfyUI but I am not sure which new to install or to update.
Thank you in advance and wish you all the best for your next projects! 4ever-AI

The nodes I showed are for native workflows, they are simply part of KJNodes and all you'd need to do is install that, then add that node to any existing WanVideo workflow.

@kijai Thank you so much for your quick reply and kind support! I am not very skilled in installing manually something. I think I can do it, by creating a new directory in the nodes folder inside comfyui, and then run these commands inside the directory ? conda create -n riflex python=3.10 pip install -r requirements.txt pip install -U bitsandbytes

or just install the requirements and the bitsandbytes without the python install since it's already installed in the comfyui ?

then just update your nodes ComfyUI-KJNodes ? I have currently your node at the version 1.0.6 nightly .

Thank you again!

You literally don't need anything but to install the ComfyUI-KJNodes, nothing else, you can install it from the Manager as well.

[4ever-AI]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.
For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

@kijai Thank you so much for all of your efforts and all the nodes you have done for ComfyUI! RIFLEx nodes are not native supported in ComfyUI yet. Is it possible to write a short description how to manually instal RIFLEx and then your nodes as well? I have many of your nodes running in my ComfyUI but I am not sure which new to install or to update.
Thank you in advance and wish you all the best for your next projects! 4ever-AI

The nodes I showed are for native workflows, they are simply part of KJNodes and all you'd need to do is install that, then add that node to any existing WanVideo workflow.

@kijai Thank you so much for your quick reply and kind support! I am not very skilled in installing manually something. I think I can do it, by creating a new directory in the nodes folder inside comfyui, and then run these commands inside the directory ? conda create -n riflex python=3.10 pip install -r requirements.txt pip install -U bitsandbytes
or just install the requirements and the bitsandbytes without the python install since it's already installed in the comfyui ?
then just update your nodes ComfyUI-KJNodes ? I have currently your node at the version 1.0.6 nightly .
Thank you again!

You literally don't need anything but to install the ComfyUI-KJNodes, nothing else, you can install it from the Manager as well.

Thank you so much @kijai! I really appreciate your kind support! I am excited and thrilled! You are the best!

[Pythonpa]

@kijai Regarding the usage of adding the modules in KJNodes to the native workflow that you mentioned, do you have a reference workflow?

[jwwangchn]

I used RIFLEx in wanx2.1 with k=6 to generate a 10s (161 frames) video, but the results were very poor, with noticeable frame jumps and abrupt changes. Could you help me identify the possible reasons?

[Pythonpa]

@kijai Regarding the usage of adding the modules in KJNodes to the native workflow that you mentioned, do you have a reference workflow?

I have found some ori workflow,Which has this node

[nitinh12]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.

For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

Where do you have this in Wan Video Wrapper?

[kijai]

For ComfyUI I have support in both Hunyuan and WanVideo wrapper nodes, for which @zhuhz22 graciously also corrected my mistakes in the implementation.
For the native ComfyUI implementations of the models I have simple patcher nodes for both in KJNodes

Where do you have this in Wan Video Wrapper?

Riflex_freq_index in the sampler node, that's the k - value.