About training module in paper

[cuijh26]

Hello, it's a great work. I read the paper, and feel confused about the training modules. Do you train the all modules (attention and ffn) of MM-Dit blocks of CogVideox? Maybe I miss some details, hoping for reply. Thanks.

[SHYuanBest]

Thanks for your interest. Yes, we train all the modules (attention and FFN) of the MM-DIT blocks in CogVideoX, but in practice, it may work by just training the LoRA.

[tyrink]

Thanks for your interest. Yes, we train all the modules (attention and FFN) of the MM-DIT blocks in CogVideoX, but in practice, it may work by just training the LoRA.

Hi, can you give a rough estimate of the amount of training data needed if just training the cross-attn adapter?

[SHYuanBest]

Thanks for your interest. Yes, we train all the modules (attention and FFN) of the MM-DIT blocks in CogVideoX, but in practice, it may work by just training the LoRA.

Hi, can you give a rough estimate of the amount of training data needed if just training the cross-attn adapter?

The specific amount of training data might be determined based on the experimental results. But the more training data, the better, as long as the quality is ensured.