In version 1.3.0, Open-Sora-Plan introduced the following five key features:

1. A more powerful and cost-efficient WFVAE. We decompose video into several sub-bands using wavelet transforms, naturally capturing information across different frequency domains, leading to more efficient and robust VAE learning.
2. Prompt Refiner. A large language model designed to refine short text inputs.
3. High-quality data cleaning strategy. The cleaned panda70m dataset retains only 27% of the original data.
4. DiT with new sparse attention. A more cost-effective and efficient learning approach.
5. Dynamic resolution and dynamic duration. This enables more efficient utilization of videos with varying lengths (treating a single frame as an image).

For further details, please refer to our report.

• COMING SOON ⚡️⚡️⚡️ For large model parallelisation training, TP & SP and more strategies are coming...

  近期将新增华为昇腾多模态MindSpeed-MM分支，借助华为MindSpeed-MM套件的能力支撑Open-Sora Plan参数的扩增，为更大参数规模的模型训练提供TP、SP等分布式训练能力。