OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data

Music-driven dance video generation aims to synthesize expressive human motion that is temporally aligned with music while maintaining high visual fidelity. Despite recent progress, existing methods still struggle to generate dance videos that simultaneously exhibit expressive motion and high visual quality. This limitation primarily arises from two factors: (1) Dataset. The lack of large-scale and high-quality datasets and effective data collection pipelines specifically tailored for dance video generation; and (2) Method. The absence of principled framework-level solutions for effectively integrating music as a complementary conditioning signal into the Video Generation Foundation Models. To address the dataset limitation, we introduce CIPE-Dance, a large-scale Internetsourced dance video dataset, equipped with Choreograph Informed text annotations and constructed via a Progressive Expert pipeline. To the best of our knowledge, CIPE-Dance is the largest dataset for dance video generation to date, comprising 300k high-quality clips (over 400 hours) and covering diverse dancers, environments, and dance genres. To overcome the method limitation, we propose OmniDance, a frameworklevel recipe for integrating music into a TI2V foundation model without sacrificing its original controllability or visual fidelity. Motivated by the complementary roles of text (low-frequency semantics) and music (highfrequency temporal dynamics), OmniDance co-designs a depth-aware specialization model architecture, an anchored easy-to-hard curriculum learning strategy, and modality-specialized time-dependent CFG strategy, achieving unified TI2V/MI2V/MTI2V generation. Extensive experiments on the CIPE-Dance dataset demonstrate that OmniDance achieves state-of-the-art performance across TI2V, MI2V, and MTI2V tasks, while exhibiting robust multimodal integration capability. Project is available at https://github.com/AMAP-ML/OmniDance.