Meric: A Unified Framework for Multimodal Music Generation and Retrieval via Representation Space Anchoring
Abstract
Cross-modal music generation from text and visual inputsis uniquely challenging due to its subjective, one-to-many nature. Cur-rent methods face three major limitations: they are bottlenecked byscarce paired data, lack explicit one-to-many modeling, and isolate gen-eration from retrieval. To address these issues, we propose Meric, aunified framework for multimodal music generation and retrieval. Ourcore innovation is a music semantic anchor that decouples multimodalunderstanding from acoustic synthesis. We employ a two-stage train-ing pipeline. First, a Flow Matching decoder learns to synthesize audiofrom anchor embeddings using large, unpaired music corpora. Second, alightweight generative diffusion module maps multimodal inputs into thiscontinuous anchor space using limited paired data. This decoupled de-sign explicitly models cross-modal ambiguity and exploits unpaired datato bypass the paired data bottleneck. During inference, the predicted an-chor embedding simultaneously acts as a condition for acoustic synthesisand a query for music retrieval. Extensive experiments demonstrate thatMeric achieves state-of-the-art performance across text-to-music andvision-to-music generation and retrieval benchmarks.