SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models
Abstract
Multimodal Large Language Models (MLLMs) have achievedremarkable success in visual understanding but remain constrained invisual generation due to the fundamental feature discrepancy betweensemantic perception and pixel-level reconstruction. Bridging this gap re-quires overcoming two core challenges: endowing semantic encoders withhigh-fidelity reconstruction capabilities, and effectively aligning gener-ative models with semantic spaces without relying on external teach-ers. To this end, we propose a novel unified multimodal framework fea-turing Semantic-Pixel self-alignment and Adaptive Routing (SPAR).First, to reconcile semantic perception with pixel-level reconstruction, weintroduce an asymmetric dual-stream unified tokenizer. A lightweightsemantic stream anchors discriminative features, while a Transformer-augmented pixel stream recovers fine-grained visual details into a uni-fied compact latent space. Second, to eliminate external dependencies, wepropose a self-aligned generation paradigm that natively leverages thisoptimized tokenizer as an internal alignment teacher for the diffusionmodel. Furthermore, to facilitate flexible multimodal interaction withinthis unified space, we introduce Dynamic Token Routing, which enableseach token to adaptively aggregate multi-layer MLLM features based onits distinct semantic demands. Extensive experiments demonstrate thatSPAR establishes the state-of-the-art for unified architectures, achiev-ing exceptional generation and reconstruction quality while preservingfoundational visual understanding capabilities.