Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding

Empowering Large Multimodal Models (LMMs) with imagegeneration often leads to catastrophic forgetting in understanding tasksdue to severe gradient conflicts. While existing paradigms like Mixture-of-Transformers (MoT) mitigate this conflict through structural isola-tion, they fundamentally sever cross-modal synergy and suffer from ca-pacity fragmentation. In this work, we present Symbiotic-MoE, a unifiedpre-training framework that resolves task interference within a nativemultimodal Mixture-of-Experts (MoE) Transformers architecture withzero-parameter overhead. We first identify that standard MoE tuningleads to routing collapse, where generative gradients dominate expertutilization. To address this, we introduce Modality-Aware Expert Dis-entanglement, which partitions experts into task-specific groups whileutilizing shared experts as a multimodal semantic bridge. Crucially, thisdesign allows shared experts to absorb fine-grained visual semantics fromgenerative tasks to enrich textual representations. To optimize this, wepropose a Progressive Training Strategy featuring differential learningrates and early-stage gradient shielding. This mechanism not only shieldspre-trained knowledge from early volatility but eventually transformsgenerative signals into constructive feedback for understanding. Exten-sive experiments demonstrate that Symbiotic-MoE achieves rapid gener-ative convergence while unlocking cross-modal synergy, boosting inherentunderstanding with remarkable gains on MMLU and OCRBench.