Enhancing Alignment for Unified Multimodal Models via Semantically-Grounded Supervision

Unified Multimodal Models (UMMs) have emerged as apromising paradigm that integrates multimodal understanding and gen-eration within a unified modeling framework. However, current gener-ative training paradigms suffer from inherent limitations. We presentSemantically-Grounded Supervision (SeGroS), a fine-tuning frameworkdesigned to resolve the granularity mismatch and supervisory redundancyin UMMs. At its core, we propose a novel visual grounding map toconstruct two complementary supervision signals. First, we formulatesemantic Visual Hints to compensate for the sparsity of text prompts.Second, we generate a semantically-grounded Corrupted Input to explic-itly enhance the supervision of masking-based UMMs by restricting theText-to-Image loss to core text-aligned regions. Extensive evaluations onGenEval, DPGBench, and CompBench demonstrate that SeGroS signif-icantly improves generation fidelity and cross-modal alignment acrossvarious UMM architectures.Project page: https://segros-project.github.io/