Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation

Recent advances in generative medical models are often con-strained by modality-specific scenarios that hinder the integration ofcomplementary evidence, such as imaging, pathology, and clinical notes.This fragmentation limits their development to true foundation modelsthat empower medical AI agents to learn from and predict across thefull spectrum of biomedical knowledge. To address these challenges, wepropose MeDiM, the first medical discrete diffusion model that learnsshared distributions across different medical modalities without requiringmodality-specific components. MeDiM unifies multiple generative tasks:it flexibly translates between images and text or jointly produces im-age–report pairs across domains in response to user prompts. It builds ona discrete diffusion framework that unifies vision and language represen-tations by modeling their shared probabilistic distribution. To empowerthe diffusion process to support unified and versatile medical generation,we employ a multimodal large language model (MLLM) as the diffusionbackbone, leveraging its rich prior knowledge and cross-modal reason-ing abilities. Because MLLMs are trained with causal (autoregressive)masking while diffusion denoising benefits from bidirectional context,MeDiM introduces two adaptation designs: 1) removing the causal at-tention mask to enable a fully bidirectional information flow essentialfor mutual alignment, and 2) injecting continuous timestep embeddingsto make the MLLM aware of the diffusion steps. Extensive experimentsvalidate MeDiM as a unified foundation model capable of high-fidelitymedical generation across various modalities, including medical imagegeneration (16.60 FID on MIMIC-CXR; 24.19 FID on PathGen) and re-port generation (0.2650 METEOR on MIMIC-CXR; 0.2580 METEOR onPathGen). In addition, the jointly generated medical image-report pairsimprove the downstream task performance (+6.43% BLEU-1, +18.57%BLEU-2, +31.58% BLEU-3, and +4.80% METEOR in PathGen), en-abling the use of multimodal inputs and the production of coherent,clinically grounded outputs.