LINA: Learning INterventions Adaptively for Physical Alignment and Counterfactual Generation in Diffusion Models

Diffusion models (DMs) have achieved remarkable successin image and video generation. However, there are still challenges in(1) physical alignment and (2) counterfactual generation. To diagnosethe root causes, we introduce the Causal Scene Graph (CSG) to modelDM’s generative process, and build the Physical Alignment Probe (PAP)dataset to quantify the failure modes. Our analysis yields three key in-sights: (1) DMs struggle with physical elements not explicitly determinedin the prompt; (2) the prompt embedding contains disentangled repre-sentations for texture and physics; (3) visual causal structure is dispro-portionately established during the initial, computationally constraineddenoising steps. Based on these findings, we introduce Lina (LearningINterventions Adaptively), a novel framework that learns to predictprompt-specific interventions. Lina employs (1) targeted guidance in theprompt and visual latent spaces, and (2) a reallocated, causality-awaredenoising schedule. Without modifying pre-trained DM weights, Linaenforces both physical alignment and counterfactual generation in imageand video DMs, achieving state-of-the-art performance on challengingcausal generation tasks and the Winoground dataset. Our project pageis at https://opencausalab.github.io/LINA