PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motion
Abstract
We present PAI-Studio, a new reference-conditioned videosynthesis task that addresses a long-standing challenge in cinematic back-ground replacement: generating dynamic backgrounds aligned with fore-ground motion while preserving foreground identity, matching referencescene appearance, and achieving globally consistent illumination with re-alistic foreground relighting. Existing open-source systems and commer-cial APIs cannot simultaneously ensure motion-consistent backgroundgeneration, high-fidelity foreground relighting and foreground identitypreservation, often resulting in static backgrounds, inconsistent bound-aries, and noticeable compositing artifacts. To bridge this gap, we buildupon a Diffusion Transformer video backbone and reformulate the prob-lem as an in-context conditional generation task. Through bidirectionalattention, our model jointly captures foreground dynamics and back-ground reference information within a unified architecture. We furtherconstruct a 30K-scale dataset sourced from high-quality films and on-line videos to support this task. Extensive evaluations demonstrate thatour method significantly outperforms existing open-source and commer-cial API solutions. Project page: https://github.com/showlab/PAI-Studio.