BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing

We present BlenderFusion, a generative visual compositing framework that recomposes objects, camera, and background to synthesize new scenes. It follows a layering-editing-compositing pipeline that (i) segments and lifts up visual inputs into editable 3D entities (layering), (ii) edits them in Blender with 3D-grounded control (editing), and (iii) fuses them into a coherent scene using a generative compositor (compositing). The generative compositor extends a pre-trained diffusion model to a novel dual-stream architecture, processing both the original (source) and edited (target) scenes in parallel, and is fine-tuned on video frames with two important training strategies: (i) source masking, enabling flexible image modifications like background replacement or object replacement; (ii) simulated object jittering, facilitating robust disentangled control over objects and the camera. Extensive experiments on synthetic and real-world datasets show that BlenderFusion significantly outperforms prior methods in precise 3D-aware control and complex compositional scene editing. The framework also generalizes to unseen data and fine-grained editing operations beyond the training distribution. See the project page for demos and more results: blenderfusion.github.io.