Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers

Modern one-step diffusion models achieve impressive qual-ity through distribution-based timestep distillation. Yet, they rely on acritical assumption: Teacher and Student must inhabit the same latentspace. This Shared-Space constraint prevents knowledge transfer frommodern high-capacity Teachers (e.g., SD 3.5 and Flux) into compact,deployment-friendly Students such as SD 1.5, whose latent resolutionand VAE parameterization differ from the Teacher. We formalize thisoverlooked regime as Cross-Space Distillation, where Teacher andStudent differ in both latent resolution and VAE space. To enable distil-lation under this mismatch, we introduce the Bridge (Bϕ ), a lightweightlatent interface that maps Student latents into the Teacher space with-out modifying the Student backbone. Bridge combines a frozen StudentVAE decoder as a spatial prior with a compact learnable projector, andis trained with latent reconstruction and attention fidelity objectives forstable Teacher-space alignment. Across diverse modern Teachers, Bridgeenables substantial gains for compact one-step Students; for example, itimproves SD 1.5 from 5.4 to 9.4 HPSv3 while preserving one-step in-ference, low latency, and broad ecosystem compatibility. These resultsshow that heterogeneous large Teachers can be distilled into efficient,deployable backbones through a lightweight latent-space interface.