REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion

Latent diffusion models (LDMs) achieve state-of-the-art im-age synthesis, yet their reconstruction-style denoising objective providesonly indirect semantic supervision: high-level semantics emerge slowly,requiring longer training and limiting sample quality. Recent works injectsemantics from Vision Foundation Models (VFMs) either externally viarepresentation alignment or internally by jointly modeling only a nar-row slice of VFM features inside the diffusion process, under-utilizingthe rich, nonlinear, multi-layer spatial semantics available. We intro-duce REGLUE (Representation Entanglement with Global-Local UnifiedEncoding), a unified latent diffusion framework that jointly models (i)VAE image latents, (ii) compact local (patch-level) VFM semantics, and(iii) a global (image-level) [CLS] token within a single SiT backbone.A lightweight convolutional semantic compressor nonlinearly aggregatesmulti-layer VFM features into a low-dimensional, spatially structuredrepresentation, which is entangled with the VAE latents in the diffusionprocess. An external alignment loss further regularizes internal repre-sentations toward frozen VFM targets. On ImageNet 256×256, REGLUEconsistently improves FID and accelerates convergence over SiT-B/2 andSiT-XL/2 baselines, as well as over REPA, ReDi, and REG. Extensive exper-iments show that (a) spatial VFM semantics are crucial, (b) non-linearcompression is key to unlocking their full benefit, and (c) global tokensand external alignment act as complementary, lightweight enhancementswithin our global–local–latent joint modeling framework. The code isavailable at https://github.com/giorgospets/reglue.