Fixed Reality, Diffused Possibility: Disentangling Stochastic and Deterministic Latent for Cluttered Grasping

Recent diffusion-based generative models have shown strongcapabilities in 3D scene understanding and affordance reasoning. How-ever, existing methods predominantly rely on deterministic global en-coders or fully stochastic diffusion backbones, which fail to disentanglestructural certainty from contextual uncertainty, a key requirement forreliable affordance-grasp prediction in cluttered, partially observed en-vironments. We propose a split-latent hierarchical diffusion frameworkthat explicitly decomposes the global scene representation into two com-plementary subspaces: a fixed latent encoding deterministic geometricand semantic priors, and a diffused latent capturing stochastic scene-level variations through a Global Conditional Diffusion guided by RGBcontext. To propagate global uncertainty to fine-grained reasoning, weintroduce a Local Conditional Diffusion that predicts dense, point-wiseaffordance and grasp fields conditioned jointly on both latents, producinggeometrically precise yet contextually diverse predictions. Furthermore,we incorporate Harmonic Grasp Field, a differential geometric regulariza-tion that enforces smooth, obstacle-aware grasp manifolds via harmonicpotential constraints. This unified formulation bridges deterministic ge-ometry with probabilistic reasoning, yielding physically consistent anduncertainty-aware grasp generation. Extensive experiments on cluttered3D affordance and grasping benchmarks demonstrate superior stability,generalization, and diversity compared to deterministic and single-latentdiffusion approaches. Project Website.