Point Diffusion Mamba: Unified Diffusion-State-Space Modeling for Single-View 3D Reconstruction under Data Scarcity
Abstract
While single-view 3D reconstruction has seen significant pro-gress, extrapolating complex 3D structures from inherently ambiguous2D observations remains fundamentally ill-posed, particularly in the crit-ically underexplored data-scarce regime. To address this challenge, wepropose Point Diffusion Mamba (PDM), a method that integrates thegenerative power of diffusion models with the efficiency of state-spacemodel for single-view 3D reconstruction under data-scarce conditions.Specifically, PDM employs a lightweight reconstruction module tailoredto handle unordered point-cloud inputs effectively. By combining a Lo-cal Geometric Aggregation module with Mamba blocks, our approachjointly models global geometric structures and local details. In 3D re-construction, each point in the initial noisy input requires a precise pre-diction, yet the high-level features extracted by the Mamba module cap-ture only abstract semantic information from sparse points. To bridgethis gap, we introduce the Hierarchical Feature Integration Network,which fuses high-level semantic and local geometric features for eachpoint, overcoming the limitations of token-based point-cloud reconstruc-tion. Furthermore, we propose a Dynamic Weighted Sampling strategythat adaptively unifies 3D generation with single-view reconstruction byleveraging generative priors to enhance reconstruction quality. Exper-imental results on the ShapeNet and Pix3D benchmarks demonstratethat PDM outperforms state-of-the-art methods, providing an effectivesolution for 3D reconstruction under data-scarce settings. Code is avail-able at: https://github.com/NWUzhouwei/PDM.