SMG: Semantic Motion Graph for Monocular Dynamic Gaussian Splatting

We study dynamic Gaussian Splatting from monocular videos.While recent advancements in dynamic Gaussian Splatting offer a promis-ing foundation for modeling dynamic scenes, they often overfit to thetraining views and fail under occlusion or complex scene motion due tothe lack of reliable regularization signals in under-constrained regions.We propose Semantic Motion Graph (SMG), a novel approach that mod-els the Gaussian motion as the semantic motion. Our key insight is thatthe real-world scene motion is often structured by semantic coherence:regions that are spatially close and semantically related tend to exhibitconsistent dynamics. To leverage this prior, we build SMG to modelstructured motion of the scene. The Gaussian motion is driven by the mo-tion of SMG nodes. We further observe that the uncertainty of Gaussianmotion arises from both unreliable off-the-shelf priors and weakly con-strained regions during optimization. SMG addresses this by using reli-able graph nodes to guide the motion of nearby unreliable nodes. To eval-uate dynamic Gaussian Splatting under challenging real-world scenarios,we introduce a new multiview dataset collected under an ego-exo setup.Extensive experiments demonstrate that SMG achieves state-of-the-artperformance on monocular dynamic Gaussian Splatting across challeng-ing real-world benchmarks. Project: https://smg-gaussian.github.io/.