Online Segment 3D Gaussians via Launching Virtual Drones

Interactive segmentation of 3D Gaussians offers a compellingopportunity for real-time manipulation of 3D scenes, thanks to the real-time rendering capability of 3D Gaussian Splatting (3DGS). However, ex-isting methods require a time-consuming per-scene setup—typically tensof seconds or even minutes—before interactive segmentation can beginon a raw 3DGS scene. This setup involves multi-view mask preparation,mask lifting, and feature distillation, creating a major bottleneck for on-line applications. To address this limitation, we aim to completely elimi-nate the setup stage for interactive 3DGS segmentation while keeping thesegmentation time practical (under 1 second). In this work, we presentSAGO (Segment Any Gaussians Online), a novel setup-free frameworkfor interactive 3DGS segmentation. By introducing virtual drones, ourmethod reframes the 3D segmentation problem as an online Next-Best-View (NBV) planning task formulated within a Markov process. Exten-sive experiments demonstrate that SAGO can extract clean 3D assetsdirectly from 3D Gaussians with sub-second latency, thereby enablinga broad range of downstream applications such as object manipulationand scene editing. Moreover, our method achieves over a 50× speedupcompared to the previous setup-free 3DGS segmentation frameworks.