Segmenting, Fast and Slow: Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing

Object-centric models inspired by DETR have become thedominant paradigm for open-vocabulary video instance segmentation(OV-VIS). While recent efforts have reduced the computational cost ofpixel decoding, textual modality fusion, and object decoding to makethese architectures more suitable for mobile devices, real-time on-deviceinference at high frame rates remains an open challenge. In this paper,we introduce SegFS, a dual-stream fast-slow framework that significantlyimproves efficiency without sacrificing accuracy. On sparse keyframes, anopen-vocabulary object-based model predicts instance-level representa-tions. These representations are then projected back into the backbonefeature space to condition a lightweight fast network, which efficientlyrelocalizes and segments the instances in subsequent frames. By shiftinginstance propagation from object decoding to feature-space condition-ing, our approach decouples multimodal semantic understanding fromdense mask prediction and enables efficient temporal propagation. Theproposed fast branch achieves up to 14× lower latency than the mobile-oriented MOBIUS model, while maintaining competitive segmentationperformance on standard OV-VIS benchmarks.