Instance Segmentation as Tracking: A New Paradigm for Multi-Small-Object Tracking with Event Cameras

Multiple small object tracking (MSOT) is critical for ap-plications such as anti-UAV systems and security surveillance, yet tra-ditional frame-based cameras struggle to track fast-moving small ob-jects in complex environments due to their low frame rates and lim-ited dynamic range. Event cameras, with their ability to continuouslyrecord subtle brightness changes, can naturally overcome these limita-tions. However, most existing event-based tracking methods follow a“convert-then-detect-and-track” pipeline. This pipeline sacrifices the in-herent high temporal resolution of event data, leading to fragmentedtrajectories of fast-moving objects. Moreover, it introduces significantbackground redundancy during framing, which reduces computational ef-ficiency. To handle these issues, we introduce “instance segmentationas tracking”, a novel paradigm that formulates event-based MSOT asan instance segmentation task in the spatio-temporal dimension. Follow-ing this paradigm, we first design Ev-ISNet, which leverages 3D sparseconvolutions to extract per-voxel features while simultaneously predict-ing object confidence and motion direction. Then we construct an eventgraph and progressively cluster trajectory instances by predicting edge-wise instance affinities. By leveraging intra-graph and inter-graph associ-ation modules, our method achieves highly efficient streaming inference.To address the lack of large-scale benchmarks for event-based MSOT,we build EV-UAV-Track, a comprehensive dataset featuring per-eventinstance-level annotations. Extensive experiments demonstrate that Ev-ISNet consistently outperforms state-of-the-art MSOT methods, achiev-ing over 30% improvement in MOTA score compared to frame-basedtracking methods.