TR-MoE: Temporal Reliability-Aware Mixture-of-Experts for Robust Tracking

In visual object tracking, SAM2 stands out in deformation adaptability and distinguishing similar objects due to its pixel-level masks, whereas traditional discriminative trackers have advantages under occlusion and motion blur thanks to global semantic feature matching. To leverage this complementarity, we propose TR-MoE, a Temporal Reliability-aware Mixture-of-Experts framework. Unlike conventional fusion strategies that rely on static or per-frame confidence, we reframe tracker fusion as a time-series modeling problem. Specifically, we introduce a Mamba-based Trajectory Refiner (MTR) that processes the historical sequence of tracker states—including relative deviations and consistency scores—to capture the temporal evolution of error regimes. This enables the model to adaptively allocate fusion weights among experts based on historical temporal context. Furthermore, to achieve precise geometric correction, we design a KAN-enhanced Disentangled Head. Leveraging Kolmogorov–Arnold Networks, this module factorizes the refinement into independent directional offsets, dynamic scale magnitudes, and fusion weights. Extensive experiments show that TR-MoE effectively adapts to varying failure patterns, significantly boosting robustness and establishing new state-of-the-art performance. The source code is available at https://github.com/Avant1v/TR-MoE.