CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Models

Embodied Visual Tracking (EVT) requires an agent to con-tinuously follow a language-specified target from egocentric observations.Most existing methods adopt single-agent imitation learning, which de-pends on expert demonstrations and can struggle to generalize to sce-narios beyond the training distribution. Inspired by competition-drivencapability evolution in nature, we propose CoMaTrack, a competitivemulti-agent reinforcement learning (RL) framework that trains agentsin a dynamic adversarial setting with competitive subtasks, yieldingstronger adaptive planning and interference-resilient strategies. We fur-ther propose CoMaTrack-Bench, a Habitat-based benchmark for language-conditioned competitive EVT, featuring dynamic dueling scenarios be-tween a tracker and adaptive opponents across diverse environments andinstructions, enabling standardized robustness evaluation under activeadversarial interactions. Experiments show that CoMaTrack achievesstate-of-the-art results on both standard benchmarks and CoMaTrack-Bench. Notably, a 3B VLM trained with our framework surpasses pre-vious single-agent imitation learning methods based on 7B models onthe challenging EVT-Bench, achieving 92.1% in STT, 74.2% in DT, and57.5% in AT.