Forecasting Animal Motion

Visual intelligence requires anticipating the future behaviorof agents, yet vision systems lack a general representation for motion andbehavior. We propose dense point trajectories as visual tokens for behavior,a structured mid-level representation that disentangles motion fromappearance and generalizes across diverse non-rigid agents, such as animalsin-the-wild. Building on this abstraction, we design a diffusion transformerthat models unordered sets of trajectories and explicitly reasons aboutocclusion, enabling coherent forecasts of complex motion patterns. Toevaluate at scale, we curate 300 hours of unconstrained animal motion fromvideo through robust shot detection and camera-motion compensation.Experiments show that forecasting trajectory tokens achieves category-agnostic, data-efficient prediction, outperforms state-of-the-art baselines,and generalizes to rare species and morphologies, providing a foundationfor predictive visual intelligence in the wild.