Path-JEPA: Path Signature Based Predictive Learning for Skeleton Action Recognition

Self-supervised learning (SSL) has become a leading paradigmfor skeleton-based action recognition, yet the choice of prediction targetremains a central limitation. Existing masked modeling methods typi-cally reconstruct raw joint coordinates, emphasizing low-level detail andremaining sensitive to noise, while recent non-reconstruction variantspredict learned latent features that improve semantics but capture lit-tle explicit motion structure. We propose Path-JEPA, a self-supervisedpredictive learning framework that adopts path signatures as the predic-tion target for skeleton sequences. Rather than predicting coordinatesor abstract embeddings, Path-JEPA predicts latent representations ofcontinuous-time, multi-scale geometric descriptors computed over joint,edge, and chain paths, capturing motion across multiple spatial and tem-poral scales. These targets encode displacement, signed area, and higher-order interactions, yielding a geometry-grounded representation of hu-man motion that is naturally robust to temporal resampling and frame-rate variation. To make such targets effective within a JEPA framework,we introduce signature-augmented masking, which propagates joint-spacemasks to all dependent signature tokens, forcing the model to infer themissing motion geometry from broader anatomical and temporal context.Because signatures summarize motion over intervals rather than indi-vidual frames, they also yield greater computational efficiency on longsequences. Extensive experiments on NTU RGB+D 60, NTU RGB+D120, and PKU-MMD show that Path-JEPA learns stronger representa-tions than prior masked prediction methods, achieving state-of-the-artperformance across diverse downstream tasks while exhibiting improvedrobustness to irregular sampling. Project website:Path-JEPA.