Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach

Skeleton-based action recognition has achieved remarkablesuccess by exploiting joint coordinates and their topological connec-tions, yet prevailing methods overwhelmingly assume complete and cleanskeleton inputs. In real-world deployments, such as egocentric vision,crowded surveillance, wearable devices, or edge robotics, limited field-of-view (FoV) frequently causes substantial joint visibility dropout, leadingto severe performance degradation that existing models are largely un-prepared to handle. To bridge this critical yet underexplored gap, weintroduce PartialVisGraph, a novel hypergraph framework tailored forrobust skeleton action recognition under constrained FoV. We first con-struct highly expressive hypergraphs by introducing learnable virtualhyperedges that form a soft incidence matrix, capturing flexible high-order dependencies beyond conventional pairwise graphs. We then pro-pose the Single-Head Sample-Adaptive Transformer, which adaptivelyaggregates joint features onto hyperedges while explicitly incorporatinga visibility prior. This prior selectively gates information flow, preventingoccluded or out-of-view joints from corrupting reliable feature propaga-tion. We further establish rigorous evaluation protocols with realisticFoV simulation benchmarks on NTU RGB+D 60 and 120. Extensive ex-periments demonstrate that PartialVisGraph consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% onsubsets with severe FoV restrictions compared to recent strong base-lines, while remaining superior on full-visibility settings. Our approachoffers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments. Resourcesrelated to the constrained FoV setting used in this work are available at:https://github.com/yaa1haa1/PartialVisGraph.