OmniPoser: Flexible Human Motion Recovery in the Wild with Masked Flow Matching

Recovering 3D human motion from sparse wearable sensorsis challenging due to the inherent noise, occlusion, and instability ofreal-world tracking signals. Existing approaches are restricted to fixedsensor configurations and, when built on diffusion models, are computa-tionally prohibitive for real-time deployment. We propose OmniPoser, auniversal framework that recovers full-body SMPL motion from a flex-ible combination of heterogeneous inputs, such as 3D keypoints, sparseIMU rotations, head-mounted 6-DoF poses, or any subset thereof, withina single model. OmniPoser introduces a cross-modal masking mechanismpaired with a Masked Conditioning Encoder, enabling one architectureto incorporate diverse sensor configurations through a masked design. Adecoupled dual-stream generative backbone generates complete motionvia Conditional Flow Matching, requiring only a single ODE integrationstep at inference. Across three settings and seven benchmarks, includingin-the-wild Nymeria and Ego-Exo4D dataset, OmniPoser performs onpar with state-of-the-art methods while running at over 700 FPS. Thecode will be open-sourced.