Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstruction

This paper focuses on a challenging setting of simultaneouslymodeling geometry and appearance of hand-object interaction sceneswithout any object priors. We follow the trend of dynamic 3D GaussianSplatting based methods, and address several significant challenges. Tomodel complex hand-object interaction with mutual occlusion and edgeblur, we present interaction-aware hand-object Gaussians with newlyintroduced optimizable parameters aiming to adopt piecewise linear hy-pothesis for clearer structural representation. Moreover, considering thecomplementarity and tightness of hand shape and object shape duringinteraction dynamics, we incorporate hand information into object de-formation field, constructing interaction-aware dynamic fields to modelflexible motions. To further address difficulties in the optimization pro-cess, we propose a progressive strategy that handles dynamic regions andstatic background step by step. Correspondingly, explicit regularizationsare designed to stabilize the hand-object representations for smooth mo-tion transition, physical interaction reality, and coherent lighting. Exper-iments show that our approach surpasses existing dynamic 3D-GS-basedmethods and achieves state-of-the-art performance in reconstructing dy-namic hand-object interaction.