Single-Query Person-Centric Bimanual Hand-Object Interaction Detection

Understanding person-level bi-manual interactions requiresnot only detecting hands, but also identifying which two hands belongto the same person and what each hand interacts with. Existing hand–object interaction methods are mostly hand-centric: they treat each handas an independent instance, which can lead to ambiguous ownership inmulti-person scenes.We propose a person-centric formulation in which a single query pre-dicts a structured output for one person, including the human box,body pose, hand boxes and states, and interaction targets. We intro-duce part-aware deformable attention to allocate attention acrosshuman, hand, and pose-specific reference regions, enabling one query tocapture the full person structure. We further unify detection and inter-action reasoning with a hand-to-query relationship matrix, whereeach hand selects its interaction target from the detected query set plusa learnable off token, directly recovering the target’s box and class with-out separate object regression.We build a COCO-based dataset with person-centric bi-manual interac-tion annotations and define structured metrics for evaluating hand statesand complete hand–object tuples. Experiments with a transformer-baseddetector show that our formulation improves person-level bi-manual in-teraction parsing and provides an effective unified framework for jointdetection, pose estimation, and hand reasoning.Project page: https://lgecto-ail-vil.github.io/SingleQuery-BHOI/