Test-time Counterfactual Calibration for Hallucination-Resistant Temporal Grounding

Existing video temporal grounding methods typically oper-ate under a closed-world assumption, relying on the implicit premise thatevery natural language query must have a corresponding event some-where in the video. This always-answer paradigm severely compressesthe decision boundary during training, as models are seldom exposed tothe alternative hypothesis that the queried event may be absent. Con-sequently, when users ask about specific events that cannot be matchedin the video within an open-world environment, these models exhibitsystematic temporal hallucinations. This may stem from incorrect at-tribution of textual, visual, or multimodal information, leading them toconfidently generate plausible time frames for events that do not actuallyexist. To address this limitation, we propose HRVTG, a test-time adapta-tion framework that dynamically calibrates the decision boundary of themodel during inference. Built upon a frozen grounding backbone, the pro-posed framework constructs three types of counterfactual hallucinationprobes in a self-supervised manner, targeting the aforementioned halluci-nations. We dynamically reshape the decision boundary through absten-tion signals from counterfactual probes and consistency rewards fromgenuine queries. This is executed by introducing the GRPO algorithm,effectively reformulating the task into an online policy optimization prob-lem in reinforcement learning. Furthermore, we introduce decoupled met-rics to independently evaluate hallucination resistance and real-eventgrounding. Experiments indicate that our method suppresses halluci-nated responses to fabricated queries. It achieves accurate selective pre-diction while maintaining strong grounding performance on real events.The code will be released at https://github.com/CVL-hub/HRVTG.