ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA

Recent multimodal large language models (MLLMs) havesubstantially advanced video understanding, yet long-form video QAremains challenging under fixed input token budgets, where uniform sam-pling can be inefficient for evidence localization. We propose ReQuest, an uncertainty-driven, question-adaptive keyframe selection pipelinethat aligns question intent with relevant video content through selec-tive computation. ReQuest integrates (i) a lightweight question-awareselector distilled from MLLM-generated supervision, (ii) Re-thinkingRouting that triggers additional inference only when the model is uncer-tain with a length-adaptive criterion, and (iii) uncertainty-guided adap-tive non-maximum suppression that selects temporally diverse frameswhile adjusting spacing based on question difficulty. As a plug-and-play method, ReQuest improves long-video QA without modifyingor fine-tuning the underlying MLLM. Experiments on Video-MME,MLVU, and LongVideoBench demonstrate consistent accuracy gainswith competitive computational cost, with particularly strong improve-ments in medium and long video regimes. The code is available athttps://geppa.github.io/ReQuest