Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views

Multi-view 3D Visual Question Answering (MV3D-VQA) re-quires integrating partial observations into a coherent 3D scene repre-sentation and selecting informative viewpoints for multi-step spatial rea-soning. However, current multimodal LLMs are typically trained withsparse, answer-level supervision, which often yields inconsistent cross-view reasoning and brittle view selection. We present DR-MV3D (DenseReward for MV3D-VQA), a map-grounded learning framework that pro-vides dense, verifiable rewards to supervise the reasoning process. Our ap-proach decomposes MV3D-VQA into (i) allocentric global map construc-tion, (ii) question-conditioned view-trajectory planning, and (iii) egocen-tric grounding for answer prediction. To make intermediate steps learn-able without manual annotations, we introduce two rewards: a global con-sistency reward that aligns the predicted map with geometry-consistentpseudo targets from frozen 3D vision foundation models (e.g., VGGT+ SAM3), and a local trajectory reward that supervises ordered view-point selection. We optimize the full pipeline with trajectory-level pol-icy optimization (GRPO). Experiments on MindCube, VSI-Bench, andBLINK (MV) show that DR-MV3D consistently improves over strongmulti-image baselines, supporting the effectiveness of process-level densesupervision for multi-view 3D reasoning.Code is available at: https://github.com/kaist-cvml/dr-mv3d