Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

This paper introduces Actor as Its Own Critic, a unifiedreinforcement learning framework, Cycle Group Relative Policy Opti-mization (CycleGRPO), that jointly optimizes region understanding andlocalization for Multimodal Large Language Models (MLLMs). Unlikeexisting separate pipelines, we leverage the inherent duality between thetwo tasks to construct a self-evaluating reinforcement learning paradigm:“region → text → region”. Specifically, a single MLLM first acts as theactor to generate region captions, then immediately transitions to a criticto ground its generated text back in the spatial domain. Therefore, Cy-cleGRPO requires only region inputs, e.g., masks or bounding boxes,entirely bypassing the need for textual ground truths. A quality-awaretoken-level cycle-consistency reward is employed to assess the seman-tic discriminability of text captions via their physical localization ac-curacy. Empirically, built upon SAMTok, our CycleGRPO frameworksuccessfully bootstraps both capabilities simultaneously. Without anytask-specific fine-tuning, the framework yields consistent performancegains across a wide range of benchmarks, including region captioning,region VQA, grounded dialogue, and referring segmentation. Overall,CycleGRPO offers a straightforward and scalable way to advance pixel-level capabilities in MLLMs. Code and models are released at https://github.com/devinxzhang/CycleGRPO.