GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Traces
Abstract
Deep research agents integrate fragmented evidence throughmulti-step tool use. BrowseComp offers a text-only testbed for such agents,but existing multimodal benchmarks rarely require both weak visual cuescomposition and BrowseComp-style multi-hop verification. Geolocationis a natural testbed because answers depend on combining multipleambiguous visual cues and validating them with open-web evidence.Thus, we introduce GeoBrowse, a geolocation benchmark that combinesvisual reasoning with knowledge-intensive multi-hop queries. Level 1 testsextracting and composing fragmented visual cues, and Level 2 increasesquery difficulty by injecting long-tail knowledge and obfuscating keyentities. To support evaluation, we provide an agentic workflow GATEwith five think-with-image tools and four knowledge-intensive tools, andrelease expert-annotated stepwise traces grounded in verifiable evidencefor trajectory-level analysis. Experiments show that GATE outperformsdirect inference and open-source agents, indicating that no-tool, search-only or image-only setups are insufficient. Gains come from coherent,level-specific tool-use plans rather than more tool calls, as they morereliably reach annotated key evidence steps and make fewer errors whenintegrating into the final decision. The GeoBrowse benchmark and codesare provided in https://github.com/ornamentt/GeoBrowse.