NeSy-Route: A Neural-Symbolic Benchmark for Constrained Route Planning in Remote Sensing

Remote sensing underpins crucial applications such as disas-ter relief and ecological field surveys, where systems must understandcomplex scenes and constraints and make reliable decisions. Currentremote-sensing benchmarks mainly focus on evaluating perception andreasoning capabilities of multimodal large language models (MLLMs).They fail to assess planning capability, stemming either from the diffi-culty of curating and validating planning tasks at scale or from evaluationprotocols that are inaccurate and inadequate. To address these limita-tions, we introduce NeSy-Route, a large-scale neuro-symbolic benchmarkfor constrained route planning in remote sensing. Within this benchmark,we introduce an automated data-generation framework that integrateshigh-fidelity semantic masks with heuristic search to produce diverseroute-planning tasks with provably optimal solutions. This allows NeSy-Route to comprehensively evaluate planning across 10,821 route-planningsamples, nearly 10 times larger than the largest prior benchmark. Fur-thermore, a three-level hierarchical neuro-symbolic evaluation protocol isdeveloped to enable accurate assessment and support fine-grained analy-sis on perception, reasoning, and planning simultaneously. Our compre-hensive evaluation of various state-of-the-art MLLMs demonstrates thatexisting MLLMs show significant deficiencies in perception and planningcapabilities. We hope NeSy-Route can support further research and de-velopment of more powerful MLLMs for remote sensing.The dataset andcode are available at https://mingyang1010.github.io/NeSy-Route/.