MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing

Multimodal large language models (MLLMs) have advancedrapidly, yet heterogeneity in architecture, alignment strategies, and effi-ciency means that no single model is uniformly superior across tasks. Inpractical deployments, workloads span lightweight OCR to complex mul-timodal reasoning; using one MLLM for all queries either over-provisionscompute on easy instances or sacrifices accuracy on hard ones. Query-level model selection (routing) addresses this tension, but extending rout-ing from text-only LLMs to MLLMs is nontrivial due to modality fusion,wide variation in computational cost across models, and the absence ofa standardized, budget-aware evaluation.We present MMR-Bench, a unified benchmark that isolates the mul-timodal routing problem and enables comparison under fixed candidatesets and cost models. MMR-Bench provides (i) a controlled environmentwith modality-aware inputs and variable compute budgets, (ii) a broadsuite of vision–language tasks covering OCR, general VQA, and mul-timodal reasoning, and (iii) strong single-model baselines, oracle upperbounds, and representative routing policies. Using MMR-Bench, we showthat incorporating multimodal signals improves routing quality. Empiri-cally, these cues improve the cost–accuracy frontier and enable the routedsystem to exceed the strongest single model’s accuracy at roughly 33\%of its cost. Furthermore, policies trained on a subset of models and tasksgeneralize zero-shot to new datasets and text-only benchmarks withoutretuning, establishing MMR-Bench as a foundation for studying adaptivemultimodal model selection and efficient MLLM deployment.