GradingBench: Evaluating End-to-End Compositional Reasoning of MLLMs for Automated Exam Grading

While Multi-modal Large Language Models (MLLMs) havedemonstrated superior performance on isolated tasks such as visual ques-tion answering, their reliability remains limited in real-world scenariosthat require highly compositional visual-language reasoning. A primeexample is automated exam grading, a real-world practical task thatdemands the integration of three core capabilities: detection and local-ization, text recognition, and reasoning for correctness judgment. Cur-rent benchmarks largely focus on isolated tasks and cannot fully eval-uate MLLMs’ end-to-end ability in such complex settings. To bridgethis gap, we present GradingBench, a comprehensive benchmark basedon automated exam grading in Chinese K–12 education, which system-atically evaluates MLLMs across the entire grading pipeline. Grading-Bench comprises full-page exam papers from real educational settings,containing 3,284 sub-questions annotated with reference answers. Weevaluate MLLMs across three levels: single-question, specified-question,and full-page grading. Our experiments show that localization failureis the main bottleneck and reveal limitations in integrated multi-taskexecution. Although multi-round interaction and supervised fine-tuningbring slight improvements, the limited gains reflect fundamental weak-nesses in compositional ability. GradingBench serves as a challenging,application-driven benchmark for the community, underscoring the needfor seamless perception-cognition fusion in MLLMs to unlock their po-tential in practical domains. All code and data are available at https://github.com/ERRORSEMI/GradingBench.