P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

Vision-Language Models (VLMs) have revolutionized docu-ment parsing by enabling end-to-end mapping from images to structuredtext, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emergedas a promising approach for accelerating inference, its potential is con-strained by optimization instability when scaling to deeper look-aheaddepth. In this paper, we propose P-MTP, a framework that leveragesProgressive Multi-Token Prediction with a lightweight MTP mod-ule to scale the look-ahead depth for high-throughput document parsing.Specifically, we introduce Progressive Curriculum Loss that adaptivelyre-weights different look-ahead depths using cumulative path reliabilityand retrospective target consistency. By effectively suppressing gradientnoise in long-range predictions, P-MTP facilitates an automated easy-to-hard optimization transition, enabling the model to master increasinglydistant look-ahead depths. Furthermore, we propose Confidence-GatedDynamic Drafting to maximize the effective look-ahead depth and ac-ceptance rate by adaptively calibrating speculative length during infer-ence, thereby minimizing computational waste and further pushing theboundaries of inference speedup. Experimental results across multiplebenchmarks and architectures demonstrate that P-MTP achieves up toa 5× speedup with negligible loss in accuracy, providing the first suc-cessful validation of extensive look-ahead MTP in the document parsingdomain.