World-in-Loop: Online Correction via Event-Triggered World Models for Robust VLA Policies
Abstract
Large-scale Vision-Language-Action (VLA) models excel atmapping natural language instructions to robotic actions. However, aprimary bottleneck in generalizing these models is the scarcity of high-quality real-world data. While fine-tuning requires massive successfuldemonstrations, the low success rates of current base policies in dy-namic environments often lead to compounding errors and inefficient,low-quality data collection. To enhance both real-world data acquisitionand long-horizon task success, we propose a proactive, online interven-tion framework. Our core innovation is a lightweight, composite WorldModel (WM) acting as an event-triggered “corrector”. Instead of con-tinuous frame-by-frame rollouts across the entire task horizon, our WMintervenes only at critical decision points (e.g., prior to grasping) to eval-uate the proposed action’s feasibility. If a high failure risk is detected, thesystem proactively rolls back to a heal state. A generative model thensynthesizes a short video of a successful future trajectory, which guidesthe robot’s re-attempt via an action decoder. We validate our frameworkacross multiple robotic grasping tasks on both simulation and real-worldsystems. Extensive results demonstrate that deploying world models asactive, on-demand correctors significantly improves task success ratesand efficiently harvests high-quality execution data by rescuing poten-tial failures before they occur.