ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Reconstruction

Reinforcement learning for end-to-end autonomous drivingin closed-loop simulations is gaining increasing attention, yet most sim-ulators differ substantially from real-world conditions, leading to a sig-nificant sim2real gap. Recent methods use scene reconstruction to buildphotorealistic simulators, improving sensor realism but remaining con-strained by the training data distribution, which limits their ability torender novel trajectories and corner cases. To address this, we proposeReconDreamer-RL, a framework that integrates video diffusion priorsinto scene reconstruction for reinforcement learning, enabling more re-alistic and diverse closed-loop autonomous driving training. Specifically,we introduce ReconSimulator, which combines video diffusion priors forappearance modeling with a kinematic model for physical modeling, re-constructing driving scenarios from real-world data and narrowing thesim2real gap. To cover more corner cases, we propose the Dynamic Ad-versary Agent (DAA), which adjusts surrounding vehicles’ trajectoriesrelative to the ego vehicle to generate challenging scenarios such as cut-ins. Finally, we introduce the Cousin Trajectory Generator (CTG) tomitigate the bias of training trajectories toward simple straight-line mo-tion. Experiments show that ReconDreamer-RL improves end-to-end au-tonomous driving training and outperforms imitation learning methodswith a 5× reduction in Collision Ratio.