One Demonstration Is Enough for Real-World Robotic Reinforcement Learning

Learning effective robot control policies on physical hard-ware is challenging due to costly data collection and the difficulty ofreward specification. Prior work has incorporated demonstrations intoreinforcement learning (RL), yet existing approaches either require largenumbers of demonstrations or depend on continuous human interventionduring training. To address these limitations, we present AutoSERL, aframework that leverages a single demonstration to fully automate the in-tervention process in real-world robot RL. The framework includes threecomplementary mechanisms to accomplish certain tasks: a sliding win-dow intervention that continuously guides exploration to prevent localoptima and unsafe deviations, a safety recovery mechanism that detectsand corrects failure states via predefined trajectory recovery points, andan intervention termination criterion that automatically disables guid-ance once the policy can independently complete the task, preserving itsexploration advantage. We evaluate AutoSERL on six contact-intensivemanipulation tasks across two robot platforms, spanning insertion, hang-ing, and hinge-based tasks. AutoSERL consistently outperforms SERLinitialized with 20 demonstrations, behavior cloning, and MILES — adedicated one-shot imitation learning baseline — across all tasks whilematching HIL-SERL, achieves 100% success rate on insertion tasks, anddemonstrates improved robustness to positional variations, all from a sin-gle demonstration. Code and videos are available on our project website:https://autoserl.github.io/.