Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigation

On-policy exploration is a crucial component for trainingrobust Vision-Language Navigation (VLN) agents, as it exposes the pol-icy to a broader state distribution. However, such exploration inevitablyleads to trajectories that deviate from expert demonstrations, resulting ina semantic mismatch between the executed visual stream and the originallanguage instruction. In this work, we address this challenge by introduc-ing Φ-Nav, a unified on-policy framework that leverages hindsight rea-soning to align instructions with the agent’s actual exploratory journey.Specifically, Φ-Nav operates through a three-stage dual-supervision cy-cle: 1) the agent performs oracle-guided on-policy exploration, samplinga trajectory while learning from expert action feedback, 2) a hindsightspeaker synthesizes a path-level hindsight instruction grounded in thecollected visual observations, and 3) the agent conducts a second imita-tion pass, treating the synthesized trajectory–instruction pair as an ad-ditional expert demonstration. Through this process, Φ-Nav bridges thecritical semantic supervision gap inherent in on-policy methods, trans-forming semantically unlabeled movement into dense training signals.Evaluations on the R2R-CE and RxR-CE benchmarks show that Φ-Navyields competitive performance while requiring only a fraction of the ex-pert demonstrations used by current baselines. These results underscorethe necessity of semantic exploration in VLN, positioning Φ-Nav as aeffective solution for training embodied agents with limited data.