340 FPS Reflection-free Video from Spikes Modulated by a Rapidly Rotating Polarizer
Abstract
Removing reflections from mixed light using polarization of-fers the benefits of robust physical constraints. However, extending thisto high-speed scenes presents challenges due to the discrete frames ofconventional cameras. In this paper, we propose 340 FPS reflection-freevideo reconstruction by leveraging the high-speed spike camera modu-lated by a rapidly rotating polarizer. The core motivation of our approachrelies on exploiting the continuous and asynchronous firing of the spikesensor to densely capture instantaneous polarization states. However, theinherently low signal-to-noise ratio of spikes under extremely short inte-gration windows and light attenuation causes severe numerical instabilityin physical layer separation. To address this, we introduce a physically-driven framework featuring a differentiable Reliability-Weighted PhysicsSolver (RWPS). RWPS robustly separates reflection and transmissionlayers by dynamically adjusting learned confidence weights to handlesevere shot noise. Furthermore, we develop a refinement module thatleverages back-projection residuals and physical-discrepancy cues to ac-curately restore high-frequency textures. Extensive experiments demon-strate that our method enables robust reconstruction of high-fidelity andreflection-free videos.