Learn to See the Unseen in Low-light Spike Streams
Abstract
Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks.Unlike traditional cameras, spike camera continuously accumulates pho-tons and fires asynchronous spike streams. Due to unique data modality,spike streams require reconstruction methods to become perceptible to thehuman eye. However, under low-light high-speed conditions, spike streamsbecome highly sparse and noisy, making faithful pixel-wise recovery in-trinsically difficult. In this work, we propose Diff-SPK, a diffusion-basedframework for perceptual reconstruction from low-light spike streams.Diff-SPK leverages generative priors to produce visually plausible re-constructions while remaining constrained by spike-derived structuralconditions. Specifically, it first employs an Enhanced Texture f rom Inter-spike Interval (ETFI) to aggregate sparse structural information fromlow-light spike streams. Then, the encoded ETFI by a suitable encoderserves as the input of ControlNet for high-speed scenes generation. Toimprove the quality of results, we introduce an ETFI-based feature fusionmodule during the generation process.