DualDn: Dual-domain Denoising via Differentiable ISP

Image denoising is a critical step in the Image Signal Processor (ISP) of a camera. There are two typical ways to inject a denoiser into the ISP pipeline: a raw domain denoiser that is directly applied to captured raw frames, and an sRGB domain denoiser that is applied to the sRGB image output by the ISP. However, both approaches have their limitations. The residual noise from the raw-domain denoising will be amplified by the ISP pipeline, and the sRGB domain cannot handle spatially varying noise as it only sees noise distorted by ISP processing. As a result, most raw-domain or sRGB-domain denoising works only for specific noise distributions and ISP configurations. To address these challenges, we propose DualDn, a novel learning-based dual-domain denoising. Unlike previous single-domain denoising, DualDn consists of two denoising networks, one in the raw domain and one in the sRGB domain. The raw domain denoising can adapt to spatially varying noise levels, and the sRGB domain denoising can remove the residual noise amplified by the ISP. Both denoising networks are connected with a differentiable ISP, which is trained end-to-end and discarded during the inference stage. With this design, DualDn achieves greater generalizability compared to most learning-based denoising, as it can adapt to different unseen noises, ISP parameters, and even novel ISP pipelines. Experiments show that DualDn achieves state-of-the-art performance and can adapt to different denoising network architectures. Moreover, DualDn can be used as a plug-and-play denoising module with real cameras without retraining, and still demonstrate better performance than commercial on-camera denoising, further showing its generalization ability.