PolarAPP: Beyond Polarization Demosaicking for Polarimetric Applications

Polarimetric imaging enables advanced vision applicationssuch as normal estimation and de-reflection by capturing unique surface-material interactions. However, existing applications (alternatively calleddownstream tasks) rely on datasets constructed by naïvely regroupingraw measurements from division-of-focal-plane sensors—where pixels ofthe same polarization angle are extracted and aligned into sparse im-ages without proper demosaicking. This reconstruction strategy resultsin suboptimal, incomplete targets that limit downstream performance.Moreover, current demosaicking methods are task-agnostic, optimizingonly for photometric fidelity rather than utility in downstream tasks.Towards this end, we propose PolarAPP, the first framework to jointlyoptimize demosaicking and its downstream tasks. PolarAPP introducesa feature alignment mechanism that semantically aligns the representa-tions of demosaicking and downstream networks via meta-learning, guid-ing the reconstruction to be task-aware. It further employs an equivalentimaging constraint for demosaicking training, enabling direct regressionto physically meaningful outputs without relying on rearranged data.Finally, a task-refinement stage fine-tunes the task network using thestable demosaicking front-end to further enhance accuracy. Extensiveexperimental results demonstrate that PolarAPP outperforms existingmethods in both demosaicking quality and downstream performance.Code is available here.