OmniDS: Dual-Stream Context Fusion for Omnidirectional Depth from Fisheye Cameras

Omnidirectional depth estimation from multi-fisheye camerarigs is complicated by visibility conflicts: wide baselines cause differentcameras to observe different portions, or even different faces, of the sameobject, so aggregating their features into a unified equirectangular (ERP)representation under fixed projection produces ambiguous matching ev-idence near occlusion boundaries and thin structures. Although existingmethods mitigate this by down-weighting unreliable views, they do notresolve the underlying discrepancy because context formation and cross-view fusion remain tied to rigid fisheye-to-ERP sampling. We presentOmniDS, an iterative depth refinement framework that replaces rigidaggregation by combining dynamic context fusion with consensus-awaremulti-view similarity. A dual-stream encoder pairs a lightweight CNNfor geometric detail with a frozen DINOv3 for semantic priors; their fea-tures are reprojected into ERP space at each refinement step via learnedview weighting and deformable cross-attention with geometric distortionbias. In parallel, a multi-view consensus volume captures global cross-camera agreement through group-wise correlation and feature variance,regularized by a 3D U-Net. For efficient deployment, we distill the dual-stream representation into a single MobileNet-based encoder. OmniDSachieves state-of-the-art performance on the OmniThings, OmniHouse,and Sunny benchmarks while maintaining competitive inference speed.Project page and codes are available here.