Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses

Feed-forward foundation models for multi-view 3-dimensional(3D) reconstruction have been trained on large-scale datasets of perspec-tive images; when tested on wide field-of-view images, e.g., from a fish-eye camera, their performance degrades. This degradation arises fromchanges in spatial arrangements of pixels induced by the non-linear pro-jection model that maps 3D points onto the 2D image plane. While onemay surmise that training on fisheye images would resolve this problem,there are far fewer fisheye images with ground truth than perspectiveimages, which limits generalization. To enable inference on imagery ex-hibiting high radial distortion, we propose Fisheye3R, a novel adaptationframework that extends these multi-view 3D reconstruction foundationmodels to natively accommodate fisheye inputs without performance re-gression on perspective images. To address the scarcity of fisheye imagesand ground truth, we introduce flexible learning schemes that supportself-supervised adaptation using only unlabeled perspective images andsupervised adaptation without any fisheye training data. Extensive ex-periments across three foundation models, including VGGT, π 3 , and Ma-pAnything, demonstrate that our approach consistently improves camerapose, depth, point map, and field-of-view estimation on fisheye images.Code is available at https://github.com/android-xr/fisheye3r.