PADFormer: Pose-agnostic Anomaly Detection from Sparse View Images

Pose-agnostic Anomaly Detection (PAD) remains challeng-ing as anomalies can appear under arbitrary viewpoints, requiring meth-ods to handle significant pose variations. Existing approaches rely oncomplex 3D reconstruction, which are computationally expensive and re-quire extensive multi-view data. We propose PADFormer, a novel image-space approach that leverages Vision Transformer (ViT) to directly re-construct anomaly-free versions of query images while preserving poseinformation. Our key insight is to adapt cross-view masked reconstruc-tion for anomaly detection through training exclusively on normal data,combined with dynamic patch selection and spatial alignment mecha-nisms that enable effective learning from sparse reference views undersignificant pose variations. During inference, we perform multiple for-ward passes with different masking patterns to generate an ensembleof anomaly-free reconstructions, ensuring comprehensive coverage of thequery image. Anomalies are detected by comparing these reconstructionswith the query image. PADFormer achieves state-of-the-art results on thePAD benchmark while maintaining comparable performance on classicfew-shot anomaly detection (FSAD) tasks, demonstrating superior effi-ciency and generalization without requiring 3D reconstruction.