VarProtoAD: Variational Prototype-Conditioned Prompting for Zero-Shot Anomaly Detection

Zero-shot anomaly detection (ZSAD) requires detecting andlocalizing diverse defects in unseen categories. Recent vision-languagemodel (VLM) based approaches improve ZSAD via prompt learning, yettheir prompts are often weakly grounded to transferable visual evidence,making them prone to semantic drift under domain shifts and limit-ing the reuse of normal/abnormal primitives. We propose a VariationalPrototype-conditioned prompting framework (VarProtoAD) that ex-plicitly conditions prompt learning on visual prototypes discovered in thefeature space. Specifically, multi-layer patch tokens are ℓ2 -normalized andmodeled with a variational von Mises–Fisher mixture to learn layer-wisenormal and abnormal prototype banks. These prototypes form multi-cluster semantic anchors on the hypersphere and are naturally alignedwith cosine-based VLM matching, where anomalies correspond to direc-tional deviations. Conditioned on these anchors, learnable prompt con-text tokens interact with the prototype banks via confidence-gated cross-attention, producing explicit prototype-conditioned prompts as well asimplicit deviation prompts derived from normal prototypes to bettercover unseen anomaly semantics. Finally, multi-layer image–text simi-larities are fused within a unified framework to jointly produce image-level anomaly scores and pixel-level anomaly maps. Experiments on 15industrial and medical datasets demonstrate strong cross-domain per-formance, indicating that VarProtoAD yields more stable and robustprompt learning.