Break Visual-Linguistic Asymmetry: Unleashing VLM's Cross-Modal Potential for General Face Forgery Detection

Face forgery detection (FFD) is essential in preventing themisuse of diverse generation methods like generative adversarial net-works (GANs) and diffusion models. Although advanced FFD methodshave started to explore the benefits of Vision-Language Model (VLM),they either rely solely on visual modality or optimize visual-linguisticmodalities independently, leaving researches on cross-modal interactionlargely unexplored. We find that visual-linguistic asymmetry in-herent in VLM tends to cause cross-modal misalignment, underminingits ability to discriminate and generalize on visually similar real-forgedfaces. This finding shows that this asymmetry is twofold: (1) cross-modal: visual-linguistic spaces affect VLM’s generalizability oppositely.(2) intra-modal: spaces with different-level semantics have inverse effectson VLM’s discriminability and generalizability. Building upon these in-sights, we propose Asy-Det, a parameter-efficient detector that follows anasymmetry-guided visual-linguistic interaction paradigm, to fully unlockthe cross-modal potential of VLM for reliable FFD. With only 4.41Mtrainable parameters, Asy-Det achieves an image-level AUC of 92.94%on unseen Celeb-DF-v2 dataset.