Evidence Triangulation for Multimodal Fact-Checking in the Wild
Abstract
The proliferation of multimedia content on social platformshas fueled multimodal misinformation, where images are used to rein-force false claims. Consequently, Multimodal Fact-Checking (MFC) hasemerged as an increasingly important research area. However, currentprogress is hindered by a reliance on synthetic training data and cu-rated benchmarks that fail to capture the complexity of in-the-wild data.Furthermore, existing detection models rely on restricted intra-modalityconsistency or unconstrained all-to-all fusion, failing to capture nuancedrelations between posts and external evidence. To address these limi-tations, we introduce X-POSE, a benchmark of real-world, community-annotated multimodal posts from X (formerly Twitter), augmented withfull-length news articles retrieved via VLM-optimized search. Addition-ally, we propose TRENT, a novel MFC model that performs evidence tri-angulation using three parallel cross-attention streams alongside a rela-tional fusion mechanism that explicitly models entailment and contradic-tion. Extensive evaluations demonstrate that TRENT consistently out-performs state-of-the-art specialized models and commercial VLMs. Thecode, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence-triangulation.