Preserving Knowledge across Space and Time for Continual Video Deepfake Detection

The continuous emergence of high-quality video deepfakesrequires detectors that continually adapt to new forgery patterns, yetexisting approaches, which are designed for deepfake images, fail to cap-ture video-specific cues. Unlike deepfake images that contain only spatialartifacts, deepfake videos leave distinct evidence along both spatial andtemporal axes, necessitating the separate preservation of each modalityduring sequential model updates. To overcome this limitation, we intro-duce a continual deepfake video detection framework, Modality-SpecificFrequency Distillation (MSFD), that explicitly decomposes video fea-tures into spatial, temporal, and spatiotemporal modalities in the fre-quency domain. This decomposition enables independent preservationof each modality, as different deepfake video types exhibit varying re-liance on spatial and temporal cues across tasks. Furthermore, MSFDadopts a cross-modality decorrelation loss that encourages spatiotempo-ral representations to remain orthogonal to single-modality cues. Exten-sive experiments show that our framework achieves stronger adaptationand preserves performance more effectively than state-of-the-art methodsacross diverse continual deepfake video scenarios.