Abstract
TSOM (Texture, Shape, and Order of Manipulations) is a novel Transformer-based framework designed to decode the step-by-step history of a DeepFake. By prioritizing fine-grained textures, spatial shape priors, and predicting the manipulation sequence in reverse order, it sets a new state-of-the-art for sequential DeepFake detection.
Introduction
Most DeepFake forensics are binary: they tell you if an image is "Real" or "Fake." However, as face-editing tools become more sophisticated, creators often apply multiple edits in a sequence (e.g., first changing the hair, then the eyes, then the lips). Sequential DeepFake Detection (SDD) is the emerging task of recovering this specific order.
Existing methods typically treat this as a standard image-to-sequence problem using vanilla Transformers. However, these approaches often ignore subtle textures, lack spatial guidance, and struggle with the "Onion" problem: the last manipulation often obscures the traces of the first one. Predicting in chronological order is like trying to describe the core of an onion without peeling the outer layers first.
Methodology
The authors propose TSOM, which introduces four key innovations to address the limitations of current sequential detection methods.
- Diversiform Pixel Difference Attention (DPDA): Instead of standard self-attention, this module calculates differences between adjacent pixels (central, angular, and radial). This forces the model to focus on the fine-grained local textures where DeepFake artifacts usually hide.
- Multi-source Cross-attention (MSCA): This module acts as a fusion hub, integrating global spatial features, the new texture-aware features, and the sequence of manipulation labels to find deep correlations.
- Shape-guided Gaussian Mapping (SGM): Using a Variational Auto-encoder (VAE), the model learns the typical "shapes" and locations of facial manipulations. These act as spatial priors, telling the attention mechanism exactly where an edit is likely to have occurred.
- Inverted Order Prediction (IOP): This is the paper's most clever insight. The authors found that predicting the sequence backward (from the most recent edit to the first) significantly improves accuracy. Since the last edit hasn't been covered up by anything else, it’s the easiest to detect; once identified, the model can "peel it away" to find the earlier, subtler edits.
Experimental Results
The model was evaluated on the primary benchmark for this field: the Sequential DeepFake Dataset. This contains two tracks: Facial-components (35,166 images with 28 types of component-level edits) and Facial-attributes (49,920 images with 26 types of attribute-level edits). Additionally, FaceForensics++ (FF++) was used to test adaptability to standard one-step detection.
TSOM consistently outperformed existing state-of-the-art methods like SeqFake-Former and MMNet. On the ResNet-50 backbone, TSOM achieved an average Fixed Accuracy (FACC) of 72.67% and an Adaptive Accuracy (AACC) of 55.44%, outperforming the previous best by a significant margin.
A key finding is the power of reversal: adding Inverted Order Prediction (IOP) to the baseline boosted FACC by over 4%. This acknowledges the physical reality of how digital artifacts are layered. Furthermore, TSOM proved much more resilient to common image "attacks" like Gaussian noise, JPEG compression, and color jittering compared to other models.
Conclusion
While TSOM sets a new state-of-the-art, its multi-branch architecture and VAE-based shape priors increase training complexity. Performance on "in-the-wild" fakes with arbitrary, non-categorical edits remains an open question. Future work may explore more diverse texture extraction and refined shape-guided priors to further improve traceability in digital forensics.
Summary updated