Texture, Shape and Order Matter: A New Transformer Design for Sequential DeepFake Detection

Yunfei Li, Yuezun Li, Xin Wang, Baoyuan Wu, Jiaran Zhou, Junyu Dong

Abstract

TSOM (Texture, Shape, and Order of Manipulations) is a novel Transformer-based framework designed to decode the step-by-step history of a DeepFake. By prioritizing fine-grained textures, spatial shape priors, and predicting the manipulation sequence in reverse order, it sets a new state-of-the-art for sequential DeepFake detection.

Introduction

Most DeepFake forensics are binary: they tell you if an image is "Real" or "Fake." However, as face-editing tools become more sophisticated, creators often apply multiple edits in a sequence (e.g., first changing the hair, then the eyes, then the lips). Sequential DeepFake Detection (SDD) is the emerging task of recovering this specific order.

Sequential editing examples showing how multiple manipulations (e.g., eye then nose, or eyeglasses then age) alter the face.

Existing methods typically treat this as a standard image-to-sequence problem using vanilla Transformers. However, these approaches often ignore subtle textures, lack spatial guidance, and struggle with the "Onion" problem: the last manipulation often obscures the traces of the first one. Predicting in chronological order is like trying to describe the core of an onion without peeling the outer layers first.

Methodology

The authors propose TSOM, which introduces four key innovations to address the limitations of current sequential detection methods.

Comparison of standard DeepFake detection vs. Sequential DeepFake detection, and the proposed TSOM architecture.
  • Diversiform Pixel Difference Attention (DPDA): Instead of standard self-attention, this module calculates differences between adjacent pixels (central, angular, and radial). This forces the model to focus on the fine-grained local textures where DeepFake artifacts usually hide.
  • Multi-source Cross-attention (MSCA): This module acts as a fusion hub, integrating global spatial features, the new texture-aware features, and the sequence of manipulation labels to find deep correlations.
  • Shape-guided Gaussian Mapping (SGM): Using a Variational Auto-encoder (VAE), the model learns the typical "shapes" and locations of facial manipulations. These act as spatial priors, telling the attention mechanism exactly where an edit is likely to have occurred.
  • Inverted Order Prediction (IOP): This is the paper's most clever insight. The authors found that predicting the sequence backward (from the most recent edit to the first) significantly improves accuracy. Since the last edit hasn't been covered up by anything else, it’s the easiest to detect; once identified, the model can "peel it away" to find the earlier, subtler edits.

Experimental Results

The model was evaluated on the primary benchmark for this field: the Sequential DeepFake Dataset. This contains two tracks: Facial-components (35,166 images with 28 types of component-level edits) and Facial-attributes (49,920 images with 26 types of attribute-level edits). Additionally, FaceForensics++ (FF++) was used to test adaptability to standard one-step detection.

TSOM consistently outperformed existing state-of-the-art methods like SeqFake-Former and MMNet. On the ResNet-50 backbone, TSOM achieved an average Fixed Accuracy (FACC) of 72.67% and an Adaptive Accuracy (AACC) of 55.44%, outperforming the previous best by a significant margin.

A key finding is the power of reversal: adding Inverted Order Prediction (IOP) to the baseline boosted FACC by over 4%. This acknowledges the physical reality of how digital artifacts are layered. Furthermore, TSOM proved much more resilient to common image "attacks" like Gaussian noise, JPEG compression, and color jittering compared to other models.

Conclusion

While TSOM sets a new state-of-the-art, its multi-branch architecture and VAE-based shape priors increase training complexity. Performance on "in-the-wild" fakes with arbitrary, non-categorical edits remains an open question. Future work may explore more diverse texture extraction and refined shape-guided priors to further improve traceability in digital forensics.

Summary updated

Abstract

Sequential DeepFake detection is an emerging task that predicts the manipulation sequence in order. Existing methods typically formulate it as an image-to-sequence problem, employing conventional Transformer architectures. However, these methods lack dedicated design and consequently result in limited performance. As such, this paper describes a new Transformer design, called TSOM, by exploring three perspectives: Texture, Shape, and Order of Manipulations. Our method features four major improvements: we describe a new texture-aware branch that effectively captures subtle manipulation traces with a Diversiform Pixel Difference Attention module. Then we introduce a Multi-source Cross-attention module to seek deep correlations among spatial and sequential features, enabling effective modeling of complex manipulation traces. To further enhance the cross-attention, we describe a Shape-guided Gaussian mapping strategy, providing initial priors of the manipulation shape. Finally, observing that the subsequent manipulation in a sequence may influence traces left in the preceding one, we intriguingly invert the prediction order from forward to backward, leading to notable gains as expected. Extensive experimental results demonstrate that our method outperforms others by a large margin, highlighting the superiority of our method.
BibTeX
@inproceedings{li2025texture,
  author = {Yunfei Li and Yuezun Li and Xin Wang and Baoyuan Wu and Jiaran Zhou and Junyu Dong},
  title = {{Texture, Shape and Order Matter: A New Transformer Design for Sequential DeepFake Detection}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00030},
}
Figure