Multi-modal Document Presentation Attack Detection With Forensics Trace Disentanglement

Changsheng Chen, Yongyi Deng, Liangwei Lin, Zitong Yu, Zhimao Lai

Abstract

Document Presentation Attack Detection (DPAD) is an important measure in protecting the authenticity of a document image. However, recent DPAD methods demand additional resources, such as manual effort in collecting additional data or knowing the parameters of acquisition devices. This work proposes a DPAD method based on multi-modal disentangled traces (MMDT) without the above drawbacks. We first disentangle the recaptured traces by a self-supervised disentanglement and synthesis network to enhance the generalization capacity in document images with different contents and layouts. Then, unlike the existing DPAD approaches that rely only on data in the RGB domain, we propose to explicitly employ the disentangled recaptured traces as new modalities in the transformer backbone through adaptive multi-modal adapters to fuse RGB/trace features efficiently. Visualization of the disentangled traces confirms the effectiveness of the proposed method in different document contents. Extensive experiments on three benchmark datasets demonstrate the superiority of our MMDT method on representing forensic traces of recapturing distortion.

Introduction

Document Presentation Attack Detection (DPAD) serves as a critical security layer for digital identity verification. When users upload document images (e.g., ID cards or transcripts), DPAD systems must distinguish between genuine originals and "recaptures"—photos of printouts or screens. Recapturing is a common technique used by attackers to mask digital forgeries. However, detecting these attacks is challenging due to the diversity of document layouts and the varying quality of acquisition devices. Existing methods often struggle to generalize across different domains and frequently require extensive manual labeling or specific hardware parameters, limiting their real-world applicability.

The MMDT Framework

To address these challenges, we introduce the Multi-modal Disentangled Traces (MMDT) framework. The core innovation lies in isolating forensic traces from the document's semantic content, allowing the model to focus on the artifacts of the recapture process rather than the text or logos.

The MMDT framework consists of a self-supervised forensics trace disentanglement and synthesis network, followed by a multi-modal DPAD network utilizing a transformer backbone.

The framework operates in two primary stages:

  1. Forensics Trace Disentanglement: A self-supervised encoder-decoder network decomposes an input image into two forensic components: Blur Content (C), which captures lens and sensor degradation, and Texture Traces (T), which captures printer-specific patterns like halftoning. A synthesis network is employed to "swap" these traces between images, enabling the model to learn these characteristics without explicit ground-truth labels.
  2. Multi-modal DPAD Network: Instead of relying solely on RGB data, the system treats the disentangled Blur and Texture components as additional modalities. These are fed into a Vision Transformer (ViT) backbone, where Adaptive Multi-modal Adapters (AMA) efficiently fuse the features to perform the final classification.

Visualizing Forensic Traces

The effectiveness of the disentanglement process is evident when examining the isolated components. By separating the "forensic noise" from the "semantic content," the model gains robustness against unseen document types. As shown in the visualization below, the network successfully isolates halftone patterns and edge blurs without being confused by the text on the cards.

Visualization of the disentanglement results across various document types, demonstrating the successful separation of blur content (C) and texture traces (T).

Experimental Evaluation

We evaluated MMDT on three benchmark datasets: RSCID (student IDs), RDID162 (diverse document templates), and a newly introduced RSCID (L) dataset. The latter contains low-quality samples with poor lighting and motion blur to simulate realistic "in-the-wild" conditions.

Key Results

  • State-of-the-Art Performance: MMDT significantly outperformed existing methods, particularly in cross-domain scenarios. On the "ALL" dataset protocol, it achieved a 7.97% improvement in AUC and a 7.22 percentage point reduction in EER compared to previous top-performing models.
  • Robustness: The model maintained high accuracy on the RSCID (L) dataset, proving its resilience to environmental factors like glare and low light.
  • Generalization: By focusing on forensic traces rather than document content, the model generalizes better to new, unseen document layouts.

Conclusion

The MMDT method provides a robust, self-supervised solution for document presentation attack detection. By treating forensic traces as distinct modalities, the framework achieves superior detection accuracy without requiring manual labeling or device-specific knowledge. Future work will explore applying this disentanglement approach to detect digital forgeries, potentially creating a unified toolkit for document forensics.

Summary updated

Abstract

Document Presentation Attack Detection (DPAD) is an important measure in protecting the authenticity of a document image. However, recent DPAD methods demand additional resources, such as manual effort in collecting additional data or knowing the parameters of acquisition devices. This work proposes a DPAD method based on multi-modal disentangled traces (MMDT) without the above drawbacks. We first disentangle the recaptured traces by a self-supervised disentanglement and synthesis network to enhance the generalization capacity in document images with different contents and layouts. Then, unlike the existing DPAD approaches that rely only on data in the RGB domain, we propose to explicitly employ the disentangled recaptured traces as new modalities in the transformer backbone through adaptive multi-modal adapters to fuse RGB/trace features efficiently. Visualization of the disentangled traces confirms the effectiveness of the proposed method in different document contents. Extensive experiments on three benchmark datasets demonstrate the superiority of our MMDT method on representing forensic traces of recapturing distortion.
BibTeX
@inproceedings{chen2024multi,
  author = {Changsheng Chen and Yongyi Deng and Liangwei Lin and Zitong Yu and Zhimao Lai},
  title = {{Multi-modal Document Presentation Attack Detection With Forensics Trace Disentanglement}},
  booktitle = {ICME},
  year = {2024},
  url = {https://openreview.net/forum?id=MnOOJ7JxBV},
}
Figure