Multi-Modal Document Presentation Attack Detection with Forensics Trace Disentanglement

Changsheng Chen, Yongyi Deng, Liangwei Lin, Zitong Yu, Zhimao Lai

TL;DR

Researchers have developed a new method for detecting recaptured document images (like a photo of a printed ID card) by "disentangling" forensic traces—specifically blur and halftone textures—from the document's actual content. By treating these extracted traces as separate data streams in a multi-modal transformer model, their approach achieves significantly better detection accuracy across different document types and lighting conditions without requiring manual data labeling or prior knowledge of the cameras used.

What Problem Does This Paper Address?

Document Presentation Attack Detection (DPAD) is the digital "bouncers" of the internet. When you upload an ID card for verification, DPAD systems check if the image is a genuine original or a "recapture" (e.g., a photo of a printout or a screen). Attackers often use recapturing to hide traces of digital tampering.

The challenge is that recaptured images are incredibly diverse. Existing detection methods often struggle to generalize; a model trained on high-quality scans might fail on a blurry smartphone photo. Furthermore, many current state-of-the-art methods require "insider info," such as the specific resolution of the printer or camera used, or massive amounts of manually labeled data—resources that aren't always available in real-world scenarios.

Methods

The authors propose a framework called Multi-modal Disentangled Traces (MMDT). It consists of two main stages:

  1. Forensics Trace Disentanglement: Using a self-supervised encoder-decoder network, the system separates an input image into two distinct forensic components:
  2. Blur Content (C): Captures the degradation of text and characters caused by lenses and sensors.
  3. Texture Traces (T): Captures fine-grained patterns like "halftoning" (the tiny dots created by printers). Because there is no "ground truth" for what these traces look like, the authors use a Self-supervised Synthesis Network. This network learns to "swap" traces between images, ensuring the model learns to identify the characteristics of the recapture process rather than the content of the document.

  4. Multi-modal DPAD Network: Instead of just looking at the standard RGB image, the researchers feed the RGB data, the Blur component, and the Texture component into a Vision Transformer (ViT) backbone. They use Adaptive Multi-modal Adapters (AMA)—lightweight modules that help the transformer efficiently fuse these three different "views" of the document to make a final "Genuine vs. Recaptured" decision.

Datasets

The study evaluated the model on three key benchmarks:

  • RSCID: A standard dataset of student ID cards.
  • RDID162: A diverse dataset containing 162 different document templates (transcripts, licenses, etc.).
  • RSCID (L) [New]: The authors introduced this extension to the RSCID dataset, adding 3,738 low-quality samples featuring poor lighting, glares, and motion blur to simulate more realistic, "in-the-wild" conditions.

Key Insights and Ideas

  • Traces vs. Content: The core "aha moment" is that the document's content (the text and logos) is actually a distraction for a recapture detector. By explicitly separating the "forensic noise" from the "semantic content," the model becomes much more robust when it encounters a document type it has never seen before.
  • Self-Supervision is Key: Since we can't easily label "how much halftone" is in a photo, using a synthesis network to transfer traces between images allows the model to teach itself what a recapture looks like in a mathematically rigorous way.
  • Modality Matters: Treating forensic traces as separate "modalities" (like audio and video) rather than just extra image filters allows the transformer architecture to attend to specific distortions that might be invisible in a standard RGB photo.

Key Results

The MMDT method significantly outperformed existing state-of-the-art (SOTA) approaches, especially in "cross-domain" scenarios (training on one type of document and testing on another):

  • Superior Accuracy: On the combined "ALL" dataset protocol, MMDT improved the Area Under the Curve (AUC) and reduced the Equal Error Rate (EER) by 7.97% and 7.22 percentage points, respectively, compared to previous top-performing methods.
  • Robustness: The model maintained high performance even on the new RSCID (L) dataset, proving it can handle the messy reality of low-light and high-glare smartphone photography.
  • Visualization: The authors provided visual proof that their disentanglement network successfully isolated halftone patterns and edge blurs without being confused by the text on the cards.

Limitations and Future Work

While the disentanglement is effective for detection, the authors noted that the reconstruction of genuine images isn't perfect; the synthesized images sometimes lack the extreme sharpness of an original digital file.

Looking ahead, the researchers plan to extend this "disentanglement" philosophy to other forensic tasks. Specifically, they want to isolate traces of digital forgery, such as artifacts left behind when someone edits the font or edges of a document image. This could lead to a universal "forensic toolkit" that can spot both physical recaptures and digital manipulations in one go.

Summary updated

Abstract

Document Presentation Attack Detection (DPAD) is an important measure in protecting the authenticity of a document image. However, recent DPAD methods demand additional resources, such as manual effort in collecting additional data or knowing the parameters of acquisition devices. This work proposes a DPAD method based on multi-modal disentangled traces (MMDT) without the above drawbacks. We first disentangle the recaptured traces by a self-supervised disentanglement and synthesis network to enhance the generalization capacity in document images with different contents and layouts. Then, unlike the existing DPAD approaches that rely only on data in the RGB domain, we propose to explicitly employ the disentangled recaptured traces as new modalities in the transformer backbone through adaptive multi-modal adapters to fuse RGB/trace features efficiently. Visualization of the disentangled traces confirms the effectiveness of the proposed method in different document contents. Extensive experiments on three benchmark datasets demonstrate the superiority of our MMDT method on representing forensic traces of recapturing distortion.
BibTeX
@inproceedings{chen2024multi,
  author = {Changsheng Chen and Yongyi Deng and Liangwei Lin and Zitong Yu and Zhimao Lai},
  title = {{Multi-Modal Document Presentation Attack Detection with Forensics Trace Disentanglement}},
  booktitle = {ICME},
  year = {2024},
  url = {https://openreview.net/forum?id=MnOOJ7JxBV},
}

Loading related content...

Figure