Fusion-based Few-Shot Morphing Attack Detection and Fingerprinting

Na Zhang, Shan Jia, Siwei Lyu, Xin LiarXiv 2022 · DOI · arXiv

Researchers have developed a new framework that doesn't just detect if a face image is a "morph" (a blend of two identities), but identifies the specific algorithm used to create it—even with only a handful of training examples. By fusing hardware-level sensor noise (PRNU) with deep-learned "noiseprints," the system achieves over 98% accuracy in identifying novel morphing attacks, significantly outperforming standard deep learning models.

The Challenge of Morphing Attacks

Face Recognition Systems (FRS) are increasingly used at border controls and for secure identity verification. However, they are vulnerable to morphing attacks, where an image is created by blending the faces of two different people. If successful, both individuals can use the same passport to bypass security.

Current Morphing Attack Detection (MAD) methods face two major hurdles:

  1. Generalization: Most detectors are trained on specific, known morphing algorithms. When a new morphing tool is released, these detectors often fail.
  2. Binary Limitation: Existing research focuses on a simple "is it a morph or not?" binary. The authors argue for Morphing Attack Fingerprinting (MAF)—a multiclass approach that identifies the specific "fingerprint" or model used to generate the morph, which is crucial for digital forensics.

The FBC-APL Framework

The authors propose a system called FBC-APL, which combines two sophisticated noise-analysis techniques with a "few-shot" learning architecture. This approach allows the system to learn the characteristics of new attack types from a very small number of examples.

Overview of the few-shot morphing attack fingerprinting framework, showing how a classifier is trained on a training set with very few samples per class to distinguish between various morphing algorithms and bona fide images.

1. Feature Fusion: PRNU + Noiseprint

The core idea is that every image contains "noise" that acts as a signature.

  • PRNU (Photo Response Non-Uniformity): A physical "fingerprint" caused by tiny imperfections in a camera's sensor. Morphing disrupts this pattern.
  • Noiseprint: A CNN-derived noise pattern that captures the "digital DNA" of image processing.
  • FBC (Factorized Bilinear Coding): Instead of just concatenating these features, the authors use FBC to fuse them into a compact, discriminative 2,048-dimensional vector. This highlights the complementary nature of hardware and software noise.

2. Few-Shot Learning: Adaptive Posterior Learning (APL)

To handle new types of attacks where data is scarce, the system uses Adaptive Posterior Learning (APL). This module utilizes an external memory store to save "surprising" examples it hasn't seen before. When a new query comes in, the system compares it to these stored examples using a k-nearest neighbor approach, allowing it to classify a new morphing type after seeing as few as one or five examples.

Datasets and Experimental Setup

The researchers compiled a massive benchmark containing over 20,000 images across five major sources, including standard face research datasets and a newly introduced, challenging Doppelgänger Morphs dataset. This new dataset features "look-alikes" who have no biological connection, simulating high-quality, realistic morphing scenarios. The data includes morphs generated by eight different algorithms, ranging from traditional landmark-based methods (like OpenCV) to advanced AI-generative models (like StyleGAN2).

Key Results

The proposed FBC-APL framework set a new state-of-the-art in few-shot morphing detection:

  • Few-Shot Accuracy: In a 5-shot scenario (learning from just 5 examples of a new attack), the system achieved 98.22% accuracy on a hybrid dataset.
  • Superiority over Baselines: It significantly outperformed standard deep learning models like Xception (43.67%) and ArcFace (29.11%) in the same few-shot settings.
  • Efficiency: The APL module only writes to its memory when it encounters a sample that yields a "surprising" result, reaching peak performance with only about 7 examples per class.

Conclusion and Future Work

By treating MAD as a multiclass fingerprinting problem, the system provides actionable intelligence for investigators tracking the source of fraudulent documents. While highly effective on digital-to-digital morphs, future work will address the "Print and Scan" gap, where the physical process of printing a passport can wash out subtle digital fingerprints. As generative models like Diffusion models evolve, the system's ability to learn from few examples will remain critical for staying ahead of new attack vectors.

Summary updated

Abstract

The vulnerability of face recognition systems to morphing attacks has posed a serious security threat due to the wide adoption of face biometrics in the real world. Most existing morphing attack detection (MAD) methods require a large amount of training data and have only been tested on a few predefined attack models. The lack of good generalization properties, especially in view of the growing interest in developing novel morphing attacks, is a critical limitation with existing MAD research. To address this issue, we propose to extend MAD from supervised learning to few-shot learning and from binary detection to multiclass fingerprinting in this paper. Our technical contributions include: 1) We propose a fusion-based few-shot learning (FSL) method to learn discriminative features that can generalize to unseen morphing attack types from predefined presentation attacks; 2) The proposed FSL based on the fusion of the PRNU model and Noiseprint network is extended from binary MAD to multiclass morphing attack fingerprinting (MAF). 3) We have collected a large-scale database, which contains five face datasets and eight different morphing algorithms, to benchmark the proposed few-shot MAF (FS-MAF) method. Extensive experimental results show the outstanding performance of our fusion-based FS-MAF. The code and data will be publicly available at https://github.com/nz0001na/mad maf.
BibTeX
@misc{zhang2022fusion,
  author = {Na Zhang and Shan Jia and Siwei Lyu and Xin Li},
  title = {{Fusion-based Few-Shot Morphing Attack Detection and Fingerprinting}},
  year = {2022},
  eprint = {2210.15510},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.15510},
}

Loading related content...

Figure