TL;DR
This paper introduces a novel framework for detecting morphed faces by disentangling facial appearance and landmark information. By maximizing mutual information between these disentangled representations, the proposed method achieves state-of-the-art performance on multiple morph detection benchmarks.
What Problem Does This Paper Address?
Biometric systems, particularly those relying on facial recognition, are increasingly vulnerable to morphing attacks. These attacks create synthetic face images that can fool recognition systems by matching the characteristics of two or more individuals. While existing methods often focus on detecting morphs based on either appearance or landmark changes, this paper addresses the need for a more robust approach that considers both aspects simultaneously. The challenge lies in effectively disentangling these two crucial facial features and leveraging their differences to reliably distinguish between genuine and morphed images.
Methods
The core of the proposed framework lies in its novel disentanglement network. This network is trained using "triplets" of face images. Each triplet consists of two real face images from different identities. An intermediate "morph" image is then generated by taking the appearance from one image and the landmarks from the other. This process forces the network to learn to separate and represent appearance and landmark information independently.
Specifically, the framework employs two networks: an appearance network ($a(\cdot)$) and a landmark network ($g(\cdot)$). These networks map input face images to their respective disentangled representations. A third network ($f(\cdot)$) then combines these representations to generate a face ID representation. The training process involves several loss functions:
- Appearance-preserving loss ($L_a$): Encourages the appearance representations of genuine and intermediate images to be similar.
- Landmark-preserving loss ($L_g$): Ensures that the landmark representations of genuine and intermediate images are close.
- ID loss ($L_{id}$): Aims to preserve sufficient classification information in the face ID representation for distinguishing individuals.
- Contrastive loss: Further refines the disentangled representations by maximizing mutual information between genuine pairs (real images of the same subject) and minimizing it for imposter pairs (real vs. morphed, or different subjects). This is achieved by defining genuine and imposter sets and using information-theoretic losses ($L_a^I$ and $L_g^I$) to encourage separation in the embedding space.
The final differential morph detection decision is made by considering the distances between the disentangled representations (landmark, appearance, and ID) of a probe image and a trusted image.
Datasets
The proposed framework is evaluated on three publicly available morph datasets:
- MorGAN: Generated using a GAN-based framework.
- VISAPP17-Splicing-Selected: A subset of the VISAPP17 dataset, generated using warping and alpha-blending techniques.
- AMSL Face Morph Image Dataset: Created using genuine and morphed face images, with specific constraints on morph generation (e.g., same category morphing).
The model is trained on the CASIA-WebFace dataset.
Key Insights and Ideas
- Disentanglement is Key: The paper strongly advocates for disentangling appearance and landmark information as a powerful strategy for robust morph detection. This allows for a more nuanced analysis of morphing artifacts.
- Triplets for Disentanglement: The use of specially constructed image triplets (real appearance + real landmarks, real appearance + morphed landmarks, etc.) is an effective way to train a network to separate these two crucial facial attributes.
- Mutual Information Maximization: Leveraging mutual information maximization, particularly in a contrastive learning setting, proves crucial for pushing genuine pairs closer and imposter pairs further apart in the learned embedding space.
- Complementary Representations: The paper highlights that while appearance and landmark representations are disentangled, they are complementary and together provide a more comprehensive signal for morph detection.
Key Results
The proposed framework achieves state-of-the-art performance across all three tested datasets (MorGAN, VISAPP17, and AMSL). For instance, on the MorGAN dataset, the proposed method achieves a D-EER of 8.75%, significantly outperforming existing methods. The paper also demonstrates strong cross-dataset performance, indicating the generalizability of the learned representations. The results show that the disentanglement approach is particularly effective for detecting morphs generated by different methodologies.
Limitations and Future Work
The paper acknowledges that the performance can be influenced by the variance of the Gaussian distribution used to perturb landmarks during training ($\delta_i$). The authors also suggest that further research could explore more sophisticated disentanglement techniques and investigate the impact of different morph generation methods on the detection performance. The paper also implicitly suggests that exploring the use of prior knowledge about trusted images could further enhance detection accuracy, as hinted by the "Ours*" experiment in Table 3.
Summary updated