TL;DR
This paper introduces a novel multispectral imaging framework for detecting sophisticated face morphing attacks, a critical vulnerability in border control systems. By leveraging seven distinct spectral bands, their approach significantly outperforms traditional visible-light methods, demonstrating the power of multispectral data in enhancing the accuracy of differential morphing attack detection.
What Problem Does This Paper Address?
Face morphing attacks pose a significant threat to border security and identity verification systems. These attacks involve seamlessly blending two or more face images to create a composite image that can fool both human border guards and automated facial recognition systems (FRS). While existing methods for detecting these attacks, known as Morphing Attack Detection (MAD), have been developed, they primarily rely on standard visible-light imagery. However, modern border control gates are increasingly equipped with multispectral cameras capable of capturing images across various spectral bands. This paper addresses the gap in understanding how this advanced imaging technology can be leveraged to improve the robustness and accuracy of differential morphing attack detection (D-MAD), particularly in the context of ePassport verification.
Methods
The core of this work is the proposed Multispectral Differential Morphing Attack Detection (D-MAD) framework. This framework operates on a differential approach, comparing a reference image (typically from an ePassport) with a live captured image (from a trusted device like an ABC gate). The novelty lies in using multispectral images for the live capture. The framework captures seven different spectral bands (650nm, 710nm, 770nm, 830nm, 890nm, 950nm, and a Whole-Light band) in addition to a visible light image.
The D-MAD framework is structured into four functional units:
- Data Capture: Enrolment (bona fide) images are captured under controlled visible light conditions, while trusted capture images are acquired using a multispectral camera across the specified bands.
- D-MAD Features: The framework is designed to be compatible with existing D-MAD techniques. The authors employ two established methods:
- Deep Features [24]: Based on Arcface features, this method extracts identity-related features.
-
Hierarchical Deep Residual SLERP [26]: This method utilizes texture features and combines them using Spherical Linear Interpolation (SLERP). These feature extraction methods are applied independently to each spectral band.
-
Classification Scores: For each spectral band, the extracted features are fed into a trained classifier to generate matching scores. This results in seven scores per spectral band.
- Fusion: Finally, the scores from all seven spectral bands are fused using a simple sum rule to produce a single fused score, which is then compared against a threshold to make the final decision on whether the image is morphed.
Datasets
The authors introduce a newly created Multispectral Morphing Dataset (MSMD). This dataset comprises:
- Visible Images: Collected from 143 unique subjects (86 male, 57 female) across two sessions, with session-1 used for enrolment and session-2 for trusted capture.
- Multispectral Images: Also collected from the same 143 subjects, using a multispectral camera capturing seven spectral bands.
- Morphing Images: Generated using visible images from session-1, employing two methods: landmark-based [10] and MIPGAN-2 [31].
The dataset is divided into training (78 subjects) and testing (65 subjects) sets to ensure a disjoint morphing dataset. The MSMD dataset is made publicly available for research purposes.
Key Insights and Ideas
- Multispectral Data Unlocks New Detection Capabilities: The study strongly suggests that multispectral imaging, by capturing information beyond the visible spectrum, provides complementary data that significantly enhances the ability to detect morphing attacks.
- Spectral Bands Offer Varying Information: Different spectral bands reveal distinct aspects of facial appearance due to varying skin reflectance. This variation means some bands are more sensitive to morphing artifacts than others.
- Fusion is Key: While individual spectral bands show improvement over visible light, the true power emerges when information from multiple bands is fused, leading to a more robust detection system.
- Morphing Generation Method Matters: The study highlights that the method used to generate morphs (e.g., MIPGAN-2 vs. landmark-based) influences detection difficulty, with MIPGAN-2 generated morphs being generally easier to detect.
- Deep Features vs. Texture Features: The performance of D-MAD techniques varies across spectral bands, with texture-based methods showing more sensitivity to spectral variations, while identity-based deep features exhibit more consistent performance.
Key Results
- Superior Performance with Multispectral Data: The proposed multispectral D-MAD framework consistently outperformed visible-light-based methods across both evaluated D-MAD algorithms and morphing generation techniques.
- Individual Bands Show Improvement: Individual spectral bands generally showed slightly better detection accuracy compared to visible images alone.
- Visible Spectral Bands are Informative: The visible range bands (650nm and 710nm) showed the lowest Detection Equal Error Rate (D-EER) for landmark-based morphing, likely due to the rich texture information they capture, consistent with visible-only imaging.
- Near-Infrared Bands Offer Complementary Information: While NIR bands (830nm, 890nm, 950nm) sometimes showed slightly higher D-EER for landmark-based morphing, they contribute valuable information when fused.
- MIPGAN-2 Morphing is More Detectable: MIPGAN-2 generated morphing images were found to be easier to detect than landmark-based morphs, a finding consistent across both visible and multispectral imaging.
- Hierarchical Deep Residual SLERP [26] Outperforms: The D-MAD method based on Hierarchical Deep Residual SLERP [26] generally showed superior performance compared to the Deep Features [24] method, especially in multispectral settings.
Limitations and Future Work
- Limitations:
- The study acknowledges that the variation in D-EER (%) was not entirely consistent across the two D-MAD techniques.
- The performance of the Hierarchical Deep Residual SLERP [26] on the Whole-Light (WL) image showed degraded performance compared to VIS spectral bands, possibly because its backbone model was trained primarily on VIS images.
- Future Work:
- Extend the work to include different print and scan sources for morphing image generation.
- Investigate the use of different light sources for multispectral image capture.
Summary updated