Differential 3D Facial Recognition: Adding 3D to Your State-of-the-Art 2D Method

J. Matías Di Martino, Fernando Suzacq, Mauricio Delbracio, Qiang Qiu, Guillermo Sapiro

Abstract: Active illumination is a prominent complement to enhance 2D face recognition and make it more robust, e.g., to spoofing attacks and low-light conditions. In the present work we show that it is possible to adopt active illumination to enhance state-of-the-art 2D face recognition approaches with 3D features, while bypassing the complicated task of 3D reconstruction. The key idea is to project over the test face a high spatial frequency pattern, which allows us to simultaneously recover real 3D information plus a standard 2D facial image. Therefore, state-of-the-art 2D face recognition solution can be transparently applied, while from the high frequency component of the input image, complementary 3D facial features are extracted. Experimental results on ND-2006 dataset show that the proposed ideas can significantly boost face recognition performance and dramatically improve the robustness to spoofing attacks.

Introduction

While 2D face recognition has reached human-level performance in controlled settings, it still struggles with "in-the-wild" challenges such as extreme poses, poor lighting, and spoofing. A simple high-resolution photo or tablet screen can often trick a 2D system because it cannot distinguish between a flat surface and a real 3D face.

Figure 3 illustrates how different 3D surfaces (a, b, c) can result in nearly identical 2D projections, highlighting the inherent ambiguity and vulnerability of 2D-only systems to spoofing.

True 3D face recognition exists but usually requires expensive specialized hardware or complex reconstruction algorithms. This paper proposes a middle ground: "Differential 3D Facial Recognition," which extracts local 3D depth gradients (slopes) to verify identity and authenticity without needing a full, perfect 3D mesh.

Methods

The core innovation is the use of Active Illumination combined with Spectral Decomposition. The system projects a high-frequency light pattern (thin horizontal stripes) onto the subject's face. Modern smartphone cameras capture images at high resolutions, but most face recognition models downsample this significantly. The authors exploit this "wasted" high-frequency space to embed the 3D pattern.

The proposed pipeline decomposes a single 2D input image into texture and depth components to extract complementary features for recognition and identity verification.

Decomposition and Architecture

A deep neural network (DNN) splits the captured image into two components:

  1. Texture (2D): A standard low-resolution RGB image where the pattern has been filtered out.
  2. Depth Gradient (3D): A map showing the local deformation of the stripes, representing the 3D "slope" of the face.

Because the 2D and 3D information is separated, the system is modular. Any existing 2D model (like VGG-Face or ArcFace) can be used for the texture branch, while a custom depth feature extractor handles the 3D branch. The system also utilizes an Anisotropic Distance function to compare embeddings; if 3D gradients differ wildly (as between a real face and a flat photo), a massive penalty is applied to block spoofing attempts.

Key Insights

  • Differential > Absolute: Knowing the change in depth (the gradient) across the face provides enough unique geometric information to distinguish individuals without needing absolute distance measurements.
  • Bypassing Reconstruction: By focusing only on local gradients, the authors bypass the mathematically "ill-posed" problem of full 3D reconstruction, which often fails at sharp edges like the nose.
  • Frequency Domain Exploitation: Texture and 3D patterns reside in different parts of the image's frequency spectrum, allowing for the recovery of a clean 2D image despite the projected stripes.

Experimental Results

The researchers evaluated their method using the ND-2006 dataset for training and the CASIA Anti-spoofing dataset for robustness testing. Key findings include:

  • Accuracy Boost: On the ND-2006 dataset, adding the 3D branch improved Rank-1 recognition accuracy from 78.5% (2D only) to 90.6% (2D+3D).
  • Spoofing Defense: The system proved incredibly resilient. The Average Classification Error Rate (ACER) dropped from 38.9% for the 2D baseline to just 3.9% with the differential 3D method.
  • ArcFace Integration: When combined with ArcFace, the system maintained 100% accuracy while improving spoofing detection from total failure (0%) to nearly 85% at very low false-positive rates.

Conclusion and Future Work

Differential 3D facial recognition provides a practical path to enhancing 2D systems with 3D robustness. While the current method requires a projector, the authors suggest that using Infrared (IR) light would make the pattern invisible to users and more practical for consumer devices. Future work will also address challenges related to sharp depth discontinuities and extremely bright ambient light that can wash out projected patterns.

Summary updated

Abstract

Active illumination is a prominent complement to enhance 2D face recognition and make it more robust, e.g., to spoofing attacks and low-light conditions. In the present work we show that it is possible to adopt active illumination to enhance state-of-the-art 2D face recognition approaches with 3D features, while bypassing the complicated task of 3D reconstruction. The key idea is to project over the test face a high spatial frequency pattern, which allows us to simultaneously recover real 3D information plus a standard 2D facial image. Therefore, state-of-the-art 2D face recognition solution can be transparently applied, while from the high frequency component of the input image, complementary 3D facial features are extracted. Experimental results on ND-2006 dataset show that the proposed ideas can significantly boost face recognition performance and dramatically improve the robustness to spoofing attacks.
BibTeX
@article{martino2020differential,
  author = {J. Matías Di Martino and Fernando Suzacq and Mauricio Delbracio and Qiang Qiu and Guillermo Sapiro},
  title = {{Differential 3D Facial Recognition: Adding 3D to Your State-of-the-Art 2D Method}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2020},
  url = {https://openreview.net/forum?id=GHr2HnJK4},
}
Figure