Abstract
Recent years have witnessed a growing interest in developing methods for 3D face recognition. However, 3D scans often suffer from the problems of missing parts, large facial expressions, and occlusions. To be useful in real-world applications, a 3D face recognition approach should be able to handle these challenges. In this paper, we propose a novel general approach to deal with the 3D face recognition problem by making use of multiple keypoint descriptors (MKD) and the sparse representation-based classification (SRC). We call the proposed method 3DMKDSRC for short. Specifically, with 3DMKDSRC, each 3D face scan is represented as a set of descriptor vectors extracted from keypoints by meshSIFT. Descriptor vectors of gallery samples form the gallery dictionary. Given a probe 3D face scan, its descriptors are extracted at first and then its identity can be determined by using a multitask SRC. The proposed 3DMKDSRC approach does not require the pre-alignment between two face scans and is quite robust to the problems of missing data, occlusions and expressions.
Introduction
While 3D facial scans offer significantly more geometric detail than traditional 2D images, they present unique processing challenges. Most "holistic" 3D recognition methods—which analyze the face as a single, unified surface—depend heavily on precise alignment (registration) via algorithms like Iterative Closest Point (ICP). These methods often fail when confronted with non-rigid deformations caused by facial expressions or when data is missing due to occlusions such as hair, glasses, or hands.
To address these limitations, we introduce 3DMKDSRC (3D Multiple Keypoint Descriptors and Sparse Representation-based Classification). This framework moves away from global alignment and landmark-dependent measurements, instead leveraging a dense set of local descriptors and a robust multi-task sparse coding scheme.
The 3DMKDSRC Methodology
The 3DMKDSRC pipeline is divided into two primary stages: local feature extraction and sparse classification.
1. Feature Extraction via meshSIFT
The system treats the 3D mesh as a scale-space, identifying "keypoints" at various scales by searching for extrema in mean curvature. This approach ensures that the features captured are stable across different resolutions and orientations.
For each detected keypoint, a descriptor vector is calculated based on two geometric properties:
- Shape Index: This characterizes the local "type" of the surface (e.g., spherical cap, ridge, or rut).
- Slant Angles: These represent the orientation of the surface normal across nine local sub-regions surrounding the keypoint.
By pooling these descriptors, the face is represented as a collection of local geometric "signatures" rather than a single rigid shape. This allows the system to remain robust even if parts of the face are missing or distorted.
2. Multi-task Sparse Representation-based Classification (SRC)
Once features are extracted, the recognition problem is framed as a sparse reconstruction task. All descriptors from the "gallery" (the database of known individuals) are compiled into a large dictionary. For an unknown "probe" scan, the algorithm attempts to represent each of its descriptors as a sparse linear combination of the gallery descriptors.
To maintain efficiency and accuracy, the system employs "Dictionary Shrinking" to pre-select likely matches. Furthermore, a Sparsity Criterion is used to filter out non-discriminative keypoints—those that match many subjects equally poorly—ensuring that only high-quality, unique features contribute to the final identity decision.
Robustness and Key Insights
A defining characteristic of 3DMKDSRC is its alignment-free nature. By relying on local descriptors, the system bypasses the computationally expensive and error-prone ICP registration step.
Furthermore, the method relies on a consensus of keypoints rather than a few specific landmarks like the nose tip or eye corners. If a specific region of the face is occluded or distorted by an extreme expression, the hundreds of other stable keypoints (e.g., on the forehead or chin) provide sufficient information for accurate identification.
Experimental Evaluation
The 3DMKDSRC method was rigorously tested on three benchmark databases:
- Bosphorus: Known for its variety of expressions and occlusions, the method achieved a 95.03% Rank-1 recognition rate.
- GavabDB: Despite the high noise levels in this dataset, the system reached 100% accuracy on neutral scans and 92.62% overall.
- FRGC2.0: On this large-scale standard benchmark, 3DMKDSRC achieved 89.29% accuracy, outperforming the baseline meshSIFT without requiring any 3D preprocessing like hole-filling.
Conclusion
3DMKDSRC provides a robust, general-purpose solution for 3D face recognition that excels in real-world conditions involving expressions and occlusions. While the computational cost of $l_1$-minimization remains a consideration for real-time applications, the method's ability to function without complex pre-alignment or landmark detection marks a significant advancement in the field.
Summary updated