Abstract
3D face recognition has shown its potential in many application scenarios. Among numerous 3D face recognition methods, deep-learning-based methods have developed vigorously in recent years. In this paper, an end-to-end deep learning network entitled Sur3dNet-Face for point-cloud-based 3D face recognition is proposed. The network uses PointNet as the backbone, which is a successful point cloud classification solution but does not work properly in face recognition. Supplemented with modifications in network architecture and a few-data guided learning framework based on Gaussian process morphable model, the backbone is successfully modified for 3D face recognition. Different from existing methods training with a large amount of data in multiple datasets, our method uses the Spring2003 subset of FRGC v2.0 for training which contains only 943 facial scans, and the network is well trained with the guidance of such a small amount of real data. Without fine-tuning on the test set, the Rank-1 Recognition Rate (RR1) is achieved as follows: 98.85% on FRGC v2.0 dataset and 99.33% on Bosphorus dataset, which proves the effectiveness and the potentiality of our method.
Introduction
Deep learning has revolutionized face recognition, but its success typically relies on massive datasets. In the 3D domain, high-quality scans are significantly scarcer than 2D images. Many state-of-the-art models are trained on tens of thousands of scans, making it difficult to discern whether performance stems from architectural superiority or simply data volume. Furthermore, practical applications often face scenarios where available training data is much smaller than the eventual test set.
This research addresses two primary challenges: the scarcity of 3D facial data and the inherent limitations of standard point cloud architectures like PointNet for face recognition tasks. PointNet often struggles with faces because it tends to sample unstable edge points and loses crucial physical scale information during normalization.
Proposed Methodology
Sur3dNet-Face Architecture
The proposed Sur3dNet-Face is an end-to-end network that processes raw point cloud coordinates directly. By operating in 3D space rather than projecting data into 2D depth maps, the network maintains rotation and transformation invariance. Crucially, the model preserves the original millimeter scale of the scans, recognizing that absolute facial size is a vital biometric feature often lost in standard $(-1, 1)$ normalization.
Dithering Farthest Point Sampling (DFPS)
Standard Farthest Point Sampling (FPS) frequently selects points at the boundaries of a scan. These edge points are often noisy and vary significantly with pose, leading to instability. To counter this, the authors introduced Dithering Farthest Point Sampling (DFPS). DFPS implements a "valid radius" to focus sampling on central facial features and utilizes surface curvature (eigenvalues) to "dither" the sampling, enhancing robustness against noise.
Few-Data Guided Learning Framework
To overcome data scarcity, the framework employs a Gaussian Process Morphable Model (GPMM) to generate thousands of synthetic identities. Rather than generating random faces, the process is "guided" by a small set of real data (943 scans from the FRGC v2.0 Spring2003 subset). Real-world expression coefficients are extracted and applied to synthetic identities, ensuring the training data reflects realistic facial movements. During training, the model is validated against a small set of real data to ensure generalization beyond the synthetic generator.
Experimental Results
The model was evaluated on the FRGC v2.0 and Bosphorus datasets. Despite the limited real-world training data, Sur3dNet-Face achieved a Rank-1 Recognition Rate (RR1) of 98.85% on FRGC v2.0 and 99.33% on the Bosphorus dataset. The high performance on Bosphorus without fine-tuning demonstrates the model's strong generalization capabilities across different populations and scanning conditions. Additionally, the system is computationally efficient, requiring only 0.105s for feature extraction on a single GPU.
Conclusion
Sur3dNet-Face demonstrates that robust 3D face recognition can be achieved with limited real-world data through intelligent architectural modifications and guided synthetic data generation. By addressing the "edge problem" in point cloud sampling and preserving physical scale, the network successfully adapts PointNet for biometric tasks. Future work will focus on improving robustness to extreme poses and occlusions by incorporating more diverse "angled" faces and simulated occlusions into the data generation process.
Summary updated