Abstract
The existing face recognition datasets usually lack occlusion samples, which hinders the development of face recognition. Especially during the COVID-19 coronavirus epidemic, wearing a mask has become an effective means of preventing the virus spread. Traditional CNN-based face recognition models trained on existing datasets are almost ineffective for heavy occlusion. To this end, we pioneer a simulated occlusion face recognition dataset. In particular, we first collect a variety of glasses and masks as occlusion, and randomly combine the occlusion attributes (occlusion objects, textures, and colors) to achieve a large number of more realistic occlusion types. We then cover them in the proper position of the face image with the normal occlusion habit. Furthermore, we reasonably combine original normal face images and occluded face images to form our final dataset, termed as Webface-OCC. It covers 804,704 face images of 10,575 subjects, with diverse occlusion types to ensure its diversity and stability. Extensive experiments on public datasets show that the ArcFace retrained by our dataset significantly outperforms the state-of-the-arts.
Introduction
Deep learning has revolutionized face recognition, achieving high accuracy under controlled conditions. However, performance degrades significantly when faces are partially obscured by masks, sunglasses, or other objects. The COVID-19 pandemic underscored this vulnerability, as mask-wearing became a global necessity, rendering many existing recognition systems ineffective.
The primary obstacle to developing robust models is the lack of large-scale, high-quality training data featuring occlusions. Previous attempts to synthesize such data often relied on simplistic methods, such as applying random black or white boxes to face images. These unnatural artifacts fail to capture the complexity of real-world occlusions, leading to models that do not generalize well to actual scenarios.
The Webface-OCC Dataset
To address the limitations of existing datasets, the authors developed Webface-OCC, a massive dataset designed for realistic occlusion simulation. The creation process involves several key steps:
- Occluder Collection: A diverse library of real-world templates was gathered, including various mask types (surgical, N95) and glasses with varying colors and textures.
- Landmark-Based Alignment: Instead of random placement, the system uses 68 facial landmarks (detected via RetinaFace) to align occluders. This ensures that masks and glasses are positioned naturally according to human wearing habits.
- Balanced Composition: To maintain high performance on unoccluded faces, the final dataset features a balanced mix of normal and occluded images for each of the 10,575 subjects.
Experimental Results
The researchers evaluated the impact of training on Webface-OCC by retraining standard models like ArcFace. The results demonstrate a significant leap in robustness:
- Simulated Benchmarks: On the LFW-mask dataset, accuracy soared from 60.86% to 97.08%, a gain of over 36%. Similar improvements were observed across other simulated benchmarks like CFP-FP-mask and AgeDB-30-mask.
- Real-World Performance: When tested on the Real-World Masked Face Recognition Dataset (RMFRD), the retrained ArcFace model improved from 63.22% to 78.25%.
- Baseline Preservation: Crucially, the model's performance on standard, unoccluded faces remained nearly identical (99.01% vs. the original 99.53%), proving that robustness to occlusion does not require sacrificing general accuracy.
Discussion and Conclusion
The success of Webface-OCC highlights that realistic simulation is critical for training robust neural networks. By moving beyond simple geometric blocks to aligned, textured occluders, the network learns to focus on informative facial regions like the eyes and forehead.
While a performance gap remains between simulated and real-world masked faces—partly due to the extreme diversity of real-world occlusions and the intentional disguises used by some subjects—Webface-OCC provides a vital benchmark for the community. Future work will focus on closing this "sim-to-real" gap and developing universal algorithms capable of handling any form of visual obstruction.
Webface-OCC is available at https://github.com/Baojin-Huang/Webface-OCC.
Summary updated