Abstract
Deep Convolutional Neural Networks (CNNs) have significantly advanced face recognition, yet they struggle with occlusions in variable facial regions. Humans naturally ignore occlusions to focus on visible areas; mimicking this, we propose a mask learning strategy to identify and discard corrupted feature elements. We introduce the Pairwise Differential Siamese Network (PDSN) to establish a mask dictionary by analyzing differences between occluded and occlusion-free face pairs. Each dictionary item, a Feature Discarding Mask (FDM), maps specific occluded areas to corrupted features. For recognition, we combine relevant FDMs to eliminate noise from the feature vector. Experiments demonstrate that our approach significantly outperforms state-of-the-art methods on both synthetic and real-world occluded datasets.
Introduction
While CNNs excel at general face recognition, they are fragile under partial occlusion. Models trained on clear faces often attempt to extract features from occluding objects (e.g., sunglasses), leading to corrupted feature vectors. Common data augmentation strategies often degrade performance on clean images. We propose a method that explicitly ignores occluded regions without compromising the recognition of unobstructed faces, leveraging the spatial consistency of deep features.
Proposed Methodology
The proposed framework operates in three distinct stages: mask generation learning, dictionary establishment, and robust recognition.
Stage I: Pairwise Differential Siamese Network (PDSN)
The core of our approach is the PDSN, designed to learn the correspondence between facial occlusions and feature corruption. It processes pairs of images—one clean ($x^i$) and one occluded ($x^j_i$) of the same person.
The network computes a Differential Signal by taking the absolute difference between the features of the clean and occluded faces. This signal serves as a direct indicator of which feature elements are affected by the occlusion. A Mask Generator then uses this signal to produce a mask $m^j_i$ that identifies corrupted elements.
Stage II: Establishing the Mask Dictionary
To handle arbitrary occlusions at test time, we pre-calculate masks for a 5x5 grid of facial blocks. For each block, we learn a binary Feature Discarding Mask (FDM). Binarization (setting values to 0 or 1) was found to be more effective than soft weighting, as it completely eliminates noise.
Stage III: Occlusion-Robust Recognition
During inference, an external occlusion detector (e.g., FCN-8s) identifies covered regions. The system retrieves the corresponding FDMs from the dictionary, combines them via a logical AND operation, and applies the resulting mask to the Trunk CNN's feature vector. This ensures only reliable, non-occluded features are used for matching.
Experimental Results
The model was trained on CASIA-WebFace using a ResNet50-based ArcFace "Trunk" CNN.
- AR Dataset: On Protocol 2 (real-world occlusions), the PDSN achieved 98.19% accuracy for sunglasses and 98.33% for scarves, significantly surpassing the baseline Trunk CNN.
- MegaFace: On the synthesized occluded set (MF1occ), accuracy improved to 56.34% from the baseline's 53.03%.
- Clean Face Performance: Crucially, the model maintained 99.20% accuracy on the standard LFW benchmark, proving that robustness to occlusion does not come at the cost of general performance.
Conclusion
By explicitly discarding corrupted feature elements through a learned mask dictionary, our PDSN-based approach provides a robust solution for face recognition under occlusion. The use of a differential signal provides a strong supervision mechanism, allowing the network to precisely map spatial occlusions to feature-level noise.
Summary updated