Abstract
Deep Convolutional Neural Networks (CNNs) have significantly advanced face recognition, yet they remain vulnerable to facial occlusions such as sunglasses or masks. This paper presents a novel approach using a Pairwise Differential Siamese Network (PDSN) to identify and discard corrupted feature elements. By establishing a "Mask Dictionary" based on facial regions, the system can dynamically mask noisy features during recognition, maintaining high accuracy on both occluded and clean faces.
Introduction
While deep CNNs excel at extracting high-level features, they often operate as "black boxes" regarding spatial occlusions. When a portion of a face is obscured, standard networks still attempt to extract features from that region, leading to "corrupted" feature vectors filled with noise. Traditional methods like HOG or SIFT allowed for direct spatial-to-feature mapping, but in deep learning, a single neuron's receptive field may cover a large, ambiguous area. Simply augmenting training data with occluded images often leads to a performance trade-off on clean images. This research proposes a method to explicitly identify which deep feature elements are affected by specific occluded facial areas and discard them during the recognition process.
Methodology
The proposed framework consists of three primary stages: learning mask generators, establishing a mask dictionary, and performing occlusion-robust recognition.
Pairwise Differential Siamese Network (PDSN)
The core innovation is the PDSN, designed to learn the relationship between occluded facial blocks and corrupted feature elements. The network processes pairs of images: a clean face ($x^i$) and an occluded version of the same face ($x^i_j$). By calculating the differential signal—the absolute difference between the feature maps $f(x^i)$ and $f(x^i_j)$—the network isolates the changes specifically caused by the occlusion. A "Mask Generator" module then learns to map these differences to a mask $m^i_j$.
Mask Dictionary and Feature Discarding
The face is partitioned into a $5 \times 5$ grid. For each grid block, the PDSN learns a specific mask identifying the feature elements most susceptible to corruption when that block is occluded. These masks are binarized into Feature Discarding Masks (FDMs) and stored in a dictionary. This approach favors discarding corrupted information over attempting to reconstruct it (e.g., via GANs), as reconstruction often introduces further artifacts.
Recognition Pipeline
During inference, an external segmentation network first identifies occluded regions. The system retrieves the corresponding FDMs from the dictionary, combines them using a logical "AND" operation, and applies the resulting mask to the face's feature vector. This "zeros out" corrupted components before final classification or comparison.
Experimental Results
The method was evaluated across several benchmarks, demonstrating significant improvements over baseline models.
- AR Dataset: On Protocol 2, the proposed method achieved 98.19% accuracy for sunglasses and 98.33% for scarves, compared to the baseline Trunk CNN's 95.14% and 96.53%, respectively.
- LFW Benchmark: The model maintained a high accuracy of 99.20% on clean faces, avoiding the performance degradation typically seen with standard occlusion-augmentation strategies (which dropped to 98.68%).
- Large-Scale Scalability: On the MegaFace Challenge 1 with synthesized occlusions (MF1occ), the method reached 56.34% accuracy, outperforming the baseline of 53.03%.
Conclusion
By leveraging the differential signal between clean and occluded face pairs, the PDSN effectively identifies corrupted deep features. The use of a binary Mask Dictionary allows for a robust recognition system that ignores noise without sacrificing performance on unoccluded images. Future work may involve refining the fixed-grid approach and extending this masking philosophy to other forms of image degradation like motion blur.
Summary updated