Dictionary Representation of Deep Features for Occlusion-Robust Face Recognition

Feng Cen, Guanghui WangIEEE Access 2019 · DOI

Abstract

Deep learning has achieved exciting results in face recognition; however, the accuracy is still unsatisfying for occluded faces. To improve the robustness for occluded faces, this paper proposes a novel deep dictionary representation-based classification (DDRC) scheme, where a convolutional network (CNN) is employed as the feature extractor followed by a dictionary to linearly code the extracted deep features. The dictionary is composed of a gallery part consisting of deep features of training samples and an auxiliary part consisting of mapping vectors associated with occlusion patterns. A squared Euclidean norm is used to regularize the coding coefficients, ensuring computational efficiency. The proposed scheme is robust to large contiguous occlusion, works with a single training sample per subject, and demonstrates superior performance over state-of-the-art algorithms.

Introduction

While Convolutional Neural Networks (CNNs) have revolutionized face recognition, partial occlusion remains a significant challenge. When a face is partially covered by objects like sunglasses or scarves, the resulting deep feature vector shifts into a different subspace, often causing standard classifiers (e.g., softmax) to fail. Traditional methods like Sparse Representation Classification (SRC) were designed for image-space noise but are computationally expensive and do not directly translate to the non-linear feature spaces of modern CNNs. This research introduces Deep Dictionary Representation-based Classification (DDRC) to bridge this gap, offering a robust and efficient solution for real-time face recognition under occlusion.

Methodology: The DDRC Pipeline

The DDRC approach integrates the feature-extraction power of CNNs with the robustness of dictionary learning through several key stages:

  1. Feature Extraction: A pre-trained VGG-Face network extracts a 4096-dimensional feature vector from the input image.
  2. Augmented Dictionary Construction: The core innovation lies in the dictionary $D_a$, which includes:
  3. Gallery Part: Deep features of clean (non-occluded) training samples.
  4. Auxiliary Part: "Mapping vectors" representing the difference between non-occluded and occluded faces. These vectors capture the mathematical essence of specific occlusions (e.g., sunglasses) and can be harvested from subjects outside the primary training set.
  5. Dimensionality Reduction: PCA is applied to reduce feature dimensions, preventing overfitting and accelerating computation.
  6. $l_2$-norm Regularization: Unlike traditional SRC which uses the $l_1$-norm, DDRC employs a squared $l_2$-norm (ridge regression). This allows for an analytical solution that is significantly faster while maintaining high accuracy.
  7. Classification: Subjects are identified by determining which gallery member, combined with auxiliary occlusion vectors, best reconstructs the test sample's features.

Key Insights and Feature Analysis

A central finding of this work is that the impact of specific occlusions can be approximated as a linear shift in the deep feature space. This allows the system to generalize: it does not need to see a specific subject wearing sunglasses to recognize them; it can learn the "sunglasses-ness" shift from other subjects.

Visualization of deep feature differences for inter-subject, intra-subject, and sunglasses-occluded samples.

As shown in the feature visualizations above, the variations introduced by occlusions like sunglasses have distinct patterns compared to inter-subject differences. By modeling these shifts explicitly in the auxiliary dictionary, DDRC effectively "sees through" the obstructions.

Experimental Evaluation

The authors evaluated DDRC on three major benchmarks: the AR Database (sunglasses/scarves), the FERET Pose Database (block occlusions), and the CelebA Database (large-scale synthesized occlusions).

Key Results

  • Superior Accuracy: On the AR database, DDRC achieved 94.7% accuracy for faces with sunglasses, significantly outperforming the standard VGG+softmax setup (68.0%).
  • Single Sample Success: In tests with only one clean training image per person, DDRC reached 95.2% average accuracy, surpassing traditional SRC (56.6%) and Extended SRC (88.0%).
  • Real-time Performance: Classification takes approximately 16ms per image on a standard i7 CPU, making it suitable for practical applications.

Conclusion

DDRC provides a generic and efficient framework for face recognition that is robust to large, contiguous occlusions. While the current system performs best when occlusion types are known and represented in the auxiliary dictionary, future work aims to develop more "universal" dictionaries to handle arbitrary, novel occlusion patterns.

Summary updated

BibTeX
@article{cen2019dictionary,
  author = {Feng Cen and Guanghui Wang},
  title = {{Dictionary Representation of Deep Features for Occlusion-Robust Face Recognition}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2901376},
}
Figure