Inhibition of Occluded Facial Regions for Distance-Based Face Recognition

Daniel López Sánchez, Juan M. Corchado, Angélica González ArrietaIJCAI 2018 · DOI

TL;DR

This paper introduces a novel Case-Based Reasoning (CBR) system for face recognition that's robust to partial occlusions. It achieves this by intelligently ignoring features from occluded facial regions, outperforming traditional methods on challenging datasets. The work also explores dimensionality reduction to make the system more efficient.

What Problem Does This Paper Address?

Recognizing faces in real-world scenarios is tough because faces are often partially hidden by things like scarves, sunglasses, or even just how someone is positioned. While many face recognition systems exist, they often struggle when parts of the face are obscured. Existing methods that try to handle occlusion often require knowing what is causing the occlusion beforehand or are computationally very expensive. This paper aims to develop a face recognition system that can effectively handle partial occlusions without needing prior knowledge of the occlusion type and while remaining computationally feasible, especially for systems with limited resources.

Methods

The core of this work is a Case-Based Reasoning (CBR) system. Here's a breakdown of their approach:

  • Preprocessing: Standard steps like face detection (using HOG), face alignment (using a learned model), pose normalization, and lighting normalization (using Histogram Equalization) are applied to prepare the face images.
  • Feature Extraction: They use Local Binary Patterns (LBP), specifically Local Binary Pattern Histograms (LBPH). LBPH divides the face into blocks and creates a histogram for each block, capturing local texture information.
  • Occlusion-Robust Retrieval: This is the key innovation. They propose a method to identify which LBPH blocks are likely occluded. The assumption is that blocks corresponding to occluded facial regions will have larger minimum local distances when compared to similar regions in known faces. A threshold is used to identify these "occluded" blocks.
  • Modified Dissimilarity Function: Instead of comparing all features, their dissimilarity function inhibits the contribution of features from identified occluded blocks. This means the system focuses on the visible, reliable parts of the face for comparison.
  • Dimensionality Reduction (Optional): To handle the high dimensionality of multi-scale LBPH features, they propose a local Random Projection (RP) technique. This reduces the dimensionality of features within each "occlusion unit" (a set of blocks representing a facial region) without losing too much critical information, making the system faster.

Datasets

  • ARFace Database: This is the primary dataset used for evaluation. It contains around 4,000 color images of 126 individuals with various facial expressions, lighting conditions, and partial occlusions (like glasses and scarves). The authors created specific training, validation, and test sets from this database, including subsets for "Light," "Glasses," and "Scarf" occlusions.

Key Insights and Ideas

  • Occlusion as a Signal: Instead of treating occlusion as a problem to be solved by removing occluded parts, they leverage the difference in feature similarity caused by occlusion as a signal to identify and down-weight those features.
  • Local Feature Robustness: The block-based nature of LBPH is crucial. It allows them to isolate and analyze features from specific facial regions, making the occlusion detection and inhibition process localized and effective.
  • Computational Efficiency: By focusing on visible features and exploring dimensionality reduction, the proposed CBR system aims to be more practical for real-world applications with limited computational power, unlike some complex deep learning models.
  • Data-Independence of Random Projection: The use of Random Projection for dimensionality reduction is advantageous because the projection matrix is data-independent, meaning it can be generated once and reused, simplifying the process.

Key Results

The paper presents experimental results in Tables 1 and 2, comparing their proposed "our CBR" method against standard classifiers (wkNN, SVM, LR, NB) using various LBP features.

  • Superior Performance: The proposed CBR system ("our CBR") consistently outperforms traditional classifiers across different feature types (single-scale LBP, multi-scale LBP, and multi-scale LBP with local RP) and occlusion types (Light, Scarf, Glasses).
  • Significant Gains with Occlusion: For instance, on the "Scarf" test set with automatic alignment (Table 1), their CBR system using m.s. LBP8,2 achieves 89.2% accuracy, while the next best (wkNN) achieves 73.3%. With glasses, their system reaches 50.6% compared to 34.9% for wkNN.
  • Impact of Local RP: The integration of local Random Projection (m.s. LBP8,2 + RP) shows that dimensionality reduction can be achieved with minimal loss in accuracy, and in some cases, even slight improvements, while significantly reducing descriptor size. For example, with glasses, m.s. LBP8,2 + RP with local RP achieves 92.0% accuracy.

Limitations and Future Work

  • Automatic Face Alignment: The authors acknowledge that while they use automatic face alignment, manual alignment (as shown in Table 2) generally yields better results, suggesting that improving the robustness of automatic alignment to occlusion is a promising future direction.
  • Hyperparameter Tuning: The performance of the occlusion detection relies on setting an appropriate threshold. While they describe a method for selecting this threshold using a validation set, fine-tuning this parameter remains important.
  • Generalization to Other Occlusions: While tested on glasses and scarves, the system's performance on other types of occlusions (e.g., hands, hats) is not explicitly detailed.
  • Future Work: The authors suggest investigating more robust automatic face alignment methods for occluded faces and exploring other dimensionality reduction techniques.

Summary updated

BibTeX
@inproceedings{sanchez2018inhibition,
  author = {Daniel López Sánchez and Juan M. Corchado and Angélica González Arrieta},
  title = {{Inhibition of Occluded Facial Regions for Distance-Based Face Recognition}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/746},
}
Figure