A Supervised Learning Methodology for Real-Time Disguised Face Recognition in the Wild

Saumya Kumaar Saksena, Abhinandan Dogra, Abrar Majeedi, Hanan Gani, Ravi M. Vishwanath, S. N. OmkarOpenReview 2018 · OpenReview

TL;DR

This paper introduces a novel deep learning approach for recognizing disguised faces in real-world scenarios. By first detecting 20 facial key-points using a Convolutional Neural Network (CNN) and then classifying these key-points with a Support Vector Machine (SVM), the system achieves a respectable 72.4% accuracy in near real-time (19 FPS), outperforming previous methods on complex datasets.

What Problem Does This Paper Address?

Facial recognition technology has made significant strides, with deep learning models now approaching human-level accuracy. However, a persistent challenge is recognizing faces when they are disguised (e.g., wearing masks, hats, or glasses). This is a critical issue for security applications, as standard face recognition systems often fail under such conditions. The paper aims to bridge this gap by developing a robust method for Disguised Facial Recognition (DFR) that can handle the complexities of real-world "in the wild" scenarios, where faces are not perfectly cropped and backgrounds can be varied.

Methods

The proposed Disguised Facial Recognition (DFR) model employs a two-stage approach:

  1. Facial Key-Point Detection: A 14-layered CNN is used to detect 20 specific facial key-points. This CNN is trained as a regression problem, predicting the (x, y) coordinates of these key-points from input images. The architecture utilizes a large number of convolution layers to facilitate extensive feature extraction.
  2. Disguised Face Classification: Once the facial key-points are detected, their relative positions, ratios, and angles are used as features for a Support Vector Machine (SVM). The SVM then classifies the disguised faces. The authors chose SVMs due to their known efficiency for real-time applications.

Datasets

The authors utilized a dataset previously established by Singh et al. [16]. This dataset comprises 4,000 images of 20 different subjects, featuring:

  • 8 different backgrounds (ranging from simple to complex).
  • 5 different viewpoints (from -20° to 20°).
  • 10 different disguises, including beards, caps, glasses, and scarves, often in combination.

Crucially, the dataset was not cropped, and backgrounds were not altered, aiming to simulate more realistic "in the wild" conditions compared to previous benchmarks. Of the 4,000 images, 3,500 were used for training and 500 for verification.

Key Insights and Ideas

  • Key-points as Robust Features: The core idea is that even with disguises, certain facial key-points and their relative geometric relationships remain informative for identification. This is a more robust approach than relying solely on holistic face features that can be easily obscured.
  • Real-World Data Matters: The paper emphasizes the importance of using datasets with uncropped faces and complex backgrounds. They argue that models trained on such data will perform better in real-life scenarios, where perfect image conditions are rare.
  • SVM for Real-Time Classification: The choice of SVM for the classification stage is motivated by its computational efficiency, enabling the system to operate in near real-time, which is crucial for practical security applications.
  • Scarf Disguise Challenge: The results highlight that disguises involving scarves significantly reduce classification accuracy. This is attributed to the scarf obscuring the nose key-point, which is identified as particularly important for recognition.

Key Results

  • Key-Point Detection Accuracy: The CNN achieves an average key-point prediction accuracy of 65%.
  • Classification Accuracy: The SVM classifier achieves an average classification accuracy of 72.4% on complex background disguised faces.
  • Performance: The system operates at approximately 19 FPS, enabling near real-time performance.
  • Comparison: The proposed DFR method outperforms previous state-of-the-art algorithms by a margin of 9.8% on the complex dataset.

Limitations and Future Work

  • Scarf Obscurity: The authors acknowledge that disguises like scarves, which hide critical key-points (like the nose), pose a significant challenge and lead to reduced accuracy.
  • Dataset Expansion: They suggest that constructing a more extensive dataset for "in the wild" disguised faces, incorporating more varied orientations and spectrums (like Near-Infrared), would be beneficial.
  • Network Architecture: While the current CNN architecture is effective, the authors propose that a larger neural network might further improve key-point detection rates, though this could potentially impact real-time performance.
  • Camera Positioning: Future work could also explore camera positioning and active vision techniques to improve face description, especially in scenarios where the camera might not be directly facing the subject.

Summary updated

Abstract

Facial recognition has always been a challeng- ing task for computer vision scientists and experts. Despite complexities arising due to variations in camera parameters, illumination and face orientations, significant progress has been made in the field with deep learning algorithms now competing with human-level accuracy. But in contrast to the recent advances in face recognition techniques, Disguised Facial Identification continues to be a tougher challenge in the field of computer vision. The modern day scenario, where security is of prime concern, regular face identification techniques do not perform as required when the faces are disguised, which calls for a different approach to handle situations where intruders have their faces masked. Along the same lines, we propose a deep learning architecture for disguised facial recognition (DFR). The algorithm put forward in this paper detects 20 facial key-points in the first stage, using a 14-layered convolutional neural network (CNN). These facial key-points are later utilized by a support vector machine (SVM) for classifying the disguised faces based on the euclidean distance ratios and angles between different facial key-points. This overall architecture imparts a basic intelligence to our system. Our key-point feature prediction accuracy is 65% while the classification rate is 72.4%. Moreover, the architecture works at 19 FPS, thereby performing in almost real-time. The efficiency of our approach is also compared with the state-of-the-art Disguised Facial Identification methods.
BibTeX
@misc{saksena2018supervised,
  author = {Saumya Kumaar Saksena and Abhinandan Dogra and Abrar Majeedi and Hanan Gani and Ravi M. Vishwanath and S. N. Omkar},
  title = {{A Supervised Learning Methodology for Real-Time Disguised Face Recognition in the Wild}},
  year = {2018},
  eprint = {1809.02875},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1809.02875},
  howpublished = {OpenReview},
}
Figure