Face recognition via compact second order image gradient orientations

He-Feng Yin, Xiao-Jun Wu, Xiaoning SongarXiv 2022 · arXiv

Abstract

Face recognition in unconstrained environments remains challenging due to occlusions like sunglasses and scarves. This paper introduces Compact Second Order Image Gradient Orientations (CSOIGO), a method that utilizes second-order image gradients to capture the geometric curvature of facial features. By mapping these orientations into a complex domain and employing collaborative representation-based classification, CSOIGO demonstrates remarkable robustness against noise and occlusions, outperforming several deep learning architectures in small-sample scenarios.

Introduction

While face recognition (FR) technology has advanced significantly, "in-the-wild" conditions—specifically occlusion—remain a major hurdle. Standard algorithms often fail when discriminative features are hidden by accessories like sunglasses or scarves.

Traditional subspace learning methods typically rely on first-order image gradient orientations (IGO), which represent the "slope" of image intensity. Although IGO is more robust than raw pixel values, it remains sensitive to noise and can lose fine-grained geometric details. The authors of this work argue that the human vision system perceives images similarly to a 3D landscape. To accurately capture the "shape" of a face, it is necessary to analyze the curvature (second-order gradients) rather than just the tilt.

Visualizing images as 3D landscapes to capture geometric curvature.

Methodology

The proposed Compact Second Order Image Gradient Orientations (CSOIGO) framework consists of three primary stages:

1. Feature Extraction (SOIGO)

Instead of relying on the first derivative, the method calculates the second derivative to capture local curvature. This approach significantly suppresses noise and highlights fine details around critical facial landmarks like the eyes, nose, and mouth that first-order methods often miss.

Visualization of second-order image gradient orientations highlighting facial features.

2. Dimensionality Reduction

To manage the high dimensionality and redundancy of these gradients, orientations are mapped into a complex number domain ($e^{j\phi}$). Complex Principal Component Analysis (PCA) is then applied to create a compact and computationally efficient feature vector. This complex mapping also resolves the "wraparound" problem where 0 and 359 degrees are numerically distant but physically identical.

3. Classification

The final stage uses Collaborative Representation-based Classification (CRC). CRC is more efficient than Sparse Representation (SRC) and leverages the collaborative nature of all training samples to reconstruct the test image and determine identity.

Experimental Setup

The method was primarily benchmarked using the AR Face Database, which contains over 4,000 images of 126 subjects. This dataset includes real-world challenges such as disguises (sunglasses and scarves) across different sessions, as well as illumination and expression variations. Additionally, "Random Block Occlusion" tests were conducted by overlaying images with baboon textures at various percentages (30% to 50%) to evaluate robustness limits.

Key Results and Insights

The core innovation lies in prioritizing curvature over slope. Second-order gradients are inherently more stable against noise in the orientation domain, effectively mapping the "valleys and ridges" of the face.

  • Occlusion Performance: On the AR database with only two training samples, CSOIGO achieved an accuracy of 91.17%, surpassing the previous state-of-the-art (GD-HASLR) by nearly 3%.
  • Comparison with Deep Learning: In specific occluded scenarios, CSOIGO outperformed established models like VGGFace and Lightened CNN. For instance, in the Session 2 sunglasses test, CSOIGO reached 95.67% accuracy, whereas VGGFace (FC6) only achieved 51.00%.
  • Small Sample Size (SSS) Robustness: Unlike deep learning models that require massive datasets, CSOIGO excels when training data is scarce, maintaining high accuracy even with a single neutral face per subject.

Conclusion and Future Work

CSOIGO provides a robust solution for occluded face recognition, particularly in small-sample scenarios. Future research will explore incorporating Linear Discriminant Analysis (LDA) to further enhance feature compactness and investigating other Collaborative Representation variants. Additionally, the SOIGO feature extraction technique may find applications in other computer vision domains, such as pedestrian detection or industrial defect inspection.

Summary updated

Abstract

Conventional subspace learning approaches based on image gradient orientations only employ the first-order gradient information. However, recent researches on human vision system (HVS) uncover that the neural image is a landscape or a surface whose geometric properties can be captured through the second order gradient information. The second order image gradient orientations (SOIGO) can mitigate the adverse effect of noises in face images. To reduce the redundancy of SOIGO, we propose compact SOIGO (CSOIGO) by applying linear complex principal component analysis (PCA) in SOIGO. Combined with collaborative representation based classification (CRC) algorithm, the classification performance of CSOIGO is further enhanced. CSOIGO is evaluated under real-world disguise, synthesized occlusion and mixed variations. Experimental results indicate that the proposed method is superior to its competing approaches with few training samples, and even outperforms some prevailing deep neural network based approaches. The source code of CSOIGO is available at https://github.com/yinhefeng/SOIGO.
BibTeX
@misc{yin2022face,
  author = {He-Feng Yin and Xiao-Jun Wu and Xiaoning Song},
  title = {{Face recognition via compact second order image gradient orientations}},
  year = {2022},
  eprint = {2201.09246},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2201.09246},
}
Figure