Misclassified Samples based Hierarchical Cascaded Classifier for Video Face Recognition

Zheyi Fan, Shuqin Weng, Yajun Zeng, Jiaojiao Jiang, Fengqian Pang, Zhiwen LiuKSII 2017 · DOI

TL;DR

This paper introduces a novel hierarchical cascaded classifier designed to improve video face recognition accuracy. By focusing on misclassified samples and their similar counterparts, the method effectively reduces within-class scatter and enhances generalization, outperforming existing algorithms.

What Problem Does This Paper Address?

Face recognition, especially from video, is notoriously challenging due to variations in posture, facial expressions, and illumination. These factors can lead to a situation where faces of the same person are highly dissimilar (high within-class scatter), while faces of different people can appear similar (low between-class scatter). This makes it difficult for traditional single-classifier approaches to achieve high accuracy and generalize well to unseen data. Existing cascaded classifiers often select samples randomly, which can lead to instability.

Methods

The proposed approach tackles this by employing a hierarchical cascaded classifier. The core idea is to build a multi-layer system that specifically addresses misclassified samples.

  1. Single Classifier Construction:
    • Feature Extraction: Principal Component Analysis (PCA) is used to reduce the dimensionality of face features.
  2. Clustering: Hierarchical clustering is applied to group similar face samples. The number of classes is determined by analyzing a distance tree. This initial clustering aims to minimize the within-class scatter.
  3. Minimum Distance Method: A first-layer classifier is created using a minimum distance method to assign samples to the most similar cluster.
  4. Multi-layer Classifier Design:
    • Handling Misclassifications: If a sample is misclassified by the current layer, a new, subsequent layer is created specifically for that misclassified sample and its similar samples.
  5. Cascading: These layers are cascaded hierarchically. Each new layer focuses on refining the classification of samples that were problematic in the previous layer.
  6. Final Classification: Support Vector Machines (SVM) are used in the final layer for multi-class classification.

This approach avoids random sample selection by intelligently focusing on difficult cases and their neighbors, leading to a more robust and accurate system.

Datasets

  • Custom Database: The authors collected their own database consisting of videos of 20 persons, with each person having 3 gallery sequences and 1 query sequence. Frames were fixed at 30 per sequence for some experiments, with a total of 801 frames for the 20 persons.
  • Public Databases:
    • FERET: Used for comparison with state-of-the-art methods.
  • Extended Yale B: Also used for comparison, featuring 2414 frontal-face images of 38 individuals with varying illumination conditions. Images were resized to 32x32.

Key Insights and Ideas

  • Focusing on the "Hard" Samples: The most significant insight is the strategy of building subsequent classifier layers specifically for misclassified samples and their similar counterparts. This targeted approach is more efficient than trying to build a single, monolithic classifier that handles all variations.
  • Hierarchical Refinement: The hierarchical structure allows for progressive refinement of classification. Easier-to-classify samples are handled by earlier layers, while more challenging ones are passed down to specialized layers, improving overall accuracy.
  • Reducing Within-Class Scatter: The initial clustering step is crucial for reducing the within-class scatter, making the subsequent classification tasks more manageable.
  • Intelligent Sample Selection: By focusing on misclassified samples and their neighbors, the method avoids the "blind" and random sample selection seen in some traditional cascaded classifiers, leading to better generalization.

Key Results

  • Improved Accuracy: The proposed hierarchical cascaded classifier achieved a recognition accuracy of 91.51% with 5 layers on their custom database, outperforming sparse representation (vote: 81.15%, mean: 77.53%), neural networks (73.28%), and SVM (89.26%).
  • State-of-the-Art Performance: On the Extended Yale B dataset, the proposed classifier achieved 99.50% accuracy with 5 layers across various feature dimensions, surpassing methods like SRC (up to 98.30%) and CSRC (up to 99.30%).
  • FERET Dataset Performance: On the FERET dataset, the proposed classifier achieved 85.20% accuracy, outperforming CRC (66.39%), SCRC (74.36%), MSPCRC (31.40%), MSPSCRC (74.44%), KNN (64.47%), and PNN (58.92%).
  • Within-Class vs. Between-Class Scatter: The experiments showed that as the number of layers increases, the within-class scatter decreases, and the between-class scatter increases, leading to better separation of classes.
  • Computational Cost: While accuracy increases with more layers, computational time also increases. However, the authors note that the total time cost is still manageable.

Limitations and Future Work

  • Computational Time: The paper acknowledges that increasing the number of layers leads to higher computational time, although it is still considered reasonable.
  • Future Work: Not explicitly detailed in the provided text, but the success of the hierarchical approach suggests potential for further exploration in optimizing the number of layers or the specific algorithms used in each layer for different types of misclassifications. The authors also mention that the method is suitable for video face recognition, implying potential for real-time applications.

Summary updated

Abstract

Due to various factors such as postures, facial expressions and illuminations, face recognition by videos often suffer from poor recognition accuracy and generalization ability, since the within-class scatter might even be higher than the between-class one. Herein we address this problem by proposing a hierarchical cascaded classifier for video face recognition, which is a multi-layer algorithm and accounts for the misclassified samples plus their similar samples. Specifically, it can be decomposed into single classifier construction and multi-layer classifier design stages. In single classifier construction stage, classifier is created by clustering and the number of classes is computed by analyzing distance tree. In multi-layer classifier design stage, the next layer is created for the misclassified samples and similar ones, then cascaded to a hierarchical classifier. The experiments on the database collected by ourselves show that the recognition accuracy of the proposed classifier outperforms the compared recognition algorithms, such as neural network and sparse representation.
BibTeX
@inproceedings{fan2017misclassified,
  author = {Zheyi Fan and Shuqin Weng and Yajun Zeng and Jiaojiao Jiang and Fengqian Pang and Zhiwen Liu},
  title = {{Misclassified Samples based Hierarchical Cascaded Classifier for Video Face Recognition}},
  booktitle = {KSII},
  year = {2017},
  doi = {10.3837/tiis.2017.02.009},
}
Figure