Two-Stage Block-Based Whitened Principal Component Analysis with Application to Single Sample Face Recognition

Biao Wang, Wenming Yang, Weifeng Li, Qingmin Liao

Introduction

Face recognition systems often encounter the "Single Sample Face Recognition" (SSFR) problem, where only one training image per individual is available. This is common in law enforcement and access control. Traditional methods like Principal Component Analysis (PCA) struggle in these scenarios due to "local confusion." PCA maximizes global variance, which can inadvertently group similar-looking individuals together, especially under varying environmental conditions.

The Local Confusion Problem

Standard PCA aims to find a projection that maximizes the total scatter of all face images. While this separates very different faces, it often fails to distinguish between individuals with similar facial structures.

Figure 1: Comparison between PCA projection and an ideal projection, illustrating the local confusion problem where similar classes are merged.

Proposed Method: TS-BWPCA

To overcome these limitations, researchers proposed a Two-Stage Block-Based Whitened PCA (TS-BWPCA). This approach employs a "coarse-to-fine" strategy to refine recognition.

1. Robust Feature Extraction and the SSS Problem

Instead of using raw pixel values, the system utilizes Local Binary Patterns (LBP). The image is partitioned into non-overlapping blocks, and LBP histograms are computed for each. This not only provides robustness against changes in illumination and expression but also helps solve the Small Sample Size (SSS) problem. By treating each block as a separate data point for the covariance matrix, the system gains significantly more statistical power than it would from a single whole-image vector.

Figure 4: The LBP thresholding process used for robust feature extraction, converting local pixel neighborhoods into binary codes.

2. Two-Stage Recognition Framework

The recognition process moves from a global view to a local one:

  • Stage 1 (Coarse): A global Whitened PCA (WPCA) is applied to the entire gallery. For a given probe image, the system identifies the $K$-Nearest Neighbors ($K$-NNs). This stage acts as a filter to narrow down the search space.
  • Stage 2 (Fine): A second WPCA is performed specifically on the $K$ candidates from Stage 1. By focusing only on these similar faces, the algorithm can identify subtle, discriminative features that global PCA ignores.

3. The Role of Whitening

Whitening scales the principal components to ensure equal variance across all axes. This prevents dominant but non-discriminative factors, such as lighting, from overwhelming the fine details necessary for accurate identification.

Experimental Results

The TS-BWPCA method was evaluated using the FERET database, focusing on challenges like expression (Fb/Fc) and aging (Dup I/Dup II).

  • Aging Robustness: On the Dup II set (images taken up to 18 months apart), TS-BWPCA achieved 88.0% accuracy, significantly higher than the 60.7% achieved by standard Eigenfaces.
  • Noise Tolerance: In tests involving "salt and pepper" noise, the method maintained a 70.48% recognition rate, while standard PCA dropped to near zero (0.59%).
  • Performance Gain: Across various probe sets, the method showed a consistent improvement of 12.2% to 34.0% over baseline Eigenface techniques.

Conclusion

The TS-BWPCA framework effectively addresses the SSFR problem by shifting from global to local variance maximization. While the two-stage process increases computational complexity—requiring a new PCA projection for each probe—the significant gains in accuracy and robustness make it a compelling solution for high-stakes recognition tasks. Future work may explore applying this coarse-to-fine logic to other biometric modalities like fingerprint or iris recognition.

Summary updated

Abstract

In the task of face recognition, a challenging issue is the one sample problem, namely, there is only one training sample per person. Principal component analysis (PCA) seeks a low-dimensional representation that maximizes the global scatter of the training samples, and thus is suitable for one sample problem. However, standard PCA is sensitive to the outliers and emphasizes more on the relatively distant sample pairs, which implies that the close samples belonging to different classes tend to be merged together. In this paper, we propose two-stage block-based whitened PCA (TS-BWPCA) to address this problem. For a specific probe image, in the first stage, we seek the K-Nearest Neighbors (K-NNs) in the whitened PCA space and thus exclude most of samples which are distant to the probe. In the second stage, we maximize the “local” scatter by performing whitened PCA on the K nearest samples, which could explore the most discriminative information for similar classes. Moreover, block-based scheme is incorporated to address the small sample problem. This two-stage process is actually a coarse-to-fine scheme that can maximize both global and local scatter, and thus overcomes the aforementioned shortcomings of PCA. Experimental results on FERET face database show that our proposed algorithm is better than several representative approaches.
BibTeX
@article{wang2012two,
  author = {Biao Wang and Wenming Yang and Weifeng Li and Qingmin Liao},
  title = {{Two-Stage Block-Based Whitened Principal Component Analysis with Application to Single Sample Face Recognition}},
  journal = {IEICE Trans. Information and Systems},
  year = {2012},
  doi = {10.1587/transinf.e95.d.853},
}
Figure