Sparse Variation Dictionary Learning for Face Recognition with a Single Training Sample per Person

Meng Yang, Luc Van Gool, Lei Zhang

Abstract

Recognizing a face from a single training image is a significant challenge in computer vision because the system cannot directly observe how a specific individual's appearance changes under different lighting, expressions, or poses. This paper introduces Sparse Variation Dictionary Learning (SVDL), a framework designed to predict these variations by adaptively leveraging knowledge from a generic dataset of other individuals. By condensing this external knowledge into a compact, sparse dictionary of facial changes and learning an adaptive projection, SVDL achieves state-of-the-art performance in single-sample face recognition scenarios.

Introduction

Most modern face recognition (FR) systems rely on multiple training images per person to model "intra-class variation"—the fluctuations in a person's appearance due to external factors. However, in many practical applications, such as e-passports, driver's licenses, or law enforcement databases, only a single training sample per person (STSPP) is available.

Traditional algorithms like Linear Discriminant Analysis (LDA) or Sparse Representation-based Classification (SRC) typically fail in STSPP scenarios because they lack the necessary data to characterize these variations. While generic datasets (containing images of people not in the target gallery) can be used to fill this gap, the variations observed in a generic set often do not generalize well to the specific individuals in the gallery. Furthermore, existing methods often produce redundant, high-dimensional dictionaries that are computationally expensive and sensitive to noise.

The SVDL Framework

The proposed SVDL framework addresses these issues by jointly learning two critical components: an adaptive projection and a sparse variation dictionary.

  1. Adaptive Projection Learning: This component serves as a bridge between the generic training set and the specific gallery set. It ensures that the variations learned from the generic data are relevant and applicable to the specific individuals we aim to recognize.
  2. Sparse Variation Dictionary ($D$): Instead of using a raw matrix of all possible facial changes, SVDL learns a compact set of "atoms" (basis vectors). These atoms represent common facial variations—such as a smile or a specific shadow pattern—in a sparse, efficient manner.
Overview of the SVDL framework, illustrating the joint learning of the adaptive projection and the sparse variation dictionary from a generic training set.

Optimization and Workflow

The SVDL process begins by extracting "reference" and "variation" subsets from a generic dataset. The system then employs an alternating optimization strategy: it fixes the dictionary to update the projection, and subsequently fixes the projection to refine the dictionary atoms. This iterative process continues until the objective function converges, ensuring an efficient and stable learning phase.

Convergence of the total objective function value over iterations, demonstrating the efficiency of the alternating optimization strategy.

During classification, a new "query" image is represented as a combination of a gallery image plus a sparse selection of atoms from the learned variation dictionary. The identity is determined by finding the gallery member that requires the minimal "correction" to match the query.

Experimental Evaluation

The researchers evaluated SVDL on three major benchmarks:

  • CMU Multi-PIE: A large-scale dataset focusing on controlled variations in illumination, expression, and pose.
  • FRGC (Face Recognition Grand Challenge): A more difficult dataset involving uncontrolled illumination and blurred images.
  • LFW (Labeled Faces in the Wild): The standard for "unconstrained" face recognition, featuring real-world images with complex environmental factors.

Key Results and Insights

SVDL consistently outperformed state-of-the-art methods like Extended Sparse Representation-based Classification (ESRC) and Adaptive Generic Learning (AGL).

  • High Accuracy: On the Multi-PIE dataset, SVDL achieved a recognition rate of 94.8% under varying illumination, surpassing its closest competitor (92.6%).
  • Robustness to Noise: In tests involving 40% random pixel corruption, SVDL maintained a 97.2% accuracy, while other methods dropped below 30%. This robustness is attributed to the use of the $\ell_1$-norm, which allows the model to ignore random noise and focus on structural facial changes.
  • Generalization: On the challenging FRGC and LFW datasets, SVDL showed a consistent 3-6% improvement in Rank-1 accuracy, proving that the learned dictionary generalizes well to unconstrained, real-world environments.

Discussion and Future Work

While SVDL significantly advances the state of the art for the STSPP problem, it remains dependent on the quality of the generic training set. If the generic data differs significantly from the gallery (e.g., in terms of ethnicity or age), performance may be affected. Additionally, while the method handles minor pose changes, extreme profile views remain a challenge. Future research may explore the integration of 3D face models or deep learning architectures to further refine how variations are projected onto a single 2D sample.

Summary updated

Abstract

Face recognition (FR) with a single training sample per person (STSPP) is a very challenging problem due to the lack of information to predict the variations in the query sample. Sparse representation based classification has shown interesting results in robust FR, however, its performance will deteriorate much for FR with STSPP. To address this issue, in this paper we learn a sparse variation dictionary from a generic training set to improve the query sample representation by STSPP. Instead of learning from the generic training set independently w.r.t. the gallery set, the proposed sparse variation dictionary learning (SVDL) method is adaptive to the gallery set by jointly learning a projection to connect the generic training set with the gallery set. The learnt sparse variation dictionary can be easily integrated into the framework of sparse representation based classification so that various variations in face images, including illumination, expression, occlusion, pose, etc., can be better handled. Experiments on the large-scale CMU Multi-PIE, FRGC and LFW databases demonstrate the promising performance of SVDL on FR with STSPP.
BibTeX
@inproceedings{yang2013sparse,
  author = {Meng Yang and Luc Van Gool and Lei Zhang},
  title = {{Sparse Variation Dictionary Learning for Face Recognition with a Single Training Sample per Person}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2013},
  doi = {10.1109/iccv.2013.91},
}
Figure