Fisher Discriminative Least Square Regression with Self-Adaptive Weighting for Face Recognition.

Zhe Chen, Xiao‐Jun Wu, Josef Kittler

Abstract

This paper introduces Fisher Discriminative Least Squares Regression (FDLSR), a novel framework designed to enhance image classification by rethinking label representation. By integrating the Fisher discrimination criterion with label relaxation techniques ($\epsilon$-draggings), FDLSR simultaneously maximizes inter-class margins and minimizes intra-class scatter. Unlike computationally intensive low-rank methods, FDLSR utilizes an iterative optimization process with closed-form solutions, ensuring high efficiency. Experimental results across eight diverse datasets, including face and object recognition, demonstrate that FDLSR achieves state-of-the-art performance with significantly reduced training times.

Introduction

Least Squares Regression (LSR) is widely used in pattern recognition due to its mathematical simplicity and speed. However, traditional LSR often struggles with classification because it forces data into a rigid binary label space (e.g., 0s and 1s), assuming uniform distances between all classes. This rigid structure fails to capture the complex, non-linear relationships inherent in real-world data.

Previous attempts to address this, such as Discriminative LSR (DLSR), introduced $\epsilon$-draggings to allow labels to "drift" and create wider margins. However, excessive drifting can lead to a loss of intra-class compactness, where samples from the same class become uncorrelated. While methods like Low-Rank DLSR (LRDLSR) attempt to fix this using nuclear norm optimization, they are often computationally expensive and slow for large-scale applications.

Methodology

The proposed FDLSR framework integrates the Fisher discrimination criterion directly into the label relaxation process. The objective function is built on three pillars:

  1. Regression Error: A standard term to learn a projection matrix $Q$ mapping data to labels.
  2. Label Relaxation ($\epsilon$-draggings): A dynamic term that allows labels to move during optimization to maximize inter-class margins.
  3. Fisher Regularization: This component constrains relaxed labels to ensure that samples from the same class stay tightly clustered (minimizing intra-class scatter) while different classes are pushed further apart.

To solve this, the authors employ an iterative three-step optimization process. Crucially, each step—updating the projection matrix, the relaxation matrix, and the transition variable—has a closed-form solution. This avoids the heavy computational burden of singular value decompositions (SVD) required by low-rank alternatives.

Convergence analysis of FDLSR across eight datasets, illustrating the stable minimization of the objective function and the corresponding rise in recognition rates over iterations.

Experimental Results

The researchers evaluated FDLSR across eight datasets, ranging from face recognition (AR, CMU PIE, FERET, LFW) to fine-grained object and action recognition (Stanford-40, CUB200-2011, Oxford 102 Flowers). For large-scale datasets, features were extracted using a pre-trained VGG-19 network, demonstrating FDLSR's compatibility with modern deep learning pipelines.

Accuracy and Efficiency

FDLSR consistently outperformed or matched state-of-the-art methods like Sparse Representation-based Classification (SRC) and Coordinate Representation-based Classification (CRC). On the AR face dataset with 14 training samples, FDLSR achieved a peak accuracy of 98.18%.

Performance comparison on the AR dataset. FDLSR (ours) demonstrates superior accuracy across different training sample sizes and an order-of-magnitude advantage in training speed.

Beyond accuracy, the computational efficiency of FDLSR is a standout feature. On the AR dataset, FDLSR’s training time was a mere 0.18 seconds, compared to 55.05 seconds for Fisher Discrimination Dictionary Learning (FDDL) and 7.77 seconds for LRDLSR. Convergence is typically reached within 30 iterations, making it highly suitable for time-sensitive applications.

Conclusion

FDLSR successfully bridges the gap between the flexibility of label relaxation and the structural discipline of the Fisher criterion. By ensuring both inter-class separation and intra-class compactness, it provides a robust and efficient alternative to traditional regression-based classifiers. While the method currently faces challenges in Single Sample Per Person (SSPP) scenarios due to the lack of intra-class variance to minimize, future work may explore adapting the framework for low-data regimes or integrating it directly into end-to-end deep neural network training.

Summary updated

BibTeX
@misc{chen2019fisher,
  author = {Zhe Chen and Xiao‐Jun Wu and Josef Kittler},
  title = {{Fisher Discriminative Least Square Regression with Self-Adaptive Weighting for Face Recognition.}},
  year = {2019},
  eprint = {1903.07833},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1903.07833},
}
Figure