Abstract
Traditional ridge regression and its extensions often encounter the "small-class problem" in high-dimensional data analysis, where the number of available projections is constrained by the number of classes. This paper proposes Generalized Robust Regression (GRR), a novel framework for jointly sparse subspace learning. By employing $L_{2,1}$-norm penalties on both the loss function and regularization terms, GRR ensures joint sparsity and robustness to outliers. Furthermore, the integration of an "elastic factor" and $L_{2,1}$-norm-based local geometric measurements enhances performance in feature selection and classification. Experimental results across six benchmark datasets demonstrate that GRR is a highly effective and robust method for face recognition.
Introduction
Dimensionality reduction is a cornerstone of image feature extraction and recognition. While methods like Ridge Regression (RR) and Linear Discriminant Analysis (LDA) are standard, they suffer from significant drawbacks: sensitivity to outliers, neglect of local data structures, and a mathematical cap on the number of features (the small-class problem). In many regression-based methods, the number of projection vectors is mathematically capped by the number of classes in the dataset. If a dataset contains only five classes, traditional methods might only yield four or five features, which is insufficient for describing complex visual data. Generalized Robust Regression (GRR) is introduced to overcome these limitations, providing a robust, sparse, and flexible subspace learning approach.
Methodology
The GRR framework is built upon several key innovations designed to handle the complexities of real-world visual data.
1. Robustness via $L_{2,1}$-Norm
Unlike traditional $L_2$-norm methods that are sensitive to large errors, GRR utilizes the $L_{2,1}$-norm for the loss function, regularization, and locality preservation. The $L_{2,1}$-norm (the sum of the Euclidean norms of the rows of a matrix) encourages joint sparsity, forcing the model to select a consistent subset of important features across all projections while remaining resilient to noise and outliers. This ensures that the model focuses on the most discriminative parts of the data, such as the eyes and mouth in facial images, while ignoring noisy background pixels.
2. The Elastic Factor and Local Structure
To further enhance robustness, GRR incorporates an "elastic factor" ($h$) into the loss function. This term acts as a bias stabilizer, preventing the model from overfitting to corrupted pixels or noisy labels, which is particularly useful in scenarios involving heavy occlusion (e.g., subjects wearing sunglasses or scarves). Additionally, by using the $L_{2,1}$-norm to measure local geometric structures, GRR respects the "manifold" structure of the data, allowing it to distinguish between similar-looking individuals more effectively than global methods like PCA.
3. Overcoming the Small-Class Problem
GRR decouples the projection matrix into two components ($B$ and $A$). This mathematical decomposition allows researchers to extract any number of features ($k$) regardless of the number of classes ($c$). This breaks the rank bottleneck that often limits the discriminative power of traditional regression methods in small-sample-size problems.
Experimental Evaluation
The performance of GRR was evaluated against nine state-of-the-art methods, including SPCA, LPP, and SRC, across six diverse datasets: FERET, ORL, Yale, AR Face, Char74K_15, and LFW (Labeled Faces in the Wild).
- Superior Accuracy: GRR consistently outperformed competitors. Notably, it achieved 100% accuracy on the Yale database with only 6 training samples, while the next best method (SRC-L1LS) reached 97.47%.
- Occlusion Mastery: On the AR database, GRR showed superior resilience to occlusions. When faces were obscured by scarves or random block noise, GRR maintained significantly higher recognition rates than traditional methods.
- Computational Efficiency: Despite being an iterative method, the proposed algorithm for GRR is computationally efficient, typically converging in fewer than 10 iterations.
- Unconstrained Recognition: In tests on the challenging LFW dataset using features extracted by Deep Convolutional Neural Networks (CNNs), GRR demonstrated its versatility and effectiveness in unconstrained environments.
Conclusion
Generalized Robust Regression (GRR) provides a powerful solution for high-dimensional subspace learning. By addressing the small-class problem and incorporating robust $L_{2,1}$-norm regularization with an elastic factor, GRR offers significant improvements in accuracy and reliability for face recognition tasks. While the method is more complex than one-shot solvers, its performance gains in noisy and occluded environments make it a compelling choice for real-world applications. Future research will explore optimizations for imbalanced datasets and automated hyperparameter tuning.
Summary updated