Abstract
Minimum squared error based classification (MSEC) method establishes a unique classification model for all the test samples. However, this classification model may be not optimal for each test sample. This paper proposes an improved MSEC (IMSEC) method, which is tailored for each test sample. The proposed method first roughly identifies the possible classes of the test sample, and then establishes a minimum squared error (MSE) model based on the training samples from these possible classes of the test sample. We apply our method to face recognition. The experimental results on several datasets show that IMSEC outperforms MSEC and the other state-of-the-art methods in terms of accuracy.
Introduction
Traditional Minimum Squared Error Classification (MSEC) works by creating a single, global mapping that attempts to project all training samples into their respective class labels. While theoretically sound—and equivalent to Linear Discriminant Analysis (LDA) under certain conditions—this "one-size-fits-all" approach is often sub-optimal in complex real-world scenarios.
In a large database containing hundreds of different faces, a global model must account for the variance of every single class simultaneously. This introduces unnecessary noise when trying to classify a specific test image that likely only resembles a small handful of those classes. The authors propose an Improved Minimum Squared Error Classification (IMSEC) method to bridge the gap between global models and sample-specific representation methods, significantly improving recognition performance in complex datasets.
Methodology
The core innovation of IMSEC is a coarse-to-fine strategy that tailors the classifier to the specific test sample. The algorithm follows three main steps:
- Candidate Selection (Coarse): For a given test sample $t$, the algorithm represents it as a weighted sum of training samples from each class individually. It calculates the "representation error" for every class. The $L$ classes with the smallest errors are selected as "base classes."
- Model Construction (Fine): Instead of using the entire dataset, IMSEC builds a specific MSE model using only the training samples and labels from the $L$ base classes identified in Step 1. This creates a localized decision space optimized for the specific test sample.
- Classification: The test sample is projected through this tailored model. The final label is assigned based on the minimum Euclidean distance between the projected sample and the class labels of the base classes.
While standard MSEC solves one large problem for all samples, IMSEC finds a middle ground that is both more accurate than global models and more efficient than class-specific methods like Linear Regression Classification (LRC).
Experimental Setup
The authors evaluated IMSEC on three standard benchmarks in the computer vision community:
- ORL Database: 40 subjects with 10 images each, featuring variations in lighting and facial expressions.
- AR Database: 120 subjects with 26 images each, including significant challenges like facial occlusions (sunglasses/scarves) and varying illumination.
- FERET Database: 200 subjects with 7 images each, involving variations in pose and expression.
Key Results and Insights
IMSEC consistently achieved the lowest error rates across all tested databases, outperforming traditional methods (Eigenfaces, Fisherfaces), subspace methods (2DPCA, 2DLDA), and modern representation methods (SRC, CRC).
Performance on the AR Database
As shown in the table below, the proposed method (IMSEC) significantly outperforms other algorithms on the challenging AR database. For instance, with 7 training samples per class, IMSEC achieved an error rate of 20.92%, compared to 25.48% for standard MSEC (CMSE) and 28.90% for Collaborative Representation-based Classification (CRC).
Key Insights
- Local Relevance: A classifier doesn't need to distinguish a test subject from every person in a database—it only needs to distinguish them from the people they actually look like.
- Efficiency through Sparsity: By narrowing the field to $L$ base classes, the algorithm reduces the influence of irrelevant data (outliers or dissimilar classes) that would otherwise skew a global MSE model.
- Adaptive Manifolds: The method essentially creates an adaptive decision boundary that "follows" the test sample, making it more robust to high-dimensional intra-class variations.
Conclusion
IMSEC demonstrates that localized, sample-specific modeling can significantly enhance the accuracy of traditional squared error classifiers. While the method's performance depends on the selection of the parameter $L$, it offers a robust alternative to global models. Future work may focus on automatically determining the optimal number of base classes and integrating IMSEC with deep-learning-based feature extractors to further push the boundaries of face recognition accuracy.
Summary updated