TL;DR
This paper moves face recognition from simple geometric distance (Eigenfaces) to a probabilistic framework that classifies types of image differences. By distinguishing between "intrapersonal" variations (same person, different expression) and "extrapersonal" variations (different people), the authors achieved a 95% accuracy rate, becoming the top performer in the 1996 FERET competition.
What Problem Does This Paper Address?
In the mid-90s, the "Eigenface" method was the gold standard for face recognition. It worked by projecting face images into a low-dimensional subspace and measuring the Euclidean distance between them. However, this approach has a fundamental flaw: it treats all pixel differences equally.
In reality, not all differences are created equal. A change in a person's expression or a shift in lighting can create a large "distance" in pixel space, even though the identity remains the same. Conversely, two different people might look similar in a low-dimensional projection. The authors sought to create a similarity measure that understands which variations are "incidental" (like a smile) and which are "critical" (like identity).
Methods
The core innovation is treating face recognition as a binary pattern classification problem. Instead of comparing two images directly, the authors look at the difference image ($\Delta = I_1 - I_2$).
- Dual Subspaces: They define two classes of variations:
- Intrapersonal ($\Omega_I$): Differences between images of the same individual (e.g., different expressions, lighting).
- Extrapersonal ($\Omega_E$): Differences between images of different individuals.
- Bayesian Analysis: They use Bayes' rule to calculate the a posteriori probability $P(\Omega_I | \Delta)$—essentially asking, "Given this difference, how likely is it that these two images are the same person?"
- Density Estimation via PCA: Because the difference images are high-dimensional, the authors use Principal Components Analysis (PCA) to estimate the likelihood functions. They divide the vector space into a principal subspace (the top $M$ eigenvectors) and its orthogonal complement to make the math tractable.
- Computational Efficiency: To avoid the "costly" nonlinear computation of Bayesian similarity during a live search, the authors developed a method using whitening transformations. By pre-processing images into whitened subspace coefficients, the complex probabilistic matching reduces to a simple, fast Euclidean norm.
Datasets
The researchers evaluated their system using the ARPA FERET face database, the standard benchmark of the era.
- Internal Testing: A subset of 74 pairs for training and 38 pairs for testing.
- 1996 FERET Competition: A massive (for the time) double-blind test involving nearly 1,200 individuals.
Key Insights and Ideas
- Similarity as Classification: The "aha moment" is realizing that similarity isn't a distance; it's a probability. By modeling the distribution of differences, the system learns to ignore "noise" like facial expressions.
- The "Dual" Basis: The paper reveals that intrapersonal eigenfaces represent subtle, deformable variations (expression/lighting), while extrapersonal eigenfaces look like "standard" faces, coding for identity features like hair color, glasses, and bone structure.
- The ML vs. MAP Trade-off: The authors show that a simplified Maximum Likelihood (ML) approach—which only considers the intrapersonal class—is almost as accurate as the full Maximum a Posteriori (MAP) approach but twice as fast.
Key Results
- Superior Accuracy: In the 1996 FERET competition, the Bayesian algorithm was the top performer with 95% accuracy.
- Halving the Error: On the FERET test, the Bayesian method provided a 10% performance gain over standard Eigenfaces, effectively halving the error rate of the previous state-of-the-art.
- Internal Benchmarks: On a smaller internal set, the Bayesian approach achieved a 90% rank-1 recognition rate, compared to 84% for standard Eigenface matching.
- Speedup: The proposed offline transformation allows the system to perform matching using only about 400 arithmetic operations per image pair, making it viable for very large databases.
Limitations and Future Work
- Alignment Sensitivity: While the paper mentions an automatic face-processing system for normalization (translation, scale, rotation), the Bayesian model's performance is heavily dependent on the quality of this initial geometric alignment.
- Computational Cost of Training: While the online matching is fast, the offline training requires performing PCA on large sets of difference images, which can be computationally intensive as the training set grows.
- Simplified ML Measure: The authors note that while the simplified ML measure is faster, it hasn't been "officially" FERET tested to the same extent as the MAP version, though internal tests show a slight 2-3% drop in accuracy.
Summary updated