Ensemble and Modular Approaches for Face Detection: A Comparison

Raphaël Féraud, Olivier BernierNeurIPS 1997

In the late 1990s, face detection was effectively the "wild west" of computer vision. Researchers were grappling with a fundamental, asymmetric problem: how do you define a "non-face"? While human faces possess predictable structures and geometries, the set of everything that isn't a face is effectively infinite. This paper by Feraud and Bernier presents an elegant solution using autoassociative neural networks and modular architectures to navigate this lopsided classification problem.

The Asymmetry Problem in Face Detection

The core challenge in training a reliable face detector is data asymmetry. You can easily collect thousands of face images to define a "face class," but you can never collect a truly representative set of "non-faces."

Standard autoassociative networks (autoencoders) learn to compress and reconstruct data. Theoretically, if you train one exclusively on faces, it should reconstruct faces well and non-faces poorly. However, two major issues typically arise:

  1. Manifold Overlap: The learned "face space" often accidentally includes non-face patterns that look just similar enough to confuse the model.
  2. Linear Collapse: Due to local minima during training, these networks often settle for a simple linear solution (similar to Principal Component Analysis) rather than capturing the complex, non-linear geometry of human faces.

Methodology: Constrained Generative Models

To overcome these limitations, the authors introduce three key innovations, centered around a novel neural network training approach.

1. The Constrained Generative Model (CGM)

Instead of solely training the network on positive face examples, the authors use an iterative algorithm to find "hard" non-faces—images the model wrongly classifies as faces.

Autoassociative neural network architecture for 15x20 pixel windows.

They then constrain the network: when it processes one of these hard non-faces, it is forced to reconstruct it as its nearest face neighbor. This "pushes" the boundaries of the face manifold inward, creating a much sharper mathematical distinction between a face and a non-face.

Examples of 'hard' non-face image patches used to constrain the generative model.

2. Reconstruction Error as Probability

The model classifies a 15x20 pixel window by evaluating its reconstruction error (the distance between the input image and the network's reconstructed output). They map this error to a probability using a Gaussian-like function, allowing the system to express a confidence score rather than a rigid binary "yes/no."

3. Modular Architectures

Recognizing that a face at a 60-degree angle looks fundamentally different from a head-on portrait, the authors compared three ways to combine networks to handle variance:

  • Ensemble: Averaging the outputs of several identical detectors to reduce variance and false alarms.
  • Conditional Mixture: Using specialized experts (e.g., one for front-views, one for turned faces) and a "gating network" that evaluates the input and decides which expert to trust.
  • Conditional Ensemble: A Bayesian approach that combines the strengths of both ensembles (reducing false alarms) and mixtures (handling different orientations).

Datasets and Key Results

The study utilized several classic benchmarks, including a mix of the Usenix face database, CMU test sets, and the Sussex Face Database (used specifically to test orientation limits from 0 to 90 degrees).

The Conditional Mixture of CGMs emerged as the clear winner in the experiments:

  • Orientation Robustness: It maintained a 100% detection rate for faces turned up to 30 degrees, and an impressive 87.5% at 40 degrees. In contrast, a standard front-view detector's performance plummeted to 0% past 30 degrees.
  • Accuracy vs. False Alarms: On the CMU test set A, the Conditional Mixture achieved an 87% detection rate with only 1.15 false alarms per million windows.
  • State-of-the-Art Comparison: This significantly outperformed the then-standard Rowley (1995) model, which had a lower detection rate (85%) and nearly double the false alarm rate (2.13 per million).

Key Insights and Limitations

The fundamental "aha moment" of this research is that defining what a face is isn't enough; you must explicitly define what it isn't by forcing the model to fail gracefully on counter-examples. Furthermore, the failure of a single "general" detector compared to a mixture of specialized experts suggests that the manifold of all possible face angles is too complex for a single small network to learn efficiently. Divide and conquer is the superior strategy.

Limitations to Consider: * Convergence Proofs: Because the set of non-faces is practically infinite, the authors note they cannot mathematically prove their iterative counter-example algorithm will always converge in finite time (though it practically converged in just 8 iterations during their tests). * Extreme Angles: Performance still drops off significantly beyond 60 degrees of orientation, suggesting a need for even more specialized experts for strict profile views. * Preprocessing Dependency: The system relies heavily on manual normalization (histogram equalization and smoothing), meaning it might struggle to adapt dynamically to extremely low-light or high-noise environments without prior adjustment.

Summary updated

BibTeX
@inproceedings{feraud1997ensemble,
  author = {Raphaël Féraud and Olivier Bernier},
  title = {{Ensemble and Modular Approaches for Face Detection: A Comparison}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1997},
}
Figure