Mixed Error Coding for Face Recognition with Mixed Occlusions

Ronghua Liang, Xiao-Xin LiIJCAI 2015

Abstract

Mixed Error Coding (MEC) is a novel framework for face recognition designed to handle "mixed occlusions"—scenarios where faces are obscured by both physical objects (like scarves) and extreme lighting variations. By decoupling the error used for classification from the error used to detect occluded regions, MEC achieves significantly higher accuracy in harsh real-world conditions where traditional sparse coding methods fail.

Introduction

Automated face recognition often struggles in uncontrolled environments due to occlusions. While standard Sparse Representation-based Classification (SRC) handles simple, random noise well, it has a low "breakdown point" when faced with contiguous, structured occlusions such as sunglasses, hands, or veils. This challenge is compounded by mixed occlusions, where physical disguises are paired with extreme illumination.

Most existing models use a single reconstruction error term to both identify the person and estimate which pixels are occluded. This paper argues that this is a fundamental flaw: the discriminative error (needed to distinguish between classes) and the structural error (needed to locate the occlusion) have entirely different statistical behaviors. Treating them as identical leads to poor performance under heavy occlusion or adverse lighting.

The Mixed Error Coding (MEC) Framework

The authors propose Mixed Error Coding (MEC), which treats the recognition process as an interaction between three distinct factors:

  1. Discriminative Error ($\breve{e}$): Calculated in a high-dimensional feature space to maximize the difference between subjects.
  2. Structural Error ($\tilde{e}$): Designed to capture the spatial and statistical patterns of the occlusion itself.
  3. Occlusion Support ($s$): A binary mask indicating which pixels are "useful" for recognition versus those that are occluded.
Conceptual comparison of standard reconstruction error (a) vs. the proposed decoupled Mixed Error Coding (b).

Key Methodological Innovations

  • Image Gradient Orientation (IGO) Domain: Instead of raw pixels, the model operates in the IGO domain. Gradient directions are more stable than pixel intensities under shadows, making the system naturally robust to lighting changes.
  • Decoupled Error Metrics: The discriminative error is modeled using a Gaussian distribution in a kernel-extended IGO space, while the structural error uses a novel Cosine-based Error Metric (CEM) to highlight contiguous occluded blocks.
  • Iterative Solving: The model alternates between updating the occlusion mask (using Graph Cuts for spatial consistency) and updating the identity coefficients via Non-negative Least Squares (NNLS).

Experimental Evaluation

The model was evaluated on the Extended Yale B dataset (synthetic occlusions under extreme illumination) and the AR Face Database (real-world disguises like sunglasses and scarves).

Key Results and Insights

MEC significantly pushes the "breakdown point" of sparse coding, maintaining high accuracy even when 70-80% of the face is covered. On the Yale B Subset V (the most difficult lighting), MEC maintained high recognition rates at 60% occlusion, while competitors like SSEC and RSC saw performance drop to near zero.

Recognition rates across varying occlusion percentages on benchmark datasets, showing MEC's superior robustness.

The "Two Errors" philosophy is the most significant insight: by separating the reconstruction error from the occlusion detection error, the model can effectively "ignore" the occlusion while focusing on visible features for identification. As the model converges, the structural error distribution becomes increasingly uniform, acting as a robust stopping criterion.

Conclusion

MEC offers a powerful solution for face recognition under mixed occlusions by performing "hidden feature selection." While the iterative nature of the algorithm increases computational complexity compared to simpler subspace methods, its robustness in extreme conditions is a significant advancement. Future work may explore improving efficiency for low-dimensional images within the IGO framework.

Summary updated

BibTeX
@inproceedings{liang2015mixed,
  author = {Ronghua Liang and Xiao-Xin Li},
  title = {{Mixed Error Coding for Face Recognition with Mixed Occlusions}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure