AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses

Zixuan Xu, Banghuai Li, Ye Yuan, Miao GengAAAI 2021 · DOI

Facial landmark localization—the task of identifying specific points like eye corners, nose tips, and jawlines—is a cornerstone of modern computer vision. While performance on frontal, well-lit faces is robust, accuracy degrades significantly under large pose variations (e.g., profile views). This "large pose" problem stems from the vast geometric search space and the uncertainty introduced by self-occlusion.

Presented at AAAI-21, AnchorFace addresses these issues through a "split-and-aggregate" strategy. By employing pose-specific anchor templates, the model simplifies the search space and achieves state-of-the-art accuracy across extreme poses while maintaining real-time speeds (~45 FPS).

Methods: The Split-and-Aggregate Strategy

The authors propose a two-step pipeline that adapts the concept of anchors from object detection to facial geometry.

1. The Split Step (Anchor-based Regression)

AnchorFace introduces Anchor Templates, which are predefined sets of landmark coordinates representing "average" faces at different poses (e.g., looking left, looking right, frontal). These templates can be hand-designed or generated via K-Means clustering of training data.

Instead of asking the network to find a landmark from scratch, the network predicts the offset from the anchor template to the actual landmark. This "splits" the difficult global search into many easier, local sub-problems.

The AnchorFace pipeline: comparing direct regression with anchor-based regression and the final aggregated result.

2. The Aggregate Step

Since multiple anchors might produce predictions for the same face, the model must decide which ones to trust. A Confidence Branch predicts a score for each anchor. Predictions from anchors with low confidence are discarded, and the remaining predictions are combined via weighted averaging. This effectively "smooths out" the uncertainty caused by large poses or occlusions, acting as a mini-ensemble within a single model.

Evaluation and Results

The model was evaluated on four major benchmarks: AFLW, WFLW, 300W, and Menpo. AnchorFace achieved state-of-the-art accuracy, notably a Normalized Mean Error (NME) of 1.56% on the AFLW-Full set.

Key findings include:

  • Robustness to Pose: On the WFLW dataset, AnchorFace showed significant gains in the "Large Pose" subset, proving the effectiveness of the split-and-aggregate strategy.
  • Efficiency: By avoiding the need for 3D morphable models (3DMM) or multi-stage cascades, AnchorFace remains lean. Using a ShuffleNet-V2 backbone, it hits 45 FPS on a standard GPU, making it nearly 3x faster than previous high-performers like the "LAB" method.
  • Ablation Studies: The authors demonstrated that using 24 K-Means generated templates provided the optimal balance of accuracy and speed.

Conclusion

AnchorFace provides a robust, elegant framework for face alignment that doesn't sacrifice speed for the ability to handle "wild" real-world poses. While it shows some limitations in improving performance for pitch variations (up-and-down movement), its success in handling yaw and roll makes it a powerful tool for robust facial analysis in unconstrained environments.

Summary updated

BibTeX
@inproceedings{xu2021anchorface,
  author = {Zixuan Xu and Banghuai Li and Ye Yuan and Miao Geng},
  title = {{AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i4.16418},
}
Figure