Abstract
Recently, convolutional neural networks (CNNs)-based facial landmark detection methods have achieved great success. However, most existing CNN-based methods fail to accurately model the relationships among local details and cannot fully explore discriminative semantic features, leading to failures under partial occlusions and large pose variations. To address these problems, we propose a cross-order cross-semantic deep network (CCDN). Specifically, a cross-order two-squeeze multi-excitation (CTM) module is proposed to introduce cross-order channel correlations for more discriminative representations. Moreover, a novel cross-order cross-semantic (COCS) regularizer drives the network to learn complementary features from different activations. Experimental results on challenging benchmark datasets demonstrate the superiority of our CCDN over state-of-the-art methods.
Introduction
Facial landmark detection—the task of mapping key points like the corners of the eyes, tip of the nose, and edges of the mouth—is a foundational step for face recognition, emotion analysis, and augmented reality. While modern CNNs excel on clear, front-facing images, they often struggle "in the wild." When a face is turned sharply (large pose variation) or partially covered (heavy occlusion), standard networks fail to explicitly model the relationships between different facial parts. The CCDN architecture addresses this by learning "part-specific semantic features," training different components of the network to become experts on specific facial regions.
Methodology
The CCDN builds on a standard stacked hourglass network backbone but introduces two major novel components:
Cross-order Two-squeeze Multi-excitation (CTM) Module
Standard attention mechanisms typically use first-order statistics (Global Average Pooling) to weight feature channels. The CTM module incorporates second-order statistics (Global Covariance Pooling) to capture the variance and relationships between channels. By looking at these higher-order correlations, the network makes more informed decisions about which features to emphasize across multiple attention-specific feature maps.
Cross-order Cross-semantic (COCS) Regularizer
To ensure that multiple attention maps do not all focus on the same prominent features (like the nose), the COCS regularizer acts as a custom loss function. It forces different attention blocks to learn distinct semantic meanings by maximizing correlation within a block while minimizing it between different blocks. This ensures a diverse coverage of the entire face.
Key Insights and Ideas
By mathematically forcing different attention heads to decorrelate, the network naturally learns to assign different heads to different facial parts. This "forced complementarity" builds resilience; if a hand covers the mouth, the eye-tracking and jaw-tracking heads remain unaffected, preserving the overall facial structure.
Visualizing these activations reveals the "divide and conquer" strategy in action. As seen in the activation maps, one excitation block may focus specifically on the upper facial region, such as the eyes and brows, while another focuses on the lower face.
Experimental Results
The CCDN consistently outperformed state-of-the-art methods across several challenging benchmarks:
- Heavy Occlusion (COFW): Achieved a Normalized Mean Error (NME) of 4.91% and a failure rate of just 1.12%.
- Extreme Poses (AFLW): Reached an NME of 1.35% on the full testing set.
- Complex Attributes (WFLW): Achieved an NME of 4.29%, beating competitors across all subsets including pose, expression, and illumination.
Ablation studies confirmed that both the second-order statistics in the CTM and the COCS regularizer are essential for these accuracy gains.
Conclusion and Limitations
The CCDN demonstrates that explicitly modeling cross-order correlations and forcing semantic diversity significantly boosts the robustness of facial landmark detection. The primary trade-off is computational cost, as increasing the number of excitation blocks linearly increases the load. Future work will explore making this multi-head attention process more efficient for real-time applications on edge devices.
Summary updated