Abstract
This paper introduces the One Triangle Three Parallelograms (OTTP) sampling strategy, a geometrically principled approach to feature extraction in facial shape regression. By leveraging the spatial relationships between landmark triplets, OTTP captures both the connective texture between points and the local context surrounding them. When integrated into a cascaded linear regression framework utilizing Elastic Net regularization and explicit shape features, the method demonstrates superior accuracy and robustness. Evaluation on the HELEN and COFW datasets shows that OTTP achieves state-of-the-art results, particularly in scenarios with significant occlusion.
Introduction
Facial landmarking, or shape regression, is a fundamental task in computer vision involving the automatic identification of key anatomical points. Modern state-of-the-art approaches typically employ cascaded regression, where an initial mean shape is iteratively refined. A pivotal component of these systems is the "shape-indexed feature" extraction process, which samples image data relative to the current landmark estimates.
The effectiveness of shape regression is heavily dependent on the sampling strategy. While sampling at landmark locations provides local information, it often ignores the informative "connective tissue" between landmarks. Conversely, random sampling can introduce unnecessary noise. There is a clear need for a sampling method that is computationally efficient, geometrically consistent, and capable of capturing both local appearance and broader spatial context.
The OTTP Sampling Strategy
The core contribution of this work is the One Triangle Three Parallelograms (OTTP) sampling strategy. For any given triplet of landmarks, the method defines a central triangle connecting them. To capture the surrounding context, three parallelograms are constructed, each centered on one of the landmarks. These parallelograms are essentially scaled and mirrored versions of the central triangle.
Pixels are sampled within these geometric shapes using Barycentric coordinates, ensuring a mathematically elegant and high-speed process. While the central triangle captures the appearance between landmarks, the parallelograms provide the necessary context of the surrounding anatomy, which is vital for maintaining accuracy when certain landmarks are occluded.
Regression Framework
The OTTP features are utilized within a cascaded linear regression framework. To manage the high dimensionality of the feature space—which can exceed 140,000 dimensions—the model employs Elastic Net regularization. This combination of L1 and L2 penalties prevents overfitting while selecting the most informative pixel pairs for binary comparison tests.
Furthermore, the framework incorporates explicit shape features. By feeding the current landmark coordinates back into the regressor, the model learns geometric correlations (e.g., the relative positioning of eyes and mouth), allowing it to maintain a plausible facial structure even when image data is missing due to occlusion.
Experimental Evaluation
The method was rigorously tested on two major benchmarks: the high-resolution HELEN dataset (194 landmarks) and the COFW (Caltech Occluded Faces in the Wild) dataset (29 landmarks), which is specifically designed to test robustness against heavy occlusion.
On the HELEN dataset, OTTP achieved an average error of 4.8%, outperforming established baselines such as ESR (5.7%) and LBF (5.4%). However, the method's most significant impact was observed on the COFW dataset.
As shown in Figure 6, OTTP achieved a failure rate (defined as error > 10%) of only 18%, a marked improvement over the 20% achieved by RCPR and the 36% by ESR. This demonstrates the robustness provided by the combination of contextual sampling and explicit shape modeling.
Conclusion
The One Triangle Three Parallelograms strategy offers a robust and efficient solution for shape-indexed feature sampling. By capturing both local and inter-landmark information, it provides a significant boost to regression accuracy, particularly in challenging, occluded environments. Future work may explore automated methods for selecting optimal landmark triplets to further enhance the system's performance and generalization.
Summary updated