Introduction
Landmark localization—the task of pinpointing specific "fiducial points" on a human body or face—is a cornerstone of modern computer vision. While state-of-the-art models like the "Stacked Hourglass" network achieve remarkable accuracy, they are notoriously resource-hungry, often requiring high-end GPUs and hundreds of megabytes of memory. This hardware requirement creates a significant barrier for deployment on edge devices with limited power and memory.
In their work, Bulat and Tzimiropoulos present a comprehensive study on binarizing networks for fine-grained localization tasks. They demonstrate that through architectural innovation, it is possible to achieve near-state-of-the-art results on human pose and face alignment using models that are up to 39x smaller than their full-precision counterparts.
Rethinking the Residual Block
The core challenge in binarization is the "lossy" nature of constraining weights and activations to just -1 or 1. While this offers a 32x reduction in memory and significant speedups via bitwise operations, it often leads to a catastrophic drop in accuracy for tasks requiring sub-pixel precision.
The authors identify that standard residual blocks, which typically use a "bottleneck" structure with $1 \times 1$ convolutions, are ill-suited for binary networks. In a binary context, $1 \times 1$ filters have almost zero representational power, acting more as "feature blockers" than efficient dimensionality reducers.
To address this, the authors proposed a Hierarchical, Parallel, and Multi-scale block. Key features include:
- Removal of $1 \times 1$ Convolutions: Replacing them with wider $3 \times 3$ layers improved absolute accuracy by over 8%.
- Parallel Paths: The input is split into multiple branches to process information at different scales.
- Hierarchical Structure: Branches are organized to ensure the "shortest path" for gradients is minimized, mitigating the fading gradient problem.
- Decomposed Filters: Large receptive fields are created by stacking smaller, more efficient $3 \times 3$ binary filters.
Training Strategies for Binary Networks
Beyond architecture, the authors found that training dynamics must be adjusted for binary precision. Binary gradients are typically an order of magnitude smaller than real-valued ones, which can hinder convergence.
By switching from the standard L2 (Mean Squared Error) loss to Sigmoid cross-entropy loss for heatmap regression, the authors boosted gradient magnitude by 10-15x. Additionally, they found that maintaining a ReLU activation after binary layers—despite the "Sign" function used for binarization—provided a 2% performance boost by introducing necessary non-linearity.
Experimental Evaluation
The proposed approach was evaluated on human pose estimation (MPII) and various face alignment datasets (AFLW, AFLW2000-3D).
On the MPII dataset, the hierarchical binary block achieved 76% PCKh, a significant improvement over the 67.2% achieved by a standard binarized bottleneck. The model also proved competitive with much larger real-valued models on face alignment tasks. Furthermore, in facial part segmentation, the binary model nearly matched real-valued performance, achieving 97.91% pixel accuracy.
In terms of efficiency, the models achieved a theoretical 39x memory compression. Practical tests showed a 3.5x speedup on GPUs, with projected speedups of up to 58x for optimized CPU implementations.
Conclusion
This work proves that landmark localization can be efficiently deployed on resource-constrained hardware without sacrificing significant accuracy. By moving away from standard architectures and focusing on gradient flow and representational power in binary space, the authors have bridged the gap between high-performance research models and practical edge deployment. Future work may explore hybrid precision or novel loss functions to further close the remaining performance gap with full-precision models.
Reference: Bulat, A., & Tzimiropoulos, G. (2020). Hierarchical Binary CNNs for Landmark Localization with Limited Resources. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 343-356.
Summary updated