Abstract
Modern pose estimation has transitioned from tracking a few major joints to identifying over a hundred intricate keypoints across the face, hands, and feet. This shift introduces a significant training imbalance where dense clusters (e.g., the face) dominate the optimization process at the expense of sparse, coarse regions (e.g., limbs). We present a bottom-up approach that utilizes graph-based weighting to rebalance the loss function. By modeling the skeleton as a graph and calculating harmonic centrality within local "ego graphs," we assign higher weights to independent, coarse keypoints. Our method achieves state-of-the-art results on the COCO WholeBody and ApolloCar3D datasets while maintaining real-time inference speeds.
Introduction
Pose estimation is a fundamental task in computer vision, essential for applications ranging from human-computer interaction to autonomous driving. Recent datasets have increased the granularity of annotations, moving from simple 17-point skeletons to complex "whole-body" models with 133 keypoints. However, standard training procedures often fail to handle the uneven distribution of these points. Dense regions like the face provide redundant information, causing neural networks to neglect isolated but structurally critical points like the ankles or hips.
Previous "top-down" approaches addressed this by using multi-stage pipelines—detecting the person first and then zooming in on specific regions. While effective, these methods are computationally expensive and difficult to generalize to non-human objects. We propose a principled, bottom-up alternative that treats the pose as a community of points, using graph theory to dynamically rebalance training.
Methodology
Our approach detects all keypoints in a single pass. The core innovation lies in the calculation of training weights derived from the geometric structure of the skeleton.
Graph-Based Weighting
We represent the skeleton as a graph where vertices are keypoints and edges represent the average Euclidean distance between them in the training data. To quantify the "independence" of a keypoint, we employ harmonic centrality. A point within a dense cluster (e.g., an eyebrow point) exhibits high centrality due to its proximity to many neighbors. Conversely, isolated points like the ankle have low centrality.
Local Ego Graphs
Rather than computing global centrality, we focus on "ego graphs" of radius three. This local focus ensures that weights are determined by the immediate community of a keypoint (e.g., the structure of a hand) rather than the entire body. We then apply inverse weighting: keypoints with low centrality receive higher training weights. This strategy forces the network to prioritize learning independent, coarse keypoints that are not easily predicted by their neighbors.
Experimental Results
We evaluated our method on two major benchmarks: COCO WholeBody (human) and ApolloCar3D (vehicles).
Performance and Speed
On the COCO WholeBody dataset, our method achieved 60.4 AP, a significant improvement over the previous state-of-the-art ZoomNet (54.1 AP). The gains were particularly pronounced in fine-grained areas; face detection accuracy rose to 85.0 AP from a previous best of 62.3 AP.
The method also demonstrated excellent generalization on the ApolloCar3D dataset, reaching a 91.9% detection rate, nearly matching human annotator performance (92.4%). Crucially, the system maintains high efficiency, running at approximately 15.2 FPS on an NVIDIA GTX 1080 Ti, making it suitable for real-time deployment.
Conclusion
Keypoint Communities provides a robust framework for training pose estimation models on imbalanced skeletons. By leveraging local graph centrality to reweight the loss function, we bridge the gap between coarse and fine-grained detection without the need for complex multi-stage architectures. Future work will explore dynamic graph structures that adapt to specific poses in real-time.
Summary updated