Abstract
We propose a simple yet reliable bottom-up approach with a good trade-off between accuracy and efficiency for the problem of multi-person pose estimation. Given an image, we employ an Hourglass Network to infer all the keypoints from different persons indiscriminately as well as the guiding offsets connecting the adjacent keypoints belonging to the same persons. Then, we greedily group the candidate keypoints into multiple human poses (if any), utilizing the predicted guiding offsets. And we refer to this process as greedy offset-guided keypoint grouping (GOG). Moreover, we revisit the encoding-decoding method for the multi-person keypoint coordinates and reveal some important facts affecting accuracy. Experiments have demonstrated the obvious performance improvements brought by the introduced components. Our approach is comparable to the state of the art on the challenging COCO dataset under fair conditions.
Introduction
Human pose estimation—identifying the 2D coordinates of joints like elbows, knees, and shoulders—is a foundational task in computer vision. While relatively straightforward for a single person, the task becomes significantly more complex in images containing multiple individuals, where it transforms into a "grouping" challenge.
Current methodologies generally fall into two categories: top-down and bottom-up. Top-down methods detect every person first and then estimate their pose individually. While accurate, these methods are computationally expensive as time scales linearly with the number of people. Bottom-up methods detect all keypoints simultaneously and then associate them with specific individuals. These are faster but often struggle with accuracy in crowded scenes due to the difficulty of correct joint association. This paper introduces a streamlined bottom-up approach that balances high accuracy with real-time efficiency.
Methodology
Our pipeline is centered on an Hourglass-104 Network that performs two tasks simultaneously: Heatmap Prediction and Guiding Offset Regression. The network generates Gaussian response maps to locate 17 types of keypoints and predicts 19 types of "guiding offsets." These offsets are displacement vectors pointing from one joint to its neighbor in the human skeletal tree.
Greedy Offset-Guided Grouping (GOG)
Once the network predicts the joints and offsets, the GOG algorithm assembles individual human skeletons. Instead of solving a complex global matching problem, it calculates a "connection score" for potential limbs based on joint confidence and the alignment of the predicted offset with the actual distance between joints. The algorithm then greedily assembles these limbs into full skeletons.
Coordinate Encoding-Decoding
A critical contribution of this work is the revisit of coordinate encoding-decoding. We investigated how low-resolution heatmaps are translated back into high-resolution image coordinates. Our findings reveal that subtle choices in interpolation (e.g., bicubic vs. bilinear) and the mathematical transformations used for pixel mapping significantly impact final accuracy. Specifically, using bilinear interpolation during upsampling was found to decrease accuracy by 2.1% AP, highlighting the importance of sub-pixel localization.
Key Results and Discussion
The proposed method was evaluated on the COCO (Common Objects in Context) dataset, the industry standard for human pose estimation. Our model, "Ours 3 (plus)," achieved an Average Precision (AP) of 67.5 on the COCO validation set, outperforming the CenterNet baseline (64.0 AP) and rivaling complex systems like PIFPAF.
In terms of efficiency, the system operates at approximately 7.9 to 27.8 FPS on a single NVIDIA 2080 Ti GPU, depending on input resolution. This performance makes it suitable for near-real-time applications. Furthermore, by defining up to 39 "limbs" to connect joints, we demonstrated a theoretical upper bound for grouping logic exceeding 95% AP, suggesting that the primary bottleneck remains joint detection rather than the grouping process itself.
Conclusion
This paper presents a simple yet effective bottom-up approach for multi-person pose estimation. By utilizing directional offsets as "glue" for kinematic structures and refining coordinate decoding, we achieve a robust balance between speed and precision. While the greedy grouping is fast, it remains dependent on the initial accuracy of keypoint detection. Future work will explore more robust backbone architectures like HRNet and further refine offset regression to handle extreme occlusions where joints may be entirely hidden from view.
Summary updated