Abstract
Human pose estimation (HPE) has many wide applications such as multimedia processing, behavior understanding and human-computer interaction. Most previous studies have encountered many constraints, such as restricted scenarios and RGB inputs. To mitigate constraints to estimating the human poses in general scenarios, we present an efficient human pose estimation model (i.e., EHPE) with joint direction cues and Gaussian coordinate encoding. Specifically, we propose an anisotropic Gaussian coordinate coding method to describe the skeleton direction cues among adjacent keypoints. To the best of our knowledge, this is the first time that the skeleton direction cues is introduced to the heatmap encoding in HPE task. Then, a multi-loss function is proposed to constrain the output to prevent the overfitting. The Kullback-Leibler divergence is introduced to measure the predication label and its ground truth one. The performance of EHPE is evaluated on two HPE datasets: MS COCO and MPII. Experimental results demonstrate that EHPE can obtain robust results, and it significantly outperforms existing state-of-the-art HPE methods. Lastly, we extend the experiments on infrared images captured by our research group. The experiments achieved the impressive results regardless of insufficient color and texture information.
Introduction
Human Pose Estimation (HPE)—the task of locating joints like elbows, knees, and ankles in an image—is a foundational challenge in computer vision. Despite significant progress driven by deep learning, real-world applications still struggle with three major hurdles: object occlusion (e.g., a kite or chair hiding a limb), neighbor interference (confusion between closely situated joints), and complex backgrounds that mimic human structures.
Most existing models utilize "isotropic" Gaussian heatmaps for training, which represent joint locations as circular probability distributions. However, this approach ignores the inherent geometric structure of the human body. Human joints are not isolated points; they are connected by limbs with specific orientations. EHPE addresses this by incorporating "skeleton direction cues" directly into the ground-truth heatmaps, allowing the model to learn the anatomical "flow" of the skeleton.
Methodology
The core of EHPE is a three-stage encoding scheme termed Classification, Fitting, and Encoding (CFE).
Anisotropic Gaussian Encoding
Instead of standard circular heatmaps, EHPE generates elliptical (anisotropic) Gaussian distributions. The long axis of these ellipses is dynamically rotated to align with the actual direction of the limb (e.g., the vector from shoulder to elbow). This provides a directional "trail" that guides the network, making it less susceptible to background noise or nearby joints.
To maintain flexibility, an adaptive strategy is employed: a threshold ($T=130^\circ$) determines when to use these elliptical cues. If a joint is highly flexed, the model may revert to a standard circle to avoid ambiguity; for straighter limbs, the directional ellipse is used.
Multi-Loss Function and Architecture
The model utilizes the HRNet architecture as its backbone for feature extraction. To optimize training, a multi-loss function combines Kullback-Leibler (KL) divergence with standard $L_2$ (Euclidean) loss. KL divergence treats the heatmap as a probability distribution, helping the model learn the precise shape of the joint's location, while $L_2$ loss provides regularization to prevent overfitting. Finally, a Taylor-expansion-based decoding method is used to extract high-precision coordinates from the predicted heatmaps.
Experimental Evaluation
The EHPE model was validated across three diverse benchmarks: MS COCO 2017, MPII Human Pose, and a custom Infrared (IR) dataset.
Performance on Standard Benchmarks
On the MS COCO test-dev set, EHPE achieved a state-of-the-art Average Precision (AP) of 79.1, significantly outperforming other leading methods such as UDP (75.8) and DARK (75.5). On the MPII dataset, it reached a [email protected] score of 90.8, showing consistent improvements across all joint types, including challenging areas like the head, shoulders, and elbows.
Robustness and Generalization
One of the most notable findings is EHPE's resilience to low-resolution inputs. When image resolution was reduced to $128 \times 96$, EHPE's accuracy decreased by only 4.6%, compared to a nearly 8% drop for standard HRNet models. Furthermore, the model performed impressively on infrared images, proving that skeleton cues can effectively compensate for missing visual details like clothing texture or skin color.
Conclusion and Future Work
EHPE demonstrates that baking geometric priors into the supervision signal significantly enhances the robustness of pose estimation in cluttered and occluded scenes. While the model is highly effective, current research focuses on improving its computational efficiency for real-time deployment on edge devices. Future work will explore knowledge distillation and model pruning to maintain these accuracy gains in more lightweight architectures.
Summary updated