Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet)

Rui Li, Yan An, Shiqiang Yang, Duo He, Xin Zeng, Hongyan LiuSensors 2024 · DOI

TL;DR

This paper introduces EL-HRNet, a novel lightweight human pose estimation model that significantly reduces computational cost and parameter count compared to the standard HRNet architecture. By employing a modified residual block with grouped convolutions and integrating an attention mechanism, EL-HRNet achieves a strong balance between efficiency and accuracy, making it suitable for resource-constrained environments.

What Problem Does This Paper Address?

Human pose estimation, a crucial task in computer vision for understanding human actions and interactions, has seen significant advancements with deep learning. Architectures like HRNet achieve high accuracy by maintaining high-resolution feature maps throughout the network. However, this comes at the cost of increased model complexity, a large number of parameters, and high computational demands. This makes such models challenging to deploy on devices with limited computational resources, such as mobile phones or embedded systems. The core problem this paper addresses is the trade-off between the accuracy of high-resolution pose estimation methods and their practical deployability due to computational constraints.

Methods

The authors propose an Efficient and Lightweight HRNet (EL-HRNet) by modifying the core building blocks of the original HRNet. The key innovations are:

  • Lightweight Basicblock (L-Basicblock): The standard Basicblock in HRNet, which typically uses two 3x3 convolutions, is replaced. The new L-Basicblock employs a combination of a 1x1 convolution to reduce channel dimensions (compressing the feature map) followed by a 3x3 grouped convolution (GConv). This strategy aims to reduce parameters and computations while preserving feature information.
  • Lightweight Attention Basicblock (LA-Basicblock): To mitigate potential information loss introduced by the lightweight L-Basicblock, the Convolutional Block Attention Module (CBAM) is integrated. CBAM enhances feature representation by focusing on important channel and spatial information. This combined module, LA-Basicblock, aims to recover lost information and improve accuracy.

The overall EL-HRNet architecture retains the multi-resolution parallel subnetworks characteristic of HRNet, but with these modified blocks.

Datasets

The proposed EL-HRNet model was trained, validated, and tested on two widely used human pose estimation datasets:

  • COCO2017: A large-scale dataset with 17 keypoints per human instance, containing 200,000 images. The authors used the training set for training and the validation/test sets for evaluation.
  • MPII: This dataset features images from real-world activities with full-body annotations. It contains 25,000 images.

Key Insights and Ideas

  • Decomposition of Convolutional Blocks: The paper demonstrates that by carefully decomposing standard convolutional blocks into channel reduction (1x1 conv) and grouped convolutions (3x3 GConv), significant parameter and computation reductions are achievable without catastrophic accuracy loss.
  • Attention for Information Recovery: The integration of CBAM is a crucial step to re-emphasize the importance of feature channels and spatial locations, effectively compensating for the information compression introduced by the lightweight L-Basicblock. This highlights the power of attention mechanisms in modern deep learning architectures for fine-tuning feature representations.
  • HRNet's Continued Relevance: Despite the rise of transformer-based models, the paper shows that the core HRNet architecture, when optimized for efficiency, can still achieve competitive results, particularly in balancing accuracy and computational cost.

Key Results

The EL-HRNet model demonstrates a compelling balance between efficiency and accuracy:

  • Parameter Count: The model requires only 5 million parameters, a significant reduction compared to many other state-of-the-art methods.
  • Computational Cost: It achieves 2.0 GFlops of computation.
  • COCO2017 Performance: On the COCO2017 validation set, EL-HRNet achieved an AP score of 67.1%. This is competitive with other lightweight models, offering a good trade-off.
  • MPII Performance: On the MPII validation set, the model achieved a [email protected] of 87.7%.
  • Comparison: The paper shows EL-HRNet outperforming or being competitive with several other lightweight models like Lite-HRNet and ViPNAS, while having significantly fewer parameters and lower computational cost than larger models like ScaleNAS or Hourglass. For instance, compared to ScaleNAS, EL-HRNet has a parameter count that is an order of magnitude smaller and FLOPS that are a quarter.

Limitations and Future Work

  • Accuracy vs. State-of-the-Art: While EL-HRNet achieves a good balance, the authors acknowledge that there is still room for improvement in absolute accuracy compared to the very top-performing, but much heavier, models.
  • Mobile Deployment Focus: The paper explicitly states that the demand for human pose estimation on mobile terminals is a key motivation. Future work will focus on further optimizing the model for mobile deployment, specifically exploring how to further reduce parameters and computations while enhancing prediction accuracy and real-time detection capabilities.

Summary updated

Abstract

As an important direction in computer vision, human pose estimation has received extensive attention in recent years. A High-Resolution Network (HRNet) can achieve effective estimation results as a classical human pose estimation method. However, the complex structure of the model is not conducive to deployment under limited computer resources. Therefore, an improved Efficient and Lightweight HRNet (EL-HRNet) model is proposed. In detail, point-wise and grouped convolutions were used to construct a lightweight residual module, replacing the original 3 × 3 module to reduce the parameters. To compensate for the information loss caused by the network's lightweight nature, the Convolutional Block Attention Module (CBAM) is introduced after the new lightweight residual module to construct the Lightweight Attention Basicblock (LA-Basicblock) module to achieve high-precision human pose estimation. To verify the effectiveness of the proposed EL-HRNet, experiments were carried out using the COCO2017 and MPII datasets. The experimental results show that the EL-HRNet model requires only 5 million parameters and 2.0 GFlops calculations and achieves an AP score of 67.1% on the COCO2017 validation set. In addition, [email protected] is 87.7% on the MPII validation set, and EL-HRNet shows a good balance between model complexity and human pose estimation accuracy.
BibTeX
@inproceedings{li2024human,
  author = {Rui Li and Yan An and Shiqiang Yang and Duo He and Xin Zeng and Hongyan Liu},
  title = {{Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet)}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24020396},
}
Figure