Application of Mixed Precision Training in Human Pose Estimation Model Training

Jun Zhu, Jiwei Xu, Lei Feng, Hao ZhangProcesses 2025 · DOI

TL;DR

This paper tackles the challenge of long training times for human pose estimation models, especially on resource-constrained devices. By combining mixed-precision training (using both FP16 and FP32) with early stopping, they significantly accelerate training speed without sacrificing model accuracy, making advanced pose estimation more accessible.

What Problem Does This Paper Address?

Training sophisticated deep learning models for tasks like human pose estimation (HPE) is computationally intensive. Large datasets and complex model architectures, such as those based on Transformers, demand substantial GPU memory and processing power. This makes it difficult for beginners or researchers with limited hardware to train these models efficiently. The extended training times not only lead to high costs but also significant energy consumption. The paper aims to bridge this gap by proposing a method to drastically reduce training time and resource requirements, thereby democratizing access to HPE model development.

Methods

The core of this paper lies in the synergistic application of two techniques:

  • Mixed-Precision Training: This involves using a combination of different floating-point precisions during training. Specifically, the paper leverages FP32 (single-precision) for its high accuracy, particularly for input data and parameter updates, and FP16 (half-precision) for computations that can benefit from its speed and reduced memory footprint. This approach aims to achieve the best of both worlds: the accuracy of FP32 and the efficiency of FP16.
  • Forward Propagation: Input data is initially in FP32 for accuracy. Computations like convolutions can utilize FP16 on compatible hardware (e.g., NVIDIA's Tensor Cores) for acceleration.
  • Back Propagation: Intermediate calculations for gradients are primarily performed in FP16 to reduce memory and computation.
  • Loss Scaling: To mitigate the risk of gradient underflow (where small gradients become zero in FP16), a dynamic loss scaling mechanism is employed. This involves multiplying gradients by a scaling factor during backpropagation, and then dividing them back before parameter updates. The scaling factor is adjusted dynamically to prevent overflow or underflow.
  • Parameter Update: Final parameter updates are performed in FP32 to maintain model stability and accuracy.
  • Early Stopping: This is a regularization technique used to prevent overfitting and reduce training time. The method monitors the model's performance on a validation set. When the performance (e.g., loss or accuracy) on the validation set stops improving or starts to degrade, training is halted. This avoids unnecessary computation and prevents the model from learning noise from the training data.

The paper combines these two methods to optimize the training process.

Datasets

  • COCO (Common Objects in Context): This is the primary dataset used for training and evaluation. The paper highlights its richness, containing 80 categories and over 330,000 images, with extensive labeling for object detection, segmentation, and keypoints. Specifically for HPE, it includes 17 key points for the human body.

Key Insights and Ideas

  • The Synergy of FP16 and FP32: The paper demonstrates that simply using FP16 everywhere can lead to accuracy loss due to its limited range. The key insight is that a hybrid approach, strategically using FP32 for critical parts (input, parameter updates) and FP16 for computationally intensive parts (intermediate calculations), can achieve significant speedups without compromising accuracy.
  • Loss Scaling as a Crucial Enabler: The limited range of FP16 poses a significant challenge for gradient propagation. The dynamic loss scaling mechanism is presented as a vital component that makes mixed-precision training practical by preventing underflow and ensuring stable training.
  • Early Stopping as a Time-Saver and Regularizer: Beyond just reducing training time, early stopping is shown to be effective in preventing overfitting, leading to better generalization. Its integration with mixed-precision training offers a double benefit: faster training and improved model robustness.
  • Accessibility for Beginners: The combined approach makes advanced HPE models more accessible to individuals and institutions with less powerful hardware, lowering the barrier to entry for research and development in this field.

Key Results

The paper presents compelling quantitative results:

  • Training Time Reduction:
    • Mixed-precision training alone reduced training time by approximately 40 hours compared to single-precision training.
  • Early stopping alone reduced training time by about 28 hours.
  • The combination of mixed-precision training and early stopping achieved the most significant reduction, decreasing training time by about 60 hours compared to single-precision training.
  • Accuracy Maintenance: The proposed methods resulted in negligible loss of accuracy. In some cases, the average accuracy under a specific IoU threshold (AP.5) was even slightly higher with mixed-precision training than with single-precision training.
  • Hardware Acceleration: Table 1 shows that FP16 computation on NVIDIA's Turing architecture (RTX TITAN) can achieve acceleration ratios of up to 3x for convolutional layer forward computation, 2.55x for batch normalization, and 2.29x for gradient updates, with high Tensor Core utilization.

Limitations and Future Work

  • Hardware Dependency: The effectiveness of mixed-precision training, particularly the acceleration achieved, is highly dependent on hardware support for FP16 operations. GPUs with dedicated FP16 computing units (like NVIDIA's Volta, Turing, and Ampere architectures) offer significant benefits, while older or non-NVIDIA hardware might see limited acceleration.
  • Precision Loss Sensitivity: For extremely large models or scenarios requiring very high precision, the potential precision loss from FP16 needs careful experimental verification to ensure it's acceptable.
  • Future Work: The paper suggests that for scenarios involving ultra-large-scale models, distributed training, or extreme sensitivity to precision, further experimentation is needed to assess the trade-offs and integrate other technical optimizations.

Summary updated

Abstract

Human pose estimation is an important research direction in the field of computer vision, aiming to detect and locate key points of the human body from images or videos and infer human posture. It plays a significant role in many applications, such as action recognition, motion analysis, virtual reality, and human–computer interaction. As a popular research topic, it is often studied by beginners in deep learning. However, the task of human pose estimation is rather complex, and the mainstream datasets are huge. Even on high-end single-GPU devices, training models requires a considerable amount of time. To help beginners learn efficiently on devices with limited performance, this paper introduces the method of mixed-precision training into the model and combines it with early stopping to reduce the training time. The experimental results show that after introducing mixed-precision training, the training speed of the model was significantly improved and there was no significant decrease in model accuracy.
BibTeX
@inproceedings{zhu2025application,
  author = {Jun Zhu and Jiwei Xu and Lei Feng and Hao Zhang},
  title = {{Application of Mixed Precision Training in Human Pose Estimation Model Training}},
  booktitle = {Processes},
  year = {2025},
  doi = {10.3390/pr13061894},
}
Figure