Abstract
Training ultra-low precision (3-bit and 4-bit) YOLO models is notoriously difficult due to weight and scale factor "jitter" or oscillations during training, which leads to significant accuracy degradation. This paper introduces two primary interventions: Exponential Moving Average (EMA) to stabilize training and a one-epoch "Quantization Correction" (QC) step to refine the final model. These methods effectively close the performance gap between ultra-low precision and full-precision object detectors on standard benchmarks.
Introduction: The Challenge of Low-Bit YOLO
Quantization-Aware Training (QAT) is essential for deploying deep learning models on edge devices with limited resources. By reducing 32-bit floating-point parameters to 3 or 4 bits, models can be significantly compressed. However, QAT typically relies on the Straight Through Estimator (STE) to approximate gradients for non-differentiable rounding operations.
In compact architectures like YOLO, the STE frequently induces oscillations. Weights and learnable scale factors fail to converge, instead jumping across quantization boundaries. This instability prevents the model from reaching an optimal state, resulting in a substantial accuracy penalty compared to full-precision counterparts.
Key Insights: Beyond Weight Oscillations
A critical finding of this work is that oscillations are not limited to weights. Learnable scale factors for activations—which define the quantization range—also exhibit significant jitter. This instability in activation quantization is a major contributor to performance loss in YOLO architectures.
Furthermore, analysis suggests that latent weights oscillating near boundaries may actually be closer to an "optimal" continuous state than if they were strictly forced to the nearest integer. This observation highlights the limitations of rigid per-tensor quantization schemes and the need for more flexible correction mechanisms.
Proposed Methodology
The authors propose a two-stage stabilization and correction pipeline:
- Exponential Moving Average (EMA): To mitigate noise, EMA is applied to both latent weights and learnable scale factors during training. This acts as a low-pass filter, smoothening the trajectories of these parameters and helping the model settle into a more stable, representative configuration.
- Post-hoc Quantization Correction (QC): Following standard QAT, a lightweight "tune-up" is performed. A learnable affine (scale and shift) transformation is added to the pre-activations of every layer. With the rest of the network frozen, these correction parameters are trained for a single epoch on a small calibration set. This step compensates for residual errors from weights that remained near quantization boundaries. Crucially, these parameters can be folded into existing Batch Normalization layers, incurring zero additional inference cost.
Experimental Results
The methodology was evaluated using the COCO dataset across various YOLOv5 and YOLOv7 models. The combination of EMA and QC consistently outperformed standard LSQ-based QAT:
- YOLOv5-n (3-bit): Achieved a Mean Average Precision (mAP) of 18.2, up from 15.2 with standard methods.
- YOLOv5-s (4-bit): Reached 34.0 mAP, coming within 3.4 points of the full-precision baseline (37.4 mAP).
- Generalization: The improvements were robust across both object detection and semantic segmentation tasks, setting a new state-of-the-art for low-precision YOLO.
Conclusion and Future Work
This work demonstrates that taming oscillations in both weights and scale factors is key to effective low-bit quantization for YOLO. While the current per-channel QC is highly effective, future research may explore more granular correction factors or automated estimation techniques to further enhance accuracy. Additionally, applying this stabilization pipeline to other architectures like Transformers could broaden its impact on efficient deep learning.
Summary updated