Standard gradient descent assumes a world of infinite numerical precision. In theory, optimization moves smoothly along a continuous loss landscape by making infinitesimal adjustments to model parameters. In practice, however, deep learning models are constrained by physical hardware that relies on finite-bit representations.
As the deep learning community shifts toward low-precision formats—such as FP16, BF16, INT8, and even FP4—to reduce memory overhead and accelerate training, optimization encounters a hard numerical wall. This phenomenon, known as quantization underflow or precision-induced update loss, occurs when critical learning signals are rounded to zero, causing the optimization process to stall entirely.
The Mechanics of Update Truncation
To understand why low-precision training stagnates, we must look at how a weight update is calculated and applied. In a standard training loop, a weight $w$ is updated using a gradient $g$ and a learning rate $\eta$:
$$w_{t+1} = \text{round}(w_t - \eta \cdot g_t)$$
In a low-precision environment, this operation is highly susceptible to two distinct forms of underflow:
- Absolute Underflow: The product of the gradient and the learning rate ($\eta \cdot g_t$) is so small that it falls below the minimum representable non-zero value of the chosen format. As identified by Micikevicius et al., a significant portion of gradients in deep networks are too small to be expressed in reduced floating-point formats, causing them to vanish directly to zero during backpropagation.
- Relative Underflow (Rounding Loss): Even if the update $\eta \cdot g_t$ is large enough to be represented on its own, it may be miniscule relative to the magnitude of the weight $w_t$ it is modifying. When the update is added to the weight, the limited mantissa (significand) bits of the low-precision format cannot capture such a fine-grained difference. As a result, the update is truncated, and the weight remains unchanged.
Gupta et al. and Sun et al. demonstrate that standard deterministic rounding to the nearest value frequently zeroes out these small updates. When thousands of parameters simultaneously experience this truncation, the model loses its ability to accumulate subtle, incremental adjustments over time.
The "Stuck State" and Numerical Dead Zones
When updates are consistently rounded to zero, the optimizer enters what Blumenfeld describes as a stuck state. This is not a failure of the loss landscape itself; the underlying mathematical gradients may still point toward a better solution. Instead, it is a failure of execution. The optimizer becomes trapped in local minima or zero-initialized states simply because the hardware lacks the resolution to execute the escape steps.
This creates a numerical dead zone in the parameter space. As explored by Mitarchuk et al., when updates drop below the machine's epsilon, the learning process effectively freezes. This is particularly problematic in two phases of training:
- Late-Stage Convergence: As a model nears an optimum, its gradients naturally shrink. Chmiel et al. note that a fundamental challenge in low-precision training is the point where this shrinking gradient signal becomes indistinguishable from quantization noise, causing accuracy to plateau prematurely.
- Highly Saturated Configurations: When weights grow large, the precision floor rises. Dobler et al. observe that certain layers with large weight magnitudes (such as normalization layers) can stop updating entirely because the relative precision gap becomes too wide for standard gradient signals to bridge.
Exacerbating Factors
The risk of quantization underflow is not uniform across all training configurations; it is severely compounded by specific architectural and algorithmic choices.
Large Batch Sizes
To maximize hardware utilization, practitioners often scale up mini-batch sizes. However, Redgrave demonstrates that standard "mean loss" formulations scale down gradient magnitudes proportionally to the batch size. When these already-small gradients are compressed into low-precision formats like 16-bit floating point, they are pushed directly into the underflow zone, inadvertently halting optimization.
Extreme Quantization
Moving from 16-bit to 4-bit or fixed-point integer spaces drastically narrows the dynamic range. In low-bit integer environments, Kim et al. highlight a brutal tradeoff between clipping large-magnitude gradients and rounding off small ones. Prioritizing the dense cluster of small gradients requires a fine quantization resolution, which clips the massive gradients essential for stability. Conversely, expanding the range to capture large gradients causes the smaller, fine-grained updates to disappear entirely into the quantization gap.
Overcoming the Precision Floor
To prevent update stagnation without reverting to power-hungry 32-bit floating-point math, researchers have engineered mechanisms that preserve the statistical integrity of small updates.
Stochastic Rounding
The most prominent defense against rounding loss is stochastic rounding. Instead of deterministically truncating a fractional value to the nearest representable integer, stochastic rounding treats the fractional remainder as a probability.
If an update falls between two representable hardware states, it is rounded up or down probabilistically. Gupta et al. and Yuan et al. show that this allows infinitesimal updates to be preserved in expectation. Over many iterations, the expected value of the updates matches the true gradient signal, enabling the model to break out of numerical dead zones and converge successfully.
Dynamic Precision and Scaling
Another approach relies on monitoring the health of the gradient flow. Luo et al. propose tracking the ratio between average gradient magnitudes and the minimum representable value of the current bit-width. When this metric signals that updates are vanishing into the quantization floor, the system dynamically increases the precision or adjusts the quantization step size, ensuring that the learning signal remains visible to the hardware.