TL;DR
Standard machine learning libraries like PyTorch and JAX contain subtle numerical bugs in their softmax and cross-entropy implementations that can lead to vanishing or incorrect gradients. This paper demonstrates how to rediscover these bugs using property-based testing and proposes a more robust fix that handles extreme edge cases—like causal masking in Transformers—where previous solutions fail.
What Problem Does This Paper Address?
Softmax and cross-entropy are the fundamental building blocks of modern deep learning. To prevent numerical overflow, libraries use the "max-shift trick," subtracting the maximum value from logits before exponentiating them. However, recent research by Barron et al. (2026) revealed that even with this trick, two issues persist:
- Softmax Collapse: As a model becomes highly confident, the gradient of the hard-target cross-entropy incorrectly rounds to zero.
- Softmax Cancellation: The gradient of soft-target cross-entropy fails to reach zero even when student and teacher logits are identical.
While Barron proposed a fix, this paper identifies a remaining "edge of the edge case" where that fix produces NaN (Not a Number) gradients, particularly in scenarios involving extreme logit differences or causal attention masking.
Methods
The author employs Property-Based Testing (PBT) using the Python library Hypothesis. Unlike traditional unit tests that check specific inputs, PBT defines mathematical properties that should always be true and lets the framework search for inputs that break them.
The paper tests two core properties:
- Property 1: Every row of a softmax Jacobian must sum to zero (in exact arithmetic).
- Property 2: The gradient of the cross-entropy should be exactly zero when the student and teacher logits are identical.
To fix the identified failures, the author proposes a mathematical refinement. While Barron’s fix used a combination of softplus and logsumexp, the author simplifies this to a log1p (log(1+x)) approach:
- Barron's approach:
normalizer = softplus(logsumexp(others)) - Author's approach:
normalizer = log1p(exp(others).sum())
Datasets
This paper does not use traditional training datasets. Instead, it uses the Hypothesis library to procedurally generate tens of thousands of random logit vectors (lists of floats) to stress-test the numerical stability of the functions.
Key Insights and Ideas
- PBT as a Debugging Powerhouse: You don't need a state-of-the-art LLM to find these bugs. Simple, automated property tests can rediscover complex numerical issues that have existed in major libraries for years.
- The Precision Trap: The paper calculates exactly when softmax collapse occurs based on floating-point bits. For
float32, the collapse happens when the margin between the top logit and others exceeds ~16.636. - The "All $-\infty$" Failure: Barron’s fix fails when max-shifting results in a vector of all $-\infty$ (except the max logit which is set to 0). Taking the
logsumexpof an all $-\infty$ vector results in $-\infty$, but its automatic differentiation producesNaN. - Real-World Relevance: This isn't just theoretical. Causal masking in Transformers explicitly uses $-\infty$ to prevent looking at future tokens. A robust softmax must handle these "masked out" values without breaking the gradient flow.
Key Results
The author's implementation successfully passes the property tests where both standard PyTorch and Barron's proposed fix fail.
Specifically, for an extreme input vector like $[10^{308}, -10^{308}]$, the standard implementation gives zero gradients (collapse), and Barron's fix gives NaN gradients. The author's log1p fix provides the mathematically correct result.
Limitations and Future Work
The paper focuses specifically on the softmax and cross-entropy pair. While these are critical, other high-level functions in ML libraries might harbor similar numerical instabilities that haven't been rigorously tested via PBT.
The author suggests that property-based testing should be more widely adopted in the core test suites of libraries like PyTorch, JAX, and NumPy to catch these "silent" numerical errors before they impact model training in production.
Summary updated