TL;DR
Running AI weather prediction models in half-precision (fp16) instead of the standard 32-bit (fp32) can reduce energy consumption by up to 3.7x and speed up inference by up to 3.5x with negligible loss in forecast accuracy. The secret sauce is a hybrid approach: keep the physical state and normalization statistics in fp32 while running the heavy-lifting neural network layers in fp16.
What Problem Does This Paper Address?
AI weather models like GraphCast and Pangu-Weather have revolutionized forecasting, matching or beating traditional physics-based models while running orders of magnitude faster. However, "faster" is relative. These models are massive—often exceeding a billion parameters—making them difficult to run on consumer-grade hardware.
While the industry standard is to release and run these models in 32-bit floating-point (fp32), modern GPUs are designed to be much faster at 16-bit operations. The challenge is that weather forecasting is an "autoregressive" task: the model’s output at one time step becomes the input for the next. In such chaotic systems, small rounding errors can compound over a 10-day forecast, potentially "blowing up" the simulation or significantly degrading accuracy. This paper investigates whether we can slash the computational cost using reduced precision without losing the "skill" (accuracy) of the forecast.
Methods
The author evaluated five prominent AI weather models: Aurora, Pangu-Weather, FourCastNet, SFNO, and FastNet. These represent different architectural families, including Transformers, Graph Neural Networks, and Fourier-based models.
The key methodological contribution is the persistent conversion strategy. Instead of using "automatic mixed precision" (which keeps a copy of weights in fp32 and is memory-intensive), the author converted the network weights once to fp16. Crucially, the "physical state" (the actual weather data being passed between steps) and the normalization/denormalization layers were kept in fp32. This prevents the numerical overflows that occur when large physical values (like surface pressure in Pascals) are squeezed into the limited range of fp16.
The author also conducted "one-at-a-time" sensitivity experiments to see which specific components (encoder, backbone stages, decoder) were most sensitive to rounding errors.
Datasets
The models were initialized and verified using the ERA5 reanalysis dataset, the gold standard for atmospheric data. The study used 52 initial dates from the year 2024 (stratified to cover different weeks and times of day). This year was specifically chosen because it fell outside the training periods of all models involved, ensuring a fair "out-of-sample" evaluation.
Key Insights and Ideas
- The fp32 State is Non-Negotiable: You cannot simply convert the entire model to 16-bit. Physical units like pressure (e.g., 101,325 Pa) exceed fp16’s maximum value of 65,504, leading to immediate "NaN" (Not a Number) errors. By keeping the state in fp32 and only rounding values as they enter the neural network, the models remain stable.
- fp16 > bf16 for Inference: While Bfloat16 (bf16) is popular for training because it shares the wide range of fp32, it has fewer "mantissa" bits (precision). For inference, where we've already solved the range issue by keeping the state in fp32, fp16 is significantly more accurate because it offers more precision.
- Rounding as a Perturbation: In chaotic systems like the atmosphere, fp16 rounding errors act like tiny physical perturbations. The paper found that the fp16 forecast drifts away from the fp32 version, but it remains just as physically plausible and accurate against the "truth" (ERA5).
- Architecture Matters: Sensitivity to precision isn't just about model size. For example, SFNO (a Fourier-based model) was the most robust to fp16, while FourCastNet showed a slight "tropical penalty" in its accuracy.
Key Results
The quantitative gains are massive for a "practically lossless" change:
- Speed & Energy: 10-day forecasts were 1.3x to 3.5x faster and used 1.4x to 3.7x less energy.
- Accuracy: The global Root Mean Square Error (RMSE) changed by less than 0.5% for four out of five models. To put that in perspective, the accuracy loss is equivalent to losing less than one hour of lead time on an 8-day forecast.
- Memory: Peak memory usage dropped significantly, allowing the 1.26-billion-parameter Aurora model to fit comfortably on a single 16 GB consumer GPU without complex weight-streaming tricks.
Limitations and Future Work
- Hardware Specificity: The results assume hardware with fast fp16 arithmetic (like NVIDIA GPUs). On Google TPUs, which are optimized for bf16, the trade-offs might differ.
- 8-bit Quantization: The study found that simple 8-bit (int8) quantization is not a drop-in replacement and causes significant skill loss. Future work could explore more sophisticated 8-bit or 4-bit techniques (like block scaling) used in Large Language Models.
- Autoregressive Growth: While errors were small over 10 days, the paper notes that for climate-scale simulations (years or decades), these tiny rounding errors might eventually lead to different climatic drifts.
Summary updated