Introduction
General Circulation Models (GCMs) are fundamental tools for understanding long-term climate trends. However, their utility in predicting local extreme events, such as heatwaves, is often limited. A primary reason is that GCMs are designed as climate simulators rather than weather forecasters; they may capture the physical mechanisms of an extreme event but fail to predict its exact timing. Standard evaluation metrics like Root Mean Squared Error (RMSE) severely penalize such timing offsets, leading researchers to aggregate data into monthly or seasonal averages. Unfortunately, this temporal "blurring" obscures the high-resolution information necessary for identifying short-duration extreme heat events.
Methodology: Local Time-Invariance
To address this, researchers have introduced a new skill metric, $\mathcal{L}^w$, based on the principle of local time-invariance. This metric allows for a degree of temporal "slack" by evaluating model performance within a defined "weather window" ($w$).
The core of this approach is an Optimal Bipartite Matching algorithm. Instead of a direct day-to-day comparison, the metric identifies the best possible reordering of simulated days within the window to match observed data. This minimizes the cost (difference) between simulation and observation without requiring strict chronological alignment. This method is more flexible than Dynamic Time Warping and more structurally aware than simple distribution-based metrics like the Wasserstein distance.
The resulting metric is used to assign weights in a Bayesian Model Averaging (BMA) framework. Models that accurately simulate the local behavior and intensity of extremes—even if slightly out of sync—receive higher weights in the final ensemble.
Experimental Setup
The study utilized five models from the CMIP6 archive (GFDL-ESM4, IPSL-CM6A-LR, MPI-ESM1-2-HR, MRI-ESM2-0, and UKESM1-0-LL). ERA5 hourly surface temperatures, resampled to daily means, served as the observational ground truth. While the primary focus was on Nairobi, Kenya, the methodology was validated across eight other global cities, including Paris, Chicago, and Tokyo.
Results and Discussion
The performance of the permutation-based BMA was compared against a standard Multi-Model Mean (MMM) and traditional BMA.
Key findings include:
- Improved Extremes Prediction: In Nairobi, the ERA5 reference recorded 1,162 extreme heat days. The proposed BMA $\pi_{15}$ (using a 15-day window) predicted 994 days, a significant improvement over the 891 days predicted by the MMM.
- Error Reduction: RMSE for temperature during extreme heat days was reduced from 1.36 (MMM) to 1.29 (BMA $\pi_{15}$).
- The "Goldilocks" Window: A 15-day window was found to be optimal. Smaller windows approximate standard RMSE, while larger windows risk losing seasonal context.
The permutation-based approach also resulted in greater "sparsity" in model weighting, more effectively distinguishing skillful models from unskillful ones for specific geographic locations.
Conclusion
By incorporating local time-invariance into model evaluation, we can better leverage GCMs for local risk assessment. While the current improvements in absolute error are modest, the increased accuracy in predicting the frequency of extreme heat days is a significant step forward. Future research will explore the application of this metric to other variables, such as precipitation and wind speed, and investigate its interaction with more advanced bias correction techniques.
Summary updated