The Cost of Coverage: Conformalising Native Forecast Uncertainty for GB Wind Power

Carl SomersarXiv 2026 · arXiv

TL;DR

This paper demonstrates how to "repair" the unreliable uncertainty bands of wind power forecasts using conformal prediction. By applying regime-based (Mondrian) calibration, the researchers successfully restored 90% coverage across various forecasting models, specifically slashing missed energy in high-wind scenarios by over 40% without significantly increasing the overall width of the prediction intervals.

What Problem Does This Paper Address?

As wind becomes a dominant source of electricity, grid operators face a massive challenge: balancing the system when the wind doesn't blow as expected. In Great Britain, a single day of poor forecasting can lead to balancing costs exceeding £20 million.

To manage this, engineers rely on "prediction intervals"—uncertainty bands that should contain the actual power output a certain percentage of the time (e.g., 90%). However, most forecasting models (from simple ARIMA to complex neural networks) produce "native" bands that are often miscalibrated. They might achieve 90% coverage on average (marginal coverage) but fail miserably during high-wind events when the stakes are highest. This paper investigates whether Conformal Prediction (CP)—a framework for generating uncertainty guarantees—can fix these bands under realistic operational conditions without making them so wide they become useless.

Methods

The authors take six different forecaster classes—ranging from classical ARIMA and LightGBM to the TabPFN foundation model—and "wrap" them in conformal calibration layers.

The primary technique used is Online Sequential Split Conformal Prediction (OSSCP). This method adjusts the native bands based on the model's most recent errors. Crucially, they test Mondrian binning, where the calibration is performed separately for different wind regimes (Low: 0-5 m/s, Average: 5-11 m/s, and High: 11+ m/s). This allows the model to learn that it might need wider bands in stormy weather than on a calm day.

The study is designed to be "operationally realistic," meaning it accounts for data publication lags (6-hour delays for weather forecasts) and ensures no "data leakage" occurs during the calibration process.

Datasets

The study utilizes open SCADA (Supervisory Control and Data Acquisition) data from two operational wind farms in Great Britain:

  • Kelmarsh: 6 turbines, 12.3 MW capacity (East Midlands).
  • Penmanshiel: 14 turbines, 28.7 MW capacity (South of Scotland).

Weather data was sourced from the NOAA GEFS (Global Ensemble Forecast System) for real-time forecasting and ERA5 re-analysis for retrospective auditing of actual weather conditions. The authors have released the full pipeline and notebooks to serve as an open operational benchmark.

Figure 4: Farm layouts and corresponding Grid Supply Point (GSP) groups. Kelmarsh: 6 turbines, 12.3 MW, East Midlands, evaluated in 2024. Penmanshiel: 14 turbines, 28.7 MW, South of Scotland, evaluated in 2023.

Key Insights and Ideas

  • The Mirage of Marginal Coverage: A model can look perfectly calibrated on average while being dangerously wrong in specific conditions. The authors found that standard pooled calibration often hides significant under-coverage in high winds by over-covering in low winds.
  • Mondrian Binning is Essential: Calibrating by wind regime (Mondrian-OSSCP) is the "secret sauce." It transforms "average" reliability into "regime-level" reliability.
  • The "Cost" is Negligible: Critics of conformal prediction often fear it produces intervals that are too wide to be useful. This paper shows that the Winkler Score (which penalizes both width and misses) was maintained or even improved in 30 out of 36 comparisons.
  • The Weather Bottleneck: Even with perfect conformal wrapping, the system remains vulnerable to "regime misclassification." If the weather model predicts average wind but a storm actually hits, even the best calibration struggles to keep up.

Key Results

The researchers found that conformal calibration effectively "repaired" the marginal coverage for all six forecasters, bringing them close to the 90% target.

Figure 1: The width cost of restoring marginal coverage. Each marker is one forecaster; the grey segment runs from its native band (light) to its OSSCP band (dark), at three horizons and both farms.

The most striking result came from the high-wind regime. Using Mondrian binning:

  • Coverage in high winds improved from as low as 60% (native) to 87-92%.
  • Missed energy volume in high winds was cut by over 40% compared to standard pooled calibration.
  • At the 24-hour (day-ahead) horizon, models like LightGBM-QR required 10–25% wider bands to reach the target, but this "cost" was offset by the increased reliability of the forecasts.
Figure 2: OSSCP versus Mondrian-OSSCP day-ahead coverage and width (CF), both farms pooled, by forecast wind regime. Percentages show regime frequency. Markers follow Figure 1.

Limitations and Future Work

While the results are promising, the authors note several areas for further research:

  • Offshore Generalization: The study focused on two onshore farms; offshore conditions may present different uncertainty profiles.
  • Curtailment: The study excluded periods where turbines were intentionally slowed or stopped by the grid operator. In reality, these "curtailed" periods are common in high winds and add another layer of complexity.
  • AI Weather Models: Future work could compare traditional GEFS forecasts with newer AI-driven weather models to see if better regime prediction further reduces under-coverage.
  • Financial Integration: While the paper measures "missed energy," the next step is evaluating these bands within a full trading optimizer to measure their actual financial value in the energy market.

Summary updated

Abstract

Uncertainty bands on wind forecasts help inform reserve sizing and commitments, but often fail to deliver their stated coverage. We apply conformal prediction to native bands to see what it adds and what it costs, comparing six forecasters, from ARIMA to a tabular foundation model, at two wind farms in Great Britain (GB) under operational conditions. We show that calibration repairs empirical marginal coverage with little or no Winkler Score penalty, but this apparent success conceals under-coverage in high winds. Calibrating by forecast wind regime turns marginal coverage into regime-level coverage, cutting day-ahead out-of-band volume in high winds by over 40% relative to pooled calibration while narrowing bands overall.
BibTeX
@misc{somers2026cost,
  author = {Carl Somers},
  title = {{The Cost of Coverage: Conformalising Native Forecast Uncertainty for GB Wind Power}},
  year = {2026},
  eprint = {2611.09815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09815},
}
Figure