Deep and Probabilistic Solar Irradiance Forecast at the Arctic Circle

N. Erdmann, Lars Ødegaard Bentsen, Roy Stenbro, Heine Nygard Riise, Narada Dilp Warakagoda, Paal Engelstad2024 · arXiv

Abstract

Solar energy forecasting at high latitudes presents unique challenges due to extreme seasonal variations and limited satellite coverage. This paper introduces a probabilistic deep learning framework using Long Short-Term Memory (LSTM) networks for 36-hour solar irradiance forecasting in the Arctic Circle. By incorporating clear-sky injection and skewed probability distributions like Johnson’s SB, the proposed models improve both point accuracy and uncertainty quantification compared to traditional baselines.

Introduction

The day-ahead energy market requires reliable solar irradiance forecasts up to 36 hours in advance. Near the Arctic Circle, this task is complicated by polar nights, low solar angles, and sparse geostationary data. While most research focuses on short-term point forecasts, grid operators need long-horizon uncertainty estimates to manage risk. This study addresses this gap by applying probabilistic deep learning to high-latitude solar forecasting.

Methodology

The core of the framework is an LSTM network, chosen for its ability to model temporal dependencies. Two primary enhancements are introduced:

  1. Clear Sky Injection: Theoretical clear-sky values are injected into the output layer, allowing the model to focus on weather-induced deviations rather than learning the basic solar cycle from scratch.
  2. Probabilistic Modeling: The LSTM is extended via Quantile Regression (QR) for non-parametric interval prediction and Maximum Likelihood Estimation (MLE) for parametric distribution modeling. To account for the inherent skewness in solar data, the authors utilize skewed distributions, specifically Johnson’s SB (bounded), Johnson’s SU (unbounded), and Weibull. This is a novel application of Johnson’s distributions in the context of solar forecasting.
Comparison of probabilistic forecasts across different models showing observations and predicted uncertainty bands.

Results and Discussion

The models were evaluated using five years of data from 20 stations in Southern Norway. A key finding is the superior performance of the Johnson’s SB distribution in point-prediction metrics, likely due to its bounded nature matching the physical limits of irradiance.

Performance Metrics

As shown in the table below, the MLE-JSb-LSTM model achieved the best point accuracy with an MAE of 48.984 W/m². However, a trade-off exists between accuracy and calibration. While MLE models excel in MAE, the QR-LSTM model provides the most reliable uncertainty intervals, as evidenced by the lowest Average Coverage Error (ACE).

Summary of performance metrics for various forecasting models.

Seasonal and Calibration Insights

Forecasting remains more challenging in summer due to higher absolute irradiance values and potential for larger errors. Additionally, the study highlights that QR-based models are better "calibrated"—their predicted intervals more accurately reflect the actual error distribution—making them preferable for risk-averse applications. The authors also noted that MLE training can be sensitive and often requires manual constraints on distribution parameters to achieve stability.

Conclusion

This research demonstrates that combining LSTMs with skewed probability distributions significantly enhances solar forecasting in challenging Arctic conditions. The inclusion of clear-sky injection and bounded distributions like Johnson's SB provides a robust framework for long-horizon forecasting. Future work will explore Graph Neural Networks for spatial encoding and Variational Inference to stabilize the MLE training process.

Summary updated

Abstract

Solar irradiance forecasts can be dynamic and unreliable due to changing weather conditions. Near the Arctic circle, this also translates into a distinct set of further challenges. This work is forecasting solar irradiance with Norwegian data using variations of Long-Short-Term Memory units (LSTMs). In order to gain more trustworthiness of results, the probabilistic approaches Quantile Regression (QR) and Maximum Likelihood (MLE) are optimized on top of the LSTMs, providing measures of uncertainty for the results. MLE is further extended by using a Johnson's SU distribution, a Johnson's SB distribution, and a Weibull distribution in addition to a normal Gaussian to model parameters. Contrary to a Gaussian, Weibull, Johnson's SU and Johnson's SB can return skewed distributions, enabling it to fit the non-normal solar irradiance distribution more optimally. The LSTMs are compared against each other, a simple Multi-layer Perceptron (MLP), and a smart-persistence estimator. The proposed LSTMs are found to be more accurate than smart persistence and the MLP for a multi-horizon, day-ahead (36 hours) forecast. The deterministic LSTM showed better root mean squared error (RMSE), but worse mean absolute error (MAE) than a MLE with Johnson's SB distribution. Probabilistic uncertainty estimation is shown to fit relatively well across the distribution of observed irradiance. While QR shows better uncertainty estimation calibration, MLE with Johnson's SB, Johnson's SU, or Gaussian show better performance in the other metrics employed. Optimizing and comparing the models against each other reveals a seemingly inherent trade-off between point-prediction and uncertainty estimation calibration.
BibTeX
@misc{erdmann2024deep,
  author = {N. Erdmann and Lars Ødegaard Bentsen and Roy Stenbro and Heine Nygard Riise and Narada Dilp Warakagoda and Paal Engelstad},
  title = {{Deep and Probabilistic Solar Irradiance Forecast at the Arctic Circle}},
  year = {2024},
  eprint = {2410.07806},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.07806},
}
Figure