Abstract
Forecasting has been an essential part of the power and energy industry. Researchers and practitioners have contributed thousands of papers on forecasting electricity demand and prices, and renewable generation (e.g., wind and solar power). This article offers a brief review of influential energy forecasting papers; summarizes research trends; discusses importance of reproducible research and points out six valuable open data sources; makes recommendations about publishing high-quality research papers; and offers an outlook into the future of energy forecasting.
Introduction
Energy forecasting serves as the backbone of modern power system planning and market operations. As the grid transitions toward decentralized and renewable sources, the complexity of predicting load, price, and generation has increased significantly. However, the field currently faces several critical challenges. Research is often fragmented into silos—such as solar versus load forecasting—despite sharing common statistical hurdles. Furthermore, a "reproducibility crisis" persists, as a super-majority of papers rely on proprietary datasets that prevent verification or iterative improvement. Finally, the rise of "black box" deep learning models often ignores the physical characteristics of energy systems, leading to computationally expensive models that lack interpretability.
Methodology
To provide a holistic "bird’s eye view" of the energy forecasting landscape, this review employs a multi-pronged bibliometric and technical analysis. The authors analyzed publication trends over the last decade across 22 major journals to identify growth areas and shifts in research focus.
Beyond bibliometrics, the study synthesizes frontiers in technical modeling, including Deep Learning (CNNs, RNNs), Forecast Combination/Ensembles (such as Quantile Regression Averaging), and Hierarchical Forecasting to ensure consistency across different system levels (e.g., from feeder to system-wide). The authors also provide editorial guidance for high-quality research, emphasizing the necessity of benchmarking against "naive" models and utilizing statistically sound significance tests like the Diebold-Mariano test.
Open Data for Reproducible Research
A core tenet of this review is the call for open science. To combat the reproducibility crisis, the authors highlight six vital categories of open data sources:
- Competition Data: Datasets from the Global Energy Forecasting Competitions (GEFCom 2012, 2014, 2017).
- ISO/RTO Data: Publicly available market and load data from organizations like ISO New England.
- Smart Meter Projects: Data from the Irish smart meter project, Pecan Street, and Low Carbon London.
- In Situ Weather: High-quality ground measurements from BSRN and SURFRAD.
- Remote-Sensed Data: Satellite-derived irradiance and cloud cover data.
- NWP & Reanalysis: Global weather models such as MERRA-2 and ERA5.
Key Insights
- Probabilistic Standards: The industry is shifting from "point forecasts" (single numbers) toward probability distributions. Understanding uncertainty is now as critical as the forecast itself.
- The "Closed-Loop" Reality: As smart grids and price-responsive loads grow, forecasting changes the system's behavior. A high-price forecast may trigger demand response that lowers the actual price. Future models must account for this feedback loop.
- Domain Knowledge Integration: Successful models must integrate physical insights—such as atmospheric dynamics or the 24 solar terms—rather than relying solely on raw data inputs into neural networks.
- Valuation over Accuracy: A model with a lower Mean Absolute Percentage Error (MAPE) is not necessarily superior if it fails to predict high-cost peaks. Forecasts should be valued based on their economic impact on utility operations.
Results and Trends
The bibliometric analysis reveals that while load forecasting still accounts for roughly half of the total literature, the growth of renewable forecasting (wind and solar) is significantly stronger due to global integration efforts.
Empirical evidence also underscores the economic stakes: a mere 1% reduction in forecasting error for a 10,000 MW utility can save up to $1.6 million annually in unit commitment costs. Furthermore, the analysis shows that ensemble models consistently outperform individual "best" models by combining multiple perspectives and calibration windows.
Future Outlook
The next decade of energy forecasting must address "jargon creep" and the prevalence of "too good to be true" results stemming from data leakage. Future research should focus on:
- Interdisciplinary Collaboration: Integrating expertise from meteorologists and economists.
- Practical Error Measures: Developing metrics specifically for probabilistic forecasts.
- High-Resolution Forecasting: Refining temporal and spatial resolution to support the distributed energy transition.
Summary updated