An Adaptive Deep Learning Framework for Day-ahead Forecasting of Photovoltaic Power Generation

Xing Luo, Dongxiao ZhangarXiv 2021 · arXiv

Introduction

Photovoltaic (PV) power generation is a critical component of the transition to renewable energy, yet its inherent volatility—driven by fluctuating weather patterns and solar irradiance—poses significant challenges for grid stability. While Deep Learning models, particularly Long Short-Term Memory (LSTM) networks, have shown promise in capturing these complex temporal relationships, they are typically deployed as static, offline-trained models.

Once deployed, these standard models suffer from two primary limitations: they cannot learn from real-time data streams, and they are vulnerable to "concept drift." Concept drift occurs when the underlying statistical distribution of data shifts due to seasonal changes or hardware modifications (e.g., adding or losing PV panels), leading to a rapid decline in forecasting accuracy. To address these issues, this research introduces AD-LSTM, an adaptive deep learning framework designed for online learning and drift resilience.

The AD-LSTM Framework

The AD-LSTM framework integrates three core components to maintain high accuracy in dynamic environments: a two-phase learning strategy, a drift-detection algorithm, and a similarity-based data selection method.

Two-Phase Adaptive Learning Strategy (TP-ALS)

Rather than relying on a single training pass, the framework utilizes a Two-Phase Adaptive Learning Strategy (TP-ALS). In Phase 1 (Offline Stationary Learning), the model is pre-trained on a large historical dataset to capture general solar generation patterns. In Phase 2 (Online Adaptive Learning), the model is continuously fine-tuned using a small "supplementary dataset" of recent or highly similar days. This approach allows the model to stay "fresh" without the computational burden of full re-training.

The Two-Phase Adaptive Learning Strategy (TP-ALS) flowchart illustrating the transition from offline pre-training to online fine-tuning based on drift detection.

Concept Drift Detection with SDWIN

To manage shifts in data distribution, the framework employs a Sliding Window (SDWIN) algorithm. This mechanism acts as a monitor for the system's Mean Square Error (MSE). By evaluating batches of newly arrived data, the algorithm can detect if errors exceed a calculated threshold for a sustained period, signaling a concept drift.

The sliding window mechanism used to monitor newly-arrived data streams and evaluate model performance in real-time.

When drift is detected—such as after a hardware upgrade—the system adjusts its fine-tuning strategy to prioritize the most recent "adjacent days" over historical "similar days," allowing the model to rapidly adapt to the new system state.

Weighted K-Nearest Neighbor (W-KNN)

Effective fine-tuning requires high-quality training samples. The framework uses a Weighted K-Nearest Neighbor (W-KNN) algorithm to identify historical days that are meteorologically similar to the target forecast day. By using Pearson Correlation Coefficients, the algorithm weights critical variables like solar radiation more heavily than less relevant noise, ensuring the fine-tuning process is grounded in the most relevant data.

Experimental Evaluation

The framework was tested using 27 months of real-life data from a PV plant in Australia (GEFCom2014), combined with meteorological data from the European Center for Medium-range Weather Forecasts (ECMWF). The first 15 months were used for pre-training, while the remaining 12 months simulated a live data stream.

Key Results

  • Superior Accuracy: AD-LSTM consistently outperformed traditional offline models (OL-LSTM), Persistence models, and ARIMA across all seasons, achieving lower MAE and higher $R^2$ scores.
  • Drift Resilience: In simulations of hardware changes (adding PV units), the performance of static models collapsed. In contrast, AD-LSTM maintained high accuracy, showing an improvement rate of over 90% in MSE compared to the offline version in drift scenarios.
  • Efficiency: The study demonstrated that fine-tuning on just 3–9 days of relevant data is sufficient to maintain high performance, making the framework computationally viable for real-world deployment.

Conclusion

The AD-LSTM framework successfully bridges the gap between static deep learning models and the dynamic reality of solar power generation. By combining online fine-tuning with robust drift detection, it provides a stable and accurate solution for day-ahead forecasting. Future work may explore extending this adaptive framework to other volatile energy sectors, such as wind power and electricity price forecasting, or integrating more advanced architectures like Transformers.

Summary updated

Abstract

Accurate forecasts of photovoltaic power generation (PVPG) are essential to optimize operations between energy supply and demand. Recently, the propagation of sensors and smart meters has produced an enormous volume of data, which supports the development of data based PVPG forecasting. Although emerging deep learning (DL) models, such as the long short-term memory (LSTM) model, based on historical data, have provided effective solutions for PVPG forecasting with great successes, these models utilize offline learning. As a result, DL models cannot take advantage of the opportunity to learn from newly-arrived data, and are unable to handle concept drift caused by installing extra PV units and unforeseen PV unit failures. Consequently, to improve day-ahead PVPG forecasting accuracy, as well as eliminate the impacts of concept drift, this paper proposes an adaptive LSTM (AD-LSTM) model, which is a DL framework that can not only acquire general knowledge from historical data, but also dynamically learn specific knowledge from newly-arrived data. A two-phase adaptive learning strategy (TP-ALS) is integrated into AD-LSTM, and a sliding window (SDWIN) algorithm is proposed, to detect concept drift in PV systems. Multiple datasets from PV systems are utilized to assess the feasibility and effectiveness of the proposed approaches. The developed AD-LSTM model demonstrates greater forecasting capability than the offline LSTM model, particularly in the presence of concept drift. Additionally, the proposed AD-LSTM model also achieves superior performance in terms of day-ahead PVPG forecasting compared to other traditional machine learning models and statistical models in the literature.
BibTeX
@misc{luo2021adaptive,
  author = {Xing Luo and Dongxiao Zhang},
  title = {{An Adaptive Deep Learning Framework for Day-ahead Forecasting of Photovoltaic Power Generation}},
  year = {2021},
  eprint = {2109.13442},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.13442},
}
Figure