Vertical power flow forecast with LSTMs using regular training update strategies

Katharina Brauns, Christoph Scholz, Alexander Schultz, André Baier, Dominik Jöst

TL;DR

This paper tackles the challenge of accurately forecasting vertical power flow in electrical grids, which is crucial for grid stability due to the increasing integration of renewable energy sources. They propose a novel approach using Long Short-Term Memory (LSTM) networks with a regular retraining strategy to adapt to the dynamic and changing characteristics of power flow at transformers, achieving significant performance improvements over baseline models.

What Problem Does This Paper Address?

The electrical grid is facing significant challenges due to the rapid growth of renewable energy sources (like wind and solar) and the increasing volatility of energy consumption. This volatility makes it difficult to predict power flows, especially at transformers connecting different voltage levels (medium to high voltage). These "vertical power flows" are critical for grid operators to ensure safety, prevent overloading, and maintain a balance between energy supply and demand. Traditional forecasting methods struggle because the characteristics of these power flows can change rapidly due to dynamic grid topologies, new installations, and maintenance. This paper aims to develop a more robust forecasting method that can adapt to these changing conditions.

Methods

The core of the proposed solution is a Long Short-Term Memory (LSTM) neural network, a type of recurrent neural network well-suited for time-series forecasting. The paper introduces a novel "regular training update strategy." Instead of training a single model and using it indefinitely, this approach retrains the LSTM model daily using the most recent data. This retraining is designed to capture significant shifts in the transformer's behavior that the initial model, trained on older data, might not have encountered.

To optimize this retraining process, the authors investigate different update strategies, focusing on hyperparameters like the number of training epochs and the learning rate. The goal is to give more weight to recent data without completely discarding the knowledge learned from historical data. They compare this updated LSTM approach against two baseline models: "Persistence Last Measurement" (which simply repeats the last known value) and "Persistence Last Day" (which uses values from the previous day).

Datasets

The study utilizes real-world vertical power flow measurements from seven different transformers connecting medium and high voltage grids. These measurements are collected at a 15-minute resolution. In addition to the power flow data, numerical weather forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) are used as input features. These weather forecasts include parameters like solar radiation, wind speed, temperature, and pressure, with a forecast horizon of up to 10 days. The data spans from January 2016 to May 2018 for training and validation, with a specific split for test data.

Key Insights and Ideas

  • Adaptability is Key: The most significant insight is that the dynamic nature of power grids, particularly the changing characteristics of transformers, necessitates a forecasting model that can adapt over time. A static model, even a sophisticated one like an LSTM, will eventually degrade in performance as grid conditions evolve.
  • Daily Retraining Works: The daily retraining strategy proved highly effective in capturing recent changes in power flow characteristics, leading to substantial improvements in forecast accuracy. This suggests that a frequent update cycle is crucial for maintaining forecast quality in such dynamic environments.
  • Hyperparameter Tuning for Updates Matters: The choice of hyperparameters for the retraining process (specifically, the number of epochs and learning rate) significantly impacts performance. The paper highlights that a smaller number of epochs (e.g., 5) combined with a lower learning rate (e.g., 0.001) often yields the best results, suggesting a balance between learning new patterns and avoiding overfitting to recent, potentially noisy, data.
  • LSTM's Strength: The LSTM architecture, with its ability to handle sequential data and long-term dependencies, provides a strong foundation, but its effectiveness is greatly amplified when combined with a dynamic update mechanism.

Key Results

The proposed LSTM model with regular training updates significantly outperformed both baseline persistence models and a baseline LSTM model without updates.

  • On average, the regular update process achieved an 8% improvement in normalized RMSE compared to the non-updated LSTM model across all transformers and forecast horizons.
  • For some transformers, the improvement was as high as 1.4% (though this seems to be a typo in the text, likely meaning a much higher percentage improvement, as indicated by Figure 8 showing improvements up to 25% for some transformers and horizons).
  • The updated LSTM model consistently outperformed the persistence models, especially for longer forecast horizons (24h and 48h), which are typically harder to predict.
  • The study found that a learning rate of 0.001 generally led to better performance than 0.01, and an epoch number of 5 was often optimal for the retraining process.

Limitations and Future Work

The authors acknowledge that while their approach significantly improves forecasts, there's still room for enhancement.

  • Generalizability: The paper notes that for a few transformers, the performance improvement was marginal or even slightly reduced. Further investigation is needed to understand why this occurs and how to generalize the benefits across all transformers.
  • Hyperparameter Optimization: While they investigated different update strategies, a more exhaustive hyperparameter optimization for the update process could potentially yield even better results.
  • Transfer Learning: The authors suggest exploring transfer learning approaches as a future direction. This could involve leveraging knowledge learned from one transformer to improve forecasts for another, especially in scenarios with limited data or unique grid characteristics.
  • Encoder-Decoder Architectures: They briefly mention comparing their LSTM to an Encoder-Decoder architecture with ConvLSTM2D and Attention layers, but it did not yield better results. They suggest further work to deeply investigate these architectures for this specific problem.
  • Comparison to Existing Methods: They propose comparing their results to the method described in reference [8] for forecasting vertical power flows.

Summary updated

Abstract

The strong growth of renewable energy sources as well as the increasing amount of volatile energy consumption is leading to major challenges in the electrical grid. In order to ensure safety and reliability in the electricity grid, a high quality of power flow forecasts for the next few hours are needed. In this paper we investigate forecasts of the vertical power flow at transformer between the medium and high voltage grid. Forecasting the vertical power flow is challenging due to constantly changing characteristics of the power flow at the transformer. We present a novel approach to deal with these challenges. For the multi step time series forecasts a Long-Short Term Memory (LSTM) is used. In our presented approach an update process where the model is retrained regularly is investigated and compared to baseline models. The model is retrained as soon as a sufficient amount of new measurements are available. We show that this retraining mostly captures changes in the characteristic of the transformer that the model has not yet seen in the past and therefore cannot be predicted by the model without an update process. To give more weight to recent data, we examined different strategies in terms of the number of epochs and the learning rate. We show that our new approach significantly outperforms the investigated baseline models. On average, we achieved an improvement of about 8% with the regular update process compared to the approach without update process.
BibTeX
@inproceedings{brauns2022vertical,
  author = {Katharina Brauns and Christoph Scholz and Alexander Schultz and André Baier and Dominik Jöst},
  title = {{Vertical power flow forecast with LSTMs using regular training update strategies}},
  booktitle = {Energy},
  year = {2022},
  doi = {10.1016/j.egyai.2022.100143},
}
Figure