Reconstruction of Long-Term Historical Electricity Demand Data

Reshmi Ghosh, Michael Craig, H. Scott Matthews, Laure Berti‐ÉquilleICML 2021

TL;DR

This paper tackles the challenge of reconstructing historical hourly electricity demand data for US Balancing Authorities (BAs) to better understand the impact of natural temperature variability and climate change on grid planning. They developed a novel "back-forecasting" approach using Long Short-Term Memory (LSTM) networks, outperforming traditional methods and providing crucial insights for a cleaner energy future.

What Problem Does This Paper Address?

Planning a robust and reliable power system for the future, especially one incorporating intermittent renewable energy sources like solar and wind, requires a deep understanding of electricity demand patterns. Demand is highly sensitive to weather, particularly temperature. As the climate changes, this sensitivity is expected to amplify, but disentangling the effects of natural year-to-year temperature fluctuations (Inter-annual Variability or IAV) from long-term climate change trends is difficult. Current historical electricity demand datasets are often insufficient, especially at the granular level of Balancing Authorities (BAs) which manage grid stability for specific regions. This lack of long-term, hourly data hinders accurate capacity gap analysis and the development of effective grid planning strategies. This paper aims to fill this gap by reconstructing historical demand data to better understand the influence of natural temperature variability.

Methods

The authors propose a "back-forecasting" methodology, which is analogous to traditional forecasting but works backward in time. Instead of predicting future demand, they use recent historical data (2015-2019) to reconstruct past demand (1980-2014). They compare two primary modeling approaches:

  • Piece-wise Linear Regression: An enhanced linear regression method capable of modeling non-linear dependencies, but with limitations in generalizing to large datasets.
  • Long Short-Term Memory (LSTM) Networks: A type of recurrent neural network well-suited for time-series data, capable of capturing complex, non-linear relationships and temporal dependencies. The LSTM model was chosen for its ability to handle vanishing and exploding gradients and its memory state.

For the LSTM model, they employed embeddings for categorical variables (fixed effects like hour of day, day of week, month, year) and a fixed sequence length of 24 hours to capture past temperature influences. They also experimented with different architectures, ultimately finding a single-layer LSTM with dropout and fully connected layers to be the most effective.

Datasets

  • Training/Validation Data: Hourly electricity demand records (in megawatts) from the Electric Reliability Council of Texas (ERCOT) and hourly temperature data (in Kelvin, converted to Celsius) from the MERRA reanalysis dataset, spanning from 2015 to 2019.
  • Test Data: The reconstructed hourly demand data for the period 1980-2014, leveraging MERRA temperature data which is available back to 1980.

The authors specifically chose MERRA reanalysis data over Global Climate Models to focus on natural IAV rather than climate change projections.

Key Insights and Ideas

  • Back-forecasting is a viable approach: The paper demonstrates that by leveraging recent data and advanced models like LSTMs, it's possible to reconstruct historical electricity demand data, overcoming the limitations of missing historical records.
  • LSTM excels over traditional methods for this task: The LSTM architecture significantly outperformed piece-wise linear regression, particularly on larger datasets, highlighting its superior ability to capture the complex, non-linear relationship between temperature and electricity demand.
  • Winter demand is more complex than summer: The models struggled more with accurately back-forecasting winter demand compared to summer demand. This is attributed to a potentially non-monotonic relationship between winter temperature and demand, and the occurrence of extreme winter weather events.
  • Regional specificity is crucial: The study highlights that a "one-model-fits-all" approach is not suitable. Different regions (e.g., Texas vs. New England) exhibit distinct temperature-demand relationships and inter-annual variability, necessitating separate models for different Balancing Authorities.
  • Hourly data is essential for capacity planning: Visualizing the top twenty hourly demand peaks over decades (Figures 5 and 6) underscores the importance of granular, hourly data for understanding demand variability and ensuring adequate power system capacity, something aggregated annual data cannot provide.

Key Results

  • The best LSTM architecture achieved a Root Mean Squared Error (RMSE) of 1643 and an R-squared (R²) of 0.87 for Dallas, significantly outperforming piece-wise regression (RMSE: 1643, R²: 0.73).
  • Summer demand back-forecasting showed high accuracy, with predictions falling within ± standard error of the ground truth (R²summer ≈ 0.94).
  • Winter demand back-forecasting was less accurate, with R²winter ≈ 0.48 for a single model.
  • Using separate models for summer and winter improved winter performance to R²winter ≈ 0.76, but it still lagged behind summer performance.
  • The study successfully back-forecasted hourly demand for Dallas and the NEMA region (Massachusetts/Boston), demonstrating the model's generalizability across different weather zones.

Limitations and Future Work

  • Winter Model Performance: The authors acknowledge that the winter model's performance is still underperforming and suggest that additional weather variables like humidity might be necessary. They also note the need to identify and model extreme winter events (snow spells, tornadoes) that can cause anomalous demand.
  • Generalizability to All BAs: While the model was tested on two different regions, the authors plan to automate the process to reconstruct demand data for all 66 US Balancing Authorities.
  • Spatial Heterogeneity: To improve accuracy for BAs that don't align with major population centers, they plan to use GIS data to extract temperature more precisely.
  • Anomaly Detection: Future work includes detecting temperature anomalies and modeling outliers to enhance the robustness of the winter back-forecasting architecture.
  • Transfer Learning: The potential for using transfer learning techniques to adapt models across BAs is also mentioned.

Summary updated

BibTeX
@inproceedings{ghosh2021reconstruction,
  author = {Reshmi Ghosh and Michael Craig and H. Scott Matthews and Laure Berti‐Équille},
  title = {{Reconstruction of Long-Term Historical Electricity Demand Data}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
}
Figure