Short-Term Load Forecasting for AI-Data Center

Mariam Mughees, Yuzhuo Li, Yize Chen, Yunwei Li2025 · DOI

TL;DR

This paper tackles the challenge of predicting the highly variable power demands of AI-centric data centers, which can destabilize power grids. They propose a data-driven workflow using deep learning models (LSTM, GRU, 1D-CNN) to forecast short-term GPU power consumption, offering a path towards more stable grid operations and efficient data center management.

What Problem Does This Paper Address?

The explosive growth of AI, particularly large language models (LLMs), is driving a massive increase in the power consumption of data centers. These AI workloads are characterized by rapid, unpredictable fluctuations in power demand, sometimes spiking by over 132 kW/s at the rack level. This volatility poses significant challenges for both data center operators and power utilities. Utilities struggle to maintain grid stability and frequency control when faced with such sudden, large power swings. Data center operators need accurate short-term forecasts to dynamically allocate resources, manage costs, and ensure reliable operation. However, the complex, non-linear nature of AI workloads and the "black-box" behavior of the underlying AI algorithms make traditional forecasting methods insufficient.

Methods

The authors propose a three-stage, data-driven workflow for short-term GPU load forecasting:

  1. Data Collection and Pre-Processing: This stage involves gathering detailed GPU power metrics (power, memory utilization, temperature) and other relevant data. The raw data is then cleaned, formatted, and normalized (Min-Max scaling). Crucially, the data is sliced into sequences with a 300-step look-back window to predict 90 steps ahead.
  2. Model Training: They leverage three popular deep learning architectures well-suited for time-series forecasting:
  3. LSTM (Long Short-Term Memory): A type of recurrent neural network (RNN) designed to handle long-term dependencies in sequential data, mitigating the vanishing gradient problem common in standard RNNs.
  4. GRU (Gated Recurrent Unit): A simplified version of LSTM, also effective at capturing sequential patterns.
  5. 1D-CNN (1-Dimensional Convolutional Neural Network): CNNs are adapted to process sequential data, identifying local patterns. These models are trained to minimize the Mean Squared Error (MSE) over the forecasting horizon.

  6. Forecasting and Application: Once trained, the models generate short-term forecasts (seconds to minutes ahead). These forecasts can then be used for downstream applications like proactive maintenance, resource scheduling, and optimizing energy/power management.

Datasets

The study utilizes a real-world dataset from the MIT Supercloud [21]. This dataset comprises 100-millisecond interval logs of GPU/CPU utilization, scheduling details, and physical parameters (like temperature) for GPUs (Nvidia Volta V100) and CPUs (Intel Xeon Gold 6248). The data spans February to October 2021 and captures aggregated GPU power consumption peaking at 45 kW across 448 GPUs. The dataset is rich in workload composition, including vision networks, language models (like BERT), and graph neural networks. For their experiments, the data was pre-processed to a 1-second granularity, with a 300-lookback window and a 90-second prediction horizon. The data was split into training, validation, and testing sets with ratios of 0.7, 0.15, and 0.15, respectively.

Key Insights and Ideas

  • AI Workloads are Fundamentally Different: The paper highlights that AI-centric data centers introduce unique power dynamics, akin to those from renewable energy sources or electric vehicles, due to their high power densities and rapid fluctuations.
  • Deep Learning is Essential: Traditional forecasting models struggle with the complex, high-dimensional, and non-linear patterns of AI power consumption. Deep learning models like LSTM, GRU, and 1D-CNN are shown to be more effective at capturing these intricate dynamics.
  • Short-Term Forecasting is Crucial for Grid Stability: Accurate short-term predictions are vital for enabling grid control mechanisms like Automatic Generation Control (AGC) to respond proactively to upcoming power sags and swells, thus maintaining grid frequency and balance.
  • Data-Driven Approach is Feasible: Despite the challenges of explicit modeling, a data-driven workflow can effectively predict AI data center power loads, opening avenues for improved operational strategies.

Key Results

The study compares the performance of FC-LSTM, GRU, and 1D-CNN models using metrics like RMSE, MAE, SMAPE, and R-squared.

  • FC-LSTM Outperforms: The Fully Connected LSTM (FC-LSTM) consistently achieved the best results, demonstrating the lowest RMSE and MAE, and the highest R-squared value, indicating robust prediction accuracy and low error.
  • GRU and 1D-CNN Performance: GRUs showed slightly lower performance compared to LSTM, with some indication of negative bias. 1D-CNNs had relatively lower performance and a noticeable positive bias.
  • Prediction Accuracy: The models achieved decent accuracy, with 1-minute forecasts closely aligning with actual values. However, the paper notes that capturing sudden dips and peaks remains a challenge, with predictions sometimes lagging actual values, especially in high-variability periods. For instance, the best model (FC-LSTM) achieved an RMSE of 0.5624 and an R-squared of 0.9639.

Limitations and Future Work

  • Capturing Extreme Spikes: While the models show good performance, they struggle to perfectly capture very sudden and extreme power spikes and dips.
  • Future Work: The authors suggest exploring more robust algorithms like liquid neural networks for more resilient and energy-efficient data center operations. They also recommend incorporating more impact factors into the forecasting models, such as environmental conditions and detailed user behavior. The paper also points to the potential of using these forecasts for load-side management strategies, such as integrating Battery Energy Storage Systems (BESSs) for peak shaving, load shifting, and grid resiliency.

Summary updated

Abstract

Recent research shows large-scale AI-centric data centers could experience rapid fluctuations in power demand due to varying computation loads such as sudden spikes from inference or interruption of training large language models (LLMs). As a consequence, such huge and fluctuating power demand pose significant challenges to both data center and power utility operation. Accurate short-term power forecasting allows data centers and utilities to dynamically allocate resources and power large computing clusters as required. However, due to the complex data center power system structure and the black-box nature of the underlying AI algorithms running in data centers, explicit modeling of AI-data center is quite challenging. Alternatively, to deal with this emerging load forecasting problem, we propose a data-driven workflow to predict the short-term electricity load in an AI-data center, and such workflow is compatible with learning-based algorithms such as LSTM, GRU, 1D-CNN. We validate our framework, which achieves decent accuracy on data center GPU short-term power consumption. This provides opportunity for improved power management and sustainable data center operation.
BibTeX
@misc{mughees2025short,
  author = {Mariam Mughees and Yuzhuo Li and Yize Chen and Yunwei Li},
  title = {{Short-Term Load Forecasting for AI-Data Center}},
  year = {2025},
  doi = {10.1109/pesgm52009.2025.11225317},
}
Figure