Abstract
Accurate electricity demand forecasting is critical for grid stability but faces significant challenges in data-scarce, "few-shot" scenarios. We present DOMINO (ranDOM walk on gaussIaN prOcesses), a novel algorithm that performs a random walk across an ensemble of Gaussian Processes (GPs). By dynamically switching between models based on historical performance, DOMINO achieves substantial accuracy gains over existing multi-task GP methods like MAGMA, particularly as sequence lengths increase, while maintaining computational frugality.
Introduction
Predicting regional electricity consumption is a high-stakes task where errors can lead to price volatility or grid instability. While deep learning has revolutionized many fields, it often struggles with scenarios where data is scarce or highly granular. Gaussian Processes (GPs) are considered the "gold standard" for such tasks because they naturally provide uncertainty estimates; however, they are notoriously expensive to train on large datasets.
This paper introduces DOMINO, a frugal approach that leverages the power of GP ensembles without the prohibitive computational baggage. By intelligently navigating pre-existing models, DOMINO offers a more accurate and efficient path for short-term electricity demand prediction.
Methodology
DOMINO is built upon the MAGMA (Multi-tAsk GPs with common MeAn) framework, which shares information across different time series by learning a common mean. DOMINO extends this capability through several key mechanisms:
- The Ensemble: The process begins with a set of GPs trained via MAGMA.
- The Random Walk: Instead of relying on a single "best" prediction, DOMINO performs a random walk. At each time step, the algorithm "jumps" between different sampled time series from the GPs.
- Probabilistic Switching: The probability of jumping to a specific GP is determined by its historical performance.
- Iterative Refinement: The walk is repeated over several "epochs," updating weights based on the Kullback-Leibler (KL) divergence to measure how much probability distributions differ.
- Stopping Criteria: The algorithm terminates once the random walk converges or a maximum number of epochs is reached.
Experimental Results
The researchers evaluated DOMINO using synthetic periodic data to rigorously test the algorithm's logic against the baseline MAGMA algorithm.
Performance and Scaling
DOMINO demonstrated superior stability and accuracy compared to MAGMA. While MAGMA's error rates fluctuated or increased as the sequence length ($N$) grew, DOMINO remained remarkably consistent.
As shown in the results, at a sequence length of $N=200$, MAGMA recorded a Median Absolute Error (MAE) of 48.108, while DOMINO achieved a significantly lower MAE of 3.524. In 10-run cross-validation tests at $N=250$, DOMINO maintained an MAE of 4.608 compared to MAGMA’s 56.215.
Hyperparameter Optimization
The effectiveness of the random walk is influenced by the number of training iterations. Sensitivity analysis revealed that the model performed best with a maximum of 30 training epochs and a 5% tolerance threshold for divergence.
Discussion and Future Work
DOMINO represents a significant advancement in "frugal" learning for time-series forecasting. By using a random walk to navigate existing models rather than retraining complex architectures from scratch, it achieves high accuracy with minimal data. Crucially, because it is built on GPs, DOMINO retains the ability to quantify uncertainty, which is vital for risk management in energy markets.
Immediate next steps for this research include:
- Real-World Validation: Applying DOMINO to actual French regional electricity consumption data (RTE’s eco2mix).
- Irregular Grids: Adapting the model to handle missing or irregularly spaced data points.
- Covariate Integration: Incorporating external variables like temperature or public holidays to improve real-world accuracy.
Reference: Hashimoto-Cullen, C., & Guedj, B. (2024). Predicting Electricity Consumption with Random Walks on Gaussian Processes. arXiv:2409.05934v1 [cs.LG].
Summary updated