Introduction
Predicting electricity losses within a distribution network is a critical financial challenge for utility companies. In deregulated markets, these companies must purchase energy to cover technical losses—energy dissipated as heat—based on day-ahead forecasts. Inaccurate predictions lead to significant financial exposure, particularly amidst volatile energy prices. This paper presents an upgraded "2.0" forecasting system that leverages a diverse stacked ensemble and non-negative least squares (NNLS) to improve accuracy and operational resilience.
The Challenge: Limitations of the 1.0 System
The predecessor system, primarily based on CatBoost models, faced several production hurdles:
- Physical Impossibility: Models occasionally predicted negative grid losses or returned "Not a Number" (NaN) values.
- Fragility: A lack of model diversity made the system vulnerable to specific model failures.
- Overfitting: The selection logic often defaulted to persistence models during shifts, failing to capture actual pattern changes.
Methodology: The 2.0 Architecture
To address these issues, the authors implemented four key architectural improvements:
Forced Positivity via Log-Space
By training models in log-space and exponentiating the output, the system mathematically guarantees non-negative predictions, aligning with physical reality.
Ensemble Diversity and Feature Engineering
The ensemble was expanded beyond CatBoost to include Random Forest, k-Nearest Neighbors, and linear models (Ridge, GLM). Furthermore, the system exploits the high autocorrelation inherent in grid data.
As shown in the autocorrelation plot above, losses at a specific hour are highly correlated with the same hour in previous days, justifying the use of lagged features.
"Superlearner" Stacking with NNLS
The system employs Non-Negative Least Squares (NNLS) stacking. This "Superlearner" assigns weights to base models that are non-negative and sum to one. This approach is inherently robust; if a base model fails due to missing sensor data, the NNLS algorithm dynamically re-allocates weights among the remaining active models.
Results and Discussion
The 2.0 system was evaluated using a real-world dataset from seven electrical grids in Norway, spanning 2017 to 2022.
Performance Gains
Upon deployment, the system achieved a 10.7% reduction in Mean Absolute Error (MAE), dropping from 5.05 MW to 4.51 MW. The ensemble's diversity also improved, with the standard deviation of predictions increasing from 1.55 MW to 2.08 MW, indicating a healthier "wisdom of the crowd."
Operational Robustness
A key strength of the 2.0 system is its ability to handle real-world data gaps.
The figure above illustrates the system's dynamic nature: as the number of available models fluctuates due to data lags or sensor failures, the stacker adjusts the "used" models to maintain a stable prediction.
Conclusion
The transition to a diverse, NNLS-weighted stacked ensemble has significantly enhanced the accuracy and reliability of distribution network loss forecasting. While the system effectively handles missing data and ensures physical consistency, future work will focus on improving the capture of "spiky" data—extreme, sudden fluctuations that the current ensemble tends to smooth over.
Summary updated