Introduction
In the Nordic electricity market (Nord Pool), utility companies face the daily challenge of "nominating" expected electricity losses in their distribution networks. These nominations, due by noon for every hour of the following day, are critical for grid stability and financial efficiency. Grid loss—the discrepancy between power produced and power sold—stems from physical resistance (technical loss) and factors like meter errors or theft (non-technical loss). Inaccurate predictions force utilities into the volatile imbalance market, where prices are often punitive.
The Challenge of Grid Loss Prediction
Historically, grid loss forecasting relied on manual heuristics and simple numerical formulas. However, these methods struggle with the inherent complexities of real-world industrial data. Grid loss exhibits significant seasonal and weekly patterns, reflecting changes in consumer behavior and environmental conditions.
Predicting these losses is complicated by several factors:
- Data Latency: Definitive loss measurements are typically delayed by 5-6 days.
- Data Quality: Sensor failures often lead to missing or erroneous data points.
- Operational Stochasticity: Daily changes in grid routing based on operational needs make the signal difficult to model with static formulas.
Methodology: A Two-Step Pipeline
To automate and improve this process, researchers at TrønderEnergi and the Norwegian Open AI Lab developed a machine learning system based on CatBoost, a high-performance gradient boosting library. The system employs a two-step forecasting architecture to handle the physical relationship between power load and line losses.
The pipeline first predicts the total electricity consumption (load) for the target day using historical load, calendar features, and weather forecasts. This predicted load then serves as a primary feature for the subsequent grid loss model. This approach leverages the physical correlation where technical losses are proportional to the square of the current.
Robustness through Redundancy
To handle missing data in production, the system maintains a library of 24 different models (8 models for each of the 3 sub-grids). These models are trained on varying subsets of features. Each day, the system evaluates available data (e.g., checking if weather APIs or historical sensors are online) and selects the best-performing model from the subset of models whose required features are present. This ensures the system gracefully degrades rather than failing when data is incomplete.
Data and Implementation
The study utilized 19 months of historical data (December 2017 to June 2019) from TrønderEnergi Nett’s network. The dataset included hourly grid loss, hourly load, calendar data, and meteorological forecasts. A key finding was that a sliding training window of the most recent 180 days outperformed using the entire historical dataset, likely due to "concept drift" caused by physical changes in the grid or evolving consumer behavior. Additionally, predicting losses for individual sub-grids and then summing them was found to be 9% more accurate than a single global model.
Key Results and Impact
The automated system significantly outperformed previous manual methods during a 90-day evaluation period:
- Error Reduction: Mean Absolute Error (MAE) was reduced by 41%, dropping from 3.68 MW to 2.17 MW per hour.
- Financial Savings: This improvement translates to approximately $15,000 USD in annual savings on imbalance costs for a single utility.
- Operational Efficiency: Automation saves roughly 100 hours of manual labor per year, reducing human error and allowing engineers to focus on higher-value tasks.
Limitations and Future Work
While successful, the model has limitations. It is currently "naive" to sudden physical step changes in the grid, such as the connection of a new high-energy industrial site, taking roughly 10-12 days to adapt. Furthermore, while theft is low in Norway, the model's performance in regions with high non-technical losses remains an open question. Future iterations could integrate granular grid topology data to better anticipate the effects of network reconfigurations and physical changes in the distribution infrastructure.
Summary updated