Abstract
Load curve data in power systems refers to users' electrical energy consumption data periodically collected with meters. It has become one of the most important assets for modern power systems. Many operational decisions are made based on the information discovered in the data. Load curve data, however, usually suffers from corruptions caused by various factors, such as data transmission errors or malfunctioning meters. To solve the problem, tremendous research efforts have been made on load curve data cleansing. Most existing approaches apply outlier detection methods from the supply side (i.e., electricity service providers), which may only have aggregated load data. In this paper, we propose to seek aid from the demand side (i.e., electricity service users). With the help of readily available knowledge on consumers' appliances, we present a new appliance-driven approach to load curve data cleansing. This approach utilizes data generation rules and a Sequential Local Optimization Algorithm (SLOA) to solve the Corrupted Data Identification Problem (CDIP). We evaluate the performance of SLOA with real-world trace data and synthetic data. The results indicate that, comparing to existing load data cleansing methods, such as B-spline smoothing, our approach has an overall better performance and can effectively identify consecutive corrupted data. Experimental results also demonstrate that our method is robust in various tests. Our method provides a highly feasible and reliable solution to an emerging industry application.
1. Introduction
Modern smart grids rely heavily on "load curve data"—periodic recordings of electricity consumption—for billing, grid operations, and energy generation planning. However, this data is frequently corrupted by transmission errors, malfunctioning meters, and wireless packet loss. Traditional "appliance-oblivious" methods, such as B-spline smoothing or regression, primarily target statistical outliers. These methods often fail to detect "hidden" corrupted data: values that appear statistically normal but are physically impossible given the specific appliances in a household.
2. Methodology
The proposed appliance-driven approach formulates the Corrupted Data Identification Problem (CDIP) by incorporating known power ranges (minimum and maximum wattage) of a building's appliances.
2.1 The Virtual Appliance
To detect errors, a mathematical "virtual appliance" is introduced. If the sum of real appliance power consumption cannot match the recorded load within physical constraints, the virtual appliance "activates" to bridge the gap. By minimizing the $l_1$-norm of this virtual appliance's power, the model identifies the sparsest and most likely instances of data corruption.
2.2 Sequential Local Optimization Algorithm (SLOA)
Since solving the CDIP globally is NP-complete, the authors developed the Sequential Local Optimization Algorithm (SLOA). SLOA partitions the problem into small, overlapping time windows. An Overlap Index ($O$) heuristic is used to determine optimal window sizes based on appliance power range overlaps, ensuring computational efficiency without compromising accuracy.
3. Experimental Evaluation
The methodology was validated using two distinct datasets:
- Real-World Trace Data: Collected over two months from a laboratory at the University of Victoria (UVic), featuring 13 appliances sampled every 10 seconds via ZigBee sensors.
- Synthetic Data: Generated through Monte Carlo simulations to test the algorithm's scalability with 50 appliances and various simulated data corruption patterns.
4. Results and Discussion
The SLOA method demonstrates significant improvements over traditional statistical smoothing by prioritizing physical ground truths over statistical patterns.
- Precision: SLOA achieved a precision of up to 95.83% on real-world data, vastly outperforming the ~50% precision of B-spline smoothing by minimizing false alarms.
- Consecutive Errors: Unlike traditional methods that attempt to "fit" long streaks of corrupted data (e.g., a meter stuck at zero), SLOA correctly identifies these as errors.
- Robustness: The algorithm is resilient to "cold starts," successfully recovering and identifying errors even without initial knowledge of appliance states.
5. Conclusion
This appliance-driven approach shifts the focus from statistical patterns to physical constraints. While privacy concerns regarding appliance-level data sharing remain a hurdle, future work will explore privacy-preserving versions of this cleansing method. By leveraging domain knowledge of the demand side, SLOA provides a reliable solution for maintaining high-quality load curve data in modern power systems.
Summary updated