Abstract
Load curve data in power systems refers to users' electrical energy consumption data periodically collected with meters. It has become one of the most important assets for modern power systems. Many operational decisions are made based on the information discovered in the data. Load curve data, however, usually suffers from corruptions caused by various factors, such as data transmission errors or malfunctioning meters. To solve the problem, tremendous research efforts have been made on load curve data cleansing. Most existing approaches apply outlier detection methods from the supply side (i.e., electricity service providers), which may only have aggregated load data. In this paper, we propose to seek aid from the demand side (i.e., electricity service users). With the help of readily available knowledge on consumers' appliances, we present a new appliance-driven approach to load curve data cleansing. This approach utilizes data generation rules and a Sequential Local Optimization Algorithm (SLOA) to solve the Corrupted Data Identification Problem (CDIP). We evaluate the performance of SLOA with real-world trace data and synthetic data. The results indicate that, comparing to existing load data cleansing methods, such as B-spline smoothing, our approach has an overall better performance and can effectively identify consecutive corrupted data. Experimental results also demonstrate that our method is robust in various tests. Our method provides a highly feasible and reliable solution to an emerging industry application.
Introduction
Smart grids generate massive amounts of "load curve" data—periodic records of electricity consumption. This data is the backbone of modern utility operations, influencing everything from billing to infrastructure planning. However, this data is frequently corrupted by meter malfunctions, wireless transmission errors, or unscheduled maintenance.
The current industry standard for "cleansing" this data relies on appliance-oblivious methods. These are essentially statistical "black boxes" (like B-spline smoothing) that look for data points deviating from a historical average. The problem is that these methods cannot distinguish between a legitimate spike (like turning on a high-power microwave) and a data error. Furthermore, they struggle with "hidden" corrupted data—errors that fall within a normal-looking range but are physically impossible given the specific appliances in a home.
The Appliance-Driven Approach
The authors propose a model-driven approach that shifts the focus from the supply side (the utility) to the demand side (the user's home). Their core contribution is the Corrupted Data Identification Problem (CDIP) framework, built on two physical "Generation Rules":
- Power Bounds: At any moment, the total power must equal the sum of the power ranges of the appliances currently "on."
- Temporal Sparsity: It is highly unlikely for a household to flip dozens of switches simultaneously within a few seconds.
By leveraging physical knowledge of a household's appliances, the algorithm identifies data points that violate these rules. If a reading falls into a gap between valid aggregated power states, it is flagged as corrupted.
Methodology: SLOA and Virtual Appliances
To solve the CDIP, the researchers introduce the concept of a "Virtual Appliance." In their optimization model, the virtual appliance acts as a buffer that captures "residual" power. If the model is forced to "turn on" the virtual appliance to balance the equation, it is a clear signal that the data point is corrupted.
Because the global version of this problem is NP-complete, the authors developed the Sequential Local Optimization Algorithm (SLOA). SLOA uses a sliding window approach to process data in polynomial time, making it practical for real-world deployment. By applying $\ell_1$-norm minimization, the algorithm assumes that corruption is relatively rare (sparse), allowing it to ignore minor fluctuations and zero in on significant data integrity issues.
Evaluation and Results
The researchers evaluated SLOA using real-world trace data from the University of Victoria (UVic), monitoring 13 specific appliances, and synthetic data generated via Monte Carlo simulations.
Key findings include:
- Superior Precision: On real-world data, the appliance-driven approach achieved a precision of 85% to 95%, significantly outperforming B-spline smoothing (~50%).
- Handling Consecutive Errors: Statistical methods often fail when data is corrupted over a long period (e.g., a meter stuck at zero). SLOA correctly identifies these sequences by recognizing they violate the physical power bounds of the home.
- Self-Correction: The algorithm is robust; even if the initial state of the house is unknown, it "recovers" and begins identifying errors correctly after just a few time steps.
Conclusion
While highly effective, widespread adoption faces hurdles like privacy concerns regarding sharing granular appliance data. Future work will focus on "privacy-preserving" versions of this algorithm and data imputation to fill in identified gaps. Nevertheless, this appliance-driven approach provides a reliable solution for ensuring the accuracy of electricity consumption data.
Summary updated