Is My Electricity Bill Accurate? An Appliance-Driven Approach to Corrupted Load Curve Data Identification

Guoming Tang, Kui Wu, Jian Pei, Jiuyang Tang, Jingsheng Lei

Abstract

Load curve data in power systems refers to users' electrical energy consumption data periodically collected with meters. It has become one of the most important assets for modern power systems. Many operational decisions are made based on the information discovered in the data. Load curve data, however, usually suffers from corruptions caused by various factors, such as data transmission errors or malfunctioning meters. To solve the problem, tremendous research efforts have been made on load curve data cleansing. Most existing approaches apply outlier detection methods from the supply side (i.e., electricity service providers), which may only have aggregated load data. In this paper, we propose to seek aid from the demand side (i.e., electricity service users). With the help of readily available knowledge on consumers' appliances, we present a new appliance-driven approach to load curve data cleansing. This approach utilizes data generation rules and a Sequential Local Optimization Algorithm (SLOA) to solve the Corrupted Data Identification Problem (CDIP). We evaluate the performance of SLOA with real-world trace data and synthetic data. The results indicate that, comparing to existing load data cleansing methods, such as B-spline smoothing, our approach has an overall better performance and can effectively identify consecutive corrupted data. Experimental results also demonstrate that our method is robust in various tests. Our method provides a highly feasible and reliable solution to an emerging industry application.

Introduction

As smart grids roll out globally, "load curve data"—the record of electrical consumption over time—has become a vital asset for utilities. It dictates billing, helps balance the grid, and informs infrastructure investment. However, this data is often "dirty." Transmission errors, malfunctioning meters, or packet losses in wireless networks lead to corrupted data points.

Traditional "supply-side" cleansing methods typically look for statistical outliers—data points that appear anomalous compared to a neighborhood average. However, these approaches are "appliance-oblivious." They may flag a perfectly normal spike (like a high-power dryer) as an error, or worse, miss "hidden" corruption that falls within a statistically normal range but is physically impossible for a specific home's appliance configuration to produce.

Methodology: The Appliance-Driven Approach

This research proposes shifting the perspective to the demand side, leveraging the physical limits of household appliances to validate data. The core logic relies on the fact that electricity consumption is governed by the physical properties of hardware. If a household only contains low-power devices, a massive spike is a physical impossibility, not just a statistical outlier.

Generation Rules and Aggregated Ranges

The approach defines valid data through two primary rules. First, any reading must be explainable by a combination of the home's appliances being in an 'on' or 'off' state within their rated power bounds. As shown in Figure 1, different combinations of appliances ($A_1, A_2, A_3$) create specific valid aggregated power ranges. Data points falling into the "gaps" between these valid combinations are identified as hidden corrupted data.

Conceptual framework showing how aggregated power ranges help identify hidden corrupted data.

Second, the principle of "temporal sparsity" is applied: users typically do not switch dozens of appliances simultaneously, meaning the state of the household should not change drastically within short (e.g., 10-second) intervals.

The Virtual Appliance and SLOA

To identify corruption mathematically, the researchers introduced a "virtual appliance." If real appliances cannot explain a data point, this virtual appliance "turns on" to bridge the gap. By setting up an optimization problem to minimize the use of this virtual appliance (using L1-norm minimization), the system isolates the most likely corrupted points. To make this computationally feasible for large datasets, the Sequential Local Optimization Algorithm (SLOA) was developed, using a sliding time window to solve the problem in manageable chunks.

Experimental Evaluation

The approach was validated using two distinct sources:

  1. Real-World Trace Data: Monitoring 13 appliances (laptops, printers, refrigerators, etc.) in a laboratory environment for two months, with sampling every 10 seconds.
  2. Synthetic Data: Monte Carlo simulations generating data for a hypothetical house with 50 appliances to test scalability and robustness.

Key Results and Discussion

The appliance-driven approach demonstrated superior precision compared to standard statistical methods like B-spline smoothing. Because the model understands the specific "steps" of power consumption, it is far less likely to generate false positives for valid high-power usage.

Detection results showing the estimated power bounds and the calculated corrupted degree for identifying anomalies.

As illustrated in Figure 6, the algorithm establishes "Estimated Bounds" based on the appliance model. When the recorded load curve (red) violates these physical bounds, the "Corrupted Degree" metric spikes, allowing for precise identification. This method proved particularly effective at identifying consecutive corrupted data points, which often confuse regression-based models that try to "fit" a curve to the bad data.

Conclusion

By utilizing physical appliance constraints rather than just statistical trends, the SLOA provides a robust and reliable solution for load curve data cleansing. While future work must address privacy concerns regarding appliance-level data and the need for automated data imputation, this demand-side approach offers a significant advancement in ensuring the accuracy of modern power system data.

Summary updated

Abstract

Load curve data in power systems refers to users' electrical energy consumption data periodically collected with meters. It has become one of the most important assets for modern power systems. Many operational decisions are made based on the information discovered in the data. Load curve data, however, usually suffers from corruptions caused by various factors, such as data transmission errors or malfunctioning meters. To solve the problem, tremendous research efforts have been made on load curve data cleansing. Most existing approaches apply outlier detection methods from the supply side (i.e., electricity service providers), which may only have aggregated load data. In this paper, we propose to seek aid from the demand side (i.e., electricity service users). With the help of readily available knowledge on consumers' appliances, we present a new appliance-driven approach to load curve data cleansing. This approach utilizes data generation rules and a Sequential Local Optimization Algorithm (SLOA) to solve the Corrupted Data Identification Problem (CDIP). We evaluate the performance of SLOA with real-world trace data and synthetic data. The results indicate that, comparing to existing load data cleansing methods, such as B-spline smoothing, our approach has an overall better performance and can effectively identify consecutive corrupted data. Experimental results also demonstrate that our method is robust in various tests. Our method provides a highly feasible and reliable solution to an emerging industry application.
BibTeX
@misc{tang2013is,
  author = {Guoming Tang and Kui Wu and Jian Pei and Jiuyang Tang and Jingsheng Lei},
  title = {{Is My Electricity Bill Accurate? An Appliance-Driven Approach to Corrupted Load Curve Data Identification}},
  year = {2013},
  eprint = {1307.7757},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1307.7757},
}
Figure