Data augmentation for dealing with low sampling rates in NILM

Tai Le Quy, Sergej Zerr, Eirini Ntoutsi, Wolfgang NejdlarXiv 2021 · arXiv

TL;DR

This paper explores data augmentation as a solution for the "low sampling rate" problem in Non-Intrusive Load Monitoring (NILM). By using interpolation techniques to upsample low-frequency power data, the researchers demonstrate that appliance recognition and energy disaggregation accuracy can be significantly improved, even when the original data is sparse.

What Problem Does This Paper Address?

Non-Intrusive Load Monitoring (NILM), or energy disaggregation, is the process of breaking down a single household's total electricity consumption into the usage of individual appliances. For this to work well, algorithms typically need high-frequency data (e.g., 1Hz) to capture the unique "signatures" or power spikes that occur when a device like a refrigerator or a toaster turns on.

However, real-world data is often messy. Many public datasets and smart meter deployments provide data at much lower frequencies—sometimes only once every minute (1/60 Hz) or even every ten minutes (1/600 Hz). At these low rates, the sharp transitions that identify a device are lost. This paper investigates whether we can "reconstruct" that missing high-frequency information through data augmentation to help NILM algorithms perform better.

Methods

The authors tested four different interpolation methods to fill the gaps between low-frequency data points ($t_i$ and $t_{i+1}$):

  1. Stepwise Interpolation: The time gap is divided into $k$ parts, and the signal is increased or decreased in discrete linear steps.
  2. Cubic Spline Interpolation: A standard mathematical approach using piece-wise polynomials to create a smooth, continuous curve between points.
  3. Denton-Cholette Interpolation: A technique often used in economics for temporal disaggregation, aimed at preserving the "movement" or trend of a series.
  4. Device Interpolation: A novel approach where changes in individual device consumption are used as indicators. If a device's power changes beyond a specific threshold (5-10W), the aggregated signal is adjusted to the next timestamp's level; otherwise, it remains constant.

The visual differences between these methods are striking, as shown in the signal reconstruction below:

![Fig.

  1. A sample augmented data with different methods](/figures/art_c23ee75582664f528117d2330bf922cb)

Datasets

The study utilized two well-known NILM benchmarks, both originally recorded at 1Hz:

  • iAWE (2013): Data from a single house in Delhi, India, spanning 73 days. It is notable for having a high rate of missing values (31%).
  • ECO (2013): A comprehensive dataset from 6 households in Switzerland over 8 months, featuring very high data quality (only 0.8% missing values).

To test their methods, the authors manually downsampled these datasets to 1/60 Hz and 1/600 Hz before applying their augmentation techniques.

Key Insights and Ideas

  • Smoothness Can Be a Hindrance: While Cubic Spline interpolation creates a visually "smooth" curve, it often performed poorly. The authors noted that this smoothness can actually hide the sharp "switching events" that NILM algorithms rely on to identify appliances.
  • Algorithm Sensitivity: The impact of augmentation varies by algorithm. Weiss’s algorithm, which relies on both active and reactive power, suffered immensely from downsampling but saw the greatest gains from augmentation. Parson’s algorithm, designed for lower-frequency data, was more robust but still benefited from the reconstructed signals.
  • Device-Awareness Matters: The "Device Interpolation" method generally performed best because it respects the discrete, step-like nature of how appliances actually consume power, rather than assuming a continuous mathematical function.

Key Results

The primary metric for success was the F-score, which measures the accuracy of appliance identification.

  • ECO Dataset: For Weiss’s algorithm, the F-score dropped to 0.269 when the data was undersampled. Applying Device Interpolation brought the F-score back up to 0.467, a significant recovery toward the original 1Hz performance (0.604).
  • iAWE Dataset: On this smaller, noisier dataset, Stepwise Interpolation proved most effective for Weiss’s algorithm, increasing the F-score from 0.206 to 0.416.

The following boxplots illustrate how augmentation helps close the gap between "undersampled" data and the "original" high-frequency ground truth for the ECO dataset:

![Fig.

  1. Performance of NILM algorithms on ECO dataset](/figures/art_b5073df3063d47d68321c9da7a5ab344)

Limitations and Future Work

The authors acknowledge that the interpolation methods used here are relatively simple and "method-independent." While they provide a clear benefit, they don't perfectly recreate the original signal.

The next frontier for this research is "informed" augmentation. Instead of simple math, future models could use Deep Learning (such as GANs or LSTMs) to learn the specific usage patterns of a household and "predict" the high-frequency signal based on historical behavior and device profiles. This would move from mere interpolation to a more intelligent reconstruction of energy usage.

Summary updated

Abstract

Data have an important role in evaluating the performance of NILM algorithms. The best performance of NILM algorithms is achieved with high-quality evaluation data. However, many existing real-world data sets come with a low sampling quality, and often with gaps, lacking data for some recording periods. As a result, in such data, NILM algorithms can hardly recognize devices and estimate their power consumption properly. An important step towards improving the performance of these energy disaggregation methods is to improve the quality of the data sets. In this paper, we carry out experiments using several methods to increase the sampling rate of low sampling rate data. Our results show that augmentation of low-frequency data can support the considered NILM algorithms in estimating appliances' consumption with a higher F-score measurement.
BibTeX
@misc{quy2021data,
  author = {Tai Le Quy and Sergej Zerr and Eirini Ntoutsi and Wolfgang Nejdl},
  title = {{Data augmentation for dealing with low sampling rates in NILM}},
  year = {2021},
  eprint = {2104.02055},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.02055},
}
Figure