Outlier Detection in Energy Disaggregation Using Subspace Learning and Gaussian Mixture Model

Xiuming Tang, Rong-xiang Yuan, Jun ChenInternational Journal of Control and Automation 2015 · DOI

Abstract

Special Complex non-Gaussian processes may have dynamic operation scenario shifts so that the traditional Outlier detection approaches become ill-suited. This paper proposes a new outlier detection approach based on using subspace learning and Gaussian mixture model (GMM) in energy disaggregation. Locality preserving projections (LPP) of subspace learning can optimally preserve the neighborhood structure, reveal the intrinsic manifold structure of the data and keep outliers far away from the normal sample compared with the principal component analysis (PCA). The results show the proposed approach can significantly improve performance of outlier detection in energy disaggregation, increase the fraction true-positive from 93.8% to 97%, and decrease the fraction false-positive from 35.48% to 25.8%.

Introduction

Energy disaggregation, commonly known as Non-Intrusive Load Monitoring (NILM), aims to decompose a single aggregate power signal into the consumption patterns of individual appliances. High-quality, clean data is a prerequisite for accurate disaggregation. However, real-world datasets, such as the Reference Energy Disaggregation Data Set (REDD), are frequently contaminated by outliers resulting from sensor malfunctions or equipment failures.

Traditional outlier detection methods often rely on Principal Component Analysis (PCA) or single-Gaussian models. These approaches are often inadequate for energy data, which is inherently multi-state (e.g., varying power signatures for different appliance cycles) and non-Gaussian. This paper addresses these challenges by proposing a robust framework that leverages the local manifold structure of energy data.

Methodology

The proposed framework consists of a two-stage pipeline designed to isolate anomalies while preserving complex operational data:

  1. Subspace Learning via LPP: Unlike PCA, which focuses on global variance, Locality Preserving Projections (LPP) builds an adjacency graph to maintain the neighborhood structure of the data. This reveals the intrinsic manifold of appliance operations, ensuring that normal samples remain tightly clustered while outliers are pushed further away in the lower-dimensional space.
  2. Modeling via GMM: A Gaussian Mixture Model (GMM) is applied to the LPP-projected subspace. GMMs are particularly suited for this task as they can simultaneously model multiple operational modes (states) of appliances.
  3. Detection Metric: The system utilizes Negative Log-Likelihood Probability (NLLP). Data points exhibiting a significantly high NLLP are flagged as outliers, as they are unlikely to have been generated by the trained GMM of normal behavior.
Comparison of subspace projections: (a) PCA-based projection showing overlapping clusters vs (b) LPP-based projection revealing distinct manifold structures.

Experimental Results

The methodology was validated using House #5 from the REDD dataset, comprising 21 variables representing various household circuits. The training set included 1,000 normal samples, while the test set contained 530 samples with a mix of normal and outlier data.

The LPP-GMM approach demonstrated significant performance gains over the traditional PCA-GMM baseline:

  • True Positive (TP) Rate: Improved from 93.8% to 97.0%.
  • False Positive (FP) Rate: Reduced from 35.48% to 25.8%.

Qualitative analysis of the NLLP curves confirms that the LPP-based model achieves much clearer separation between normal baselines and anomalous spikes compared to PCA-based methods.

Outlier detection performance using the proposed LPP-GMM method, showing clear separation of anomalous samples via negative log-likelihood probability.

Conclusion

By prioritizing local manifold structures over global variance, the LPP-GMM framework significantly enhances outlier detection in energy disaggregation. This approach reduces false alarms and improves the identification of sensor errors in complex, multi-state environments. Future work will focus on integrating this robust filtering stage into real-time NILM systems to provide more accurate energy-saving recommendations for consumers.

Summary updated

Abstract

Special Complex non-Gaussian processes may have dynamic operation scenario shifts so that the traditional Outlier detection approaches become ill-suited. This paper proposes a new outlier detection approach based on using subspace learning and Gaussian mixture model(GMM) in energy disaggregation. Locality preserving projections(LPP) of subspace learning can optimally preserve the neighborhood structure, reveal the intrinsic manifold structure of the data and keep outliers far away from the normal sample compared with the principal component analysis (PCA). The results show proposed approach can significantly improve performance of outlier detection in energy disaggregation, increase the fraction true-positive from 93.8% to 97%, decrease the fraction false-positive from 35.48% to 25.8%.
BibTeX
@article{tang2015outlier,
  author = {Xiuming Tang and Rong-xiang Yuan and Jun Chen},
  title = {{Outlier Detection in Energy Disaggregation Using Subspace Learning and Gaussian Mixture Model}},
  journal = {International Journal of Control and Automation},
  year = {2015},
  doi = {10.14257/ijca.2015.8.8.17},
}
Figure