Shape-Based Approach to Household Electric Load Curve Clustering and Prediction

Thanchanok Teeraratkul, Daniel C. O'Neill, Sanjay LallIEEE Trans. Smart Grid 2018 · OpenReview

TL;DR

This paper introduces a novel shape-based approach using Dynamic Time Warping (DTW) for clustering household electricity consumption patterns. This method significantly reduces the number of consumer groups and improves prediction accuracy compared to traditional methods, enabling more effective demand response strategies.

What Problem Does This Paper Address?

The paper tackles the challenge of understanding and predicting household electricity consumption for Demand Response (DR) programs. Current DR efforts struggle because traditional clustering methods create too many granular groups, making it difficult to identify stable consumer segments and accurately predict individual energy usage. This complexity hinders the ability to effectively incentivize consumers to adjust their consumption patterns. The core problem is the inherent variability in daily energy usage, even for consistent behavioral patterns, which traditional, time-aligned clustering methods fail to capture effectively.

Methods

The paper's core innovation lies in its shape-based clustering approach using Dynamic Time Warping (DTW). Unlike traditional methods that rely on direct point-to-point comparisons (like Euclidean distance), DTW allows for non-linear stretching and compressing of time series. This means it can find similarities in the shape of electricity consumption curves even if the exact timing of device usage varies.

Key methodological components include:

  • DTW for Clustering: DTW is used as the dissimilarity measure to group load curves with similar shapes, regardless of minor temporal shifts. This is crucial because household device usage can vary by an hour or so daily.
  • Markov Models for Prediction: Once clusters are formed, a Markov chain model is employed. This model uses the sequence of cluster assignments (representing load shapes) from previous days to predict the most probable cluster (and thus load shape) for the next day.
  • Power Level Decomposition (PLD): The authors extend their work to estimate the energy usage of individual appliances within a household. This involves modeling the load curve as a matrix representing appliance usage over time and then predicting this matrix.

Datasets

The primary dataset used is a collection of 23,254 24-hour electricity load curves from 1057 households, provided by Opower Corporation. These load curves were sampled hourly and pre-processed by smoothing with cubic splines and normalizing to sum to one. The data spans from July 19, 2012, to August 9, 2012.

Key Insights and Ideas

  • Shape Matters More Than Exact Timing: The most significant insight is that for understanding household energy consumption patterns, the shape of the load curve is more informative than the precise timing of energy usage. DTW effectively captures this by allowing for temporal flexibility.
  • DTW Reduces Cluster Granularity: By focusing on shape, DTW-based clustering drastically reduces the number of representative consumer groups compared to traditional methods like K-means and Gaussian Expectation-Maximization (E&M). This simplification is key to making DR programs more manageable.
  • Predicting Load Shapes with Markov Chains: The paper demonstrates that a Markov model, built upon the DTW-derived cluster assignments, can effectively predict the next day's load shape. This is a powerful way to leverage historical patterns.
  • Probabilistic Bounds for Fine-Grained Prediction: The extension to Power Level Decomposition (PLD) provides probabilistic bounds on prediction error, offering a theoretical understanding of the reliability of predicting individual appliance usage.

Key Results

  • 50% Reduction in Clusters: The DTW-based clustering resulted in a 50% reduction in the number of representative household groups compared to traditional K-means and E&M algorithms.
  • Improved Prediction Accuracy: The DTW-Markov prediction method achieved a lower normalized DTW error (DTWE) of 0.182 on the test set, outperforming other benchmark models (Tao's vanilla, semi-parametric additive, SVR, ANN) by 15-31%.
  • Lower Household Variability: Average entropy from DTW clustering was 0.5, compared to 0.7 for K-means and 0.64 for E&M, indicating more consistent energy consumption patterns within DTW-derived clusters.
  • Tight Probabilistic Bounds: The paper establishes tight probabilistic bounds for the PLD prediction error, particularly when the power vector is appropriately chosen.

Limitations and Future Work

  • Parameter Selection: The selection of parameters for the DTW-Markov model (number of clusters K and periods np) was done through trial-and-error, suggesting potential for more automated or data-driven parameter optimization.
  • PLD Regularization: For sparse PLD estimation, the paper notes that when the input vector is non-zero, a closed-form solution is not available, requiring more complex optimization.
  • Future Work: The authors suggest exploring and comparing the performance of PLD estimates from different regularization techniques (e.g., L1 minimization, convex optimization) and investigating more advanced methods for parameter selection. They also aim to explore the performance of PLD estimates from different kinds of regularization or convex optimization.

Summary updated

Abstract

Household consumer demand response (DR) is an important research and industry problem, which seeks to categorize, predict, and modify consumer’s energy consumption. Unfortunately, traditional clustering methods have resulted in many hundreds of clusters, with a given consumer often associated with several clusters, making it difficult to classify consumers into stable representative groups and to predict individual energy consumption patterns. In this paper, we present a shape-based approach that better classifies and predicts consumer energy consumption behavior at the household level. The method is based on the dynamic time warping (DTW). DTW seeks an optimal alignment between energy consumption patterns reflecting the effect of hidden patterns of regular consumer behavior. Using real consumer 24-hour load curves from Opower Corporation, our method results in a 50% reduction in the number of representative groups and an improvement in prediction accuracy measured under DTW distance. We extend the approach to estimate which electrical devices will be used and in which hours.
BibTeX
@article{teeraratkul2018shape,
  author = {Thanchanok Teeraratkul and Daniel C. O'Neill and Sanjay Lall},
  title = {{Shape-Based Approach to Household Electric Load Curve Clustering and Prediction}},
  journal = {IEEE Trans. Smart Grid},
  year = {2018},
  url = {https://openreview.net/forum?id=h4crGpVfpR},
}
Figure