A veracity preserving model for synthesizing scalable electricity load profiles

Yunyou Huang, Jianfeng Zhan, Chunjie Luo, Lei Wang, Nana Wang, Daoyi Zheng, Fanda Fan, Rui RenarXiv 2018 · arXiv

TL;DR

Researchers have developed a Hierarchical Multi-Matrices Markov Chain (HMMC) model to generate realistic, synthetic electricity load profiles. By modeling consumption patterns across three nested time scales—daily, weekly, and yearly—the HMMC approach overcomes the limitations of traditional models, providing high-fidelity data for grid planning while bypassing the privacy hurdles associated with real-world datasets.

What Problem Does This Paper Address?

Designing and deploying modern electric systems requires massive amounts of granular electricity load data. However, researchers face a significant "data drought." Real-world load profiles are notoriously difficult to acquire because they contain sensitive information about user behavior, leading to strict privacy protections and commercial confidentiality.

While data synthesis is a promising solution, existing methods often fall short. Simple generative models, like first-order Markov Chains, struggle to preserve the "veracity" of the data. They often fail to capture how consumption habits change depending on the time of day or season, and they frequently introduce unrealistic artifacts—such as merging two distinct daily peaks into a single, nonsensical profile.

Methods

The authors propose a bottom-up, hierarchical approach to capture the complexity of human behavior across different time horizons.

  1. Pattern Extraction: The model uses a modified Adaptive K-Means algorithm to cluster raw data into a "Pattern Dictionary." This dictionary categorizes typical daily, weekly, and yearly consumption behaviors.
  2. Multi-Matrices Markov Chain (MMC): Unlike a standard Markov Chain that uses a single transition matrix, the MMC uses a unique matrix for every time step in a sequence. This ensures that a specific power state in the morning transitions differently than the same state in the evening, reflecting actual human routines.
  3. Hierarchical Modeling: The HMMC links these scales together. A yearly pattern is modeled as a sequence of weekly patterns, which are in turn sequences of daily patterns. This hierarchy (shown in Figure 1) significantly reduces the dimensionality of the problem, making it computationally feasible to model a full year of 15-minute interval data.
  4. User Modeling: To make the data "scalable," the authors use multinomial logistic regression to link demographic or industrial attributes (like building type or industry code) to specific yearly consumption patterns.
Figure 1: The summary of our approach.

Datasets

The model was trained and validated using two primary sources:

  • Pecan Street Inc. Dataport: A public dataset containing data from over 700 residential users.
  • Confidential Industrial Dataset: A massive dataset involving over 80,000 users from non-resident sectors, including education, finance, and manufacturing.

The authors have released two trained models based on these datasets to allow other researchers to synthesize their own scalable profiles.

Key Insights and Ideas

  • Timing Matters: The core "aha moment" is that electricity consumption is highly dependent on the position in a sequence. A standard Markov Chain treats a power spike at 3 AM the same as one at 3 PM. By using Multi-Matrices (MMC), the authors ensure the model respects the "timing of daily life."
  • Avoiding the "Average" Trap: Traditional models often synthesize data that looks like an average of several users but represents no one in particular. As seen in Figure 4, if two profiles intersect, a simple model might jump between them, creating a "Frankenstein" profile with unrealistic peaks. The HMMC's hierarchical structure prevents this by ensuring consistency within a chosen pattern.
Figure 4: Three electricity load profiles in the same pattern: A, B, and C. A is red, B is blue, and C is green.

Key Results

The HMMC model significantly outperformed the classical Markov Chain across several statistical metrics:

  • Variance Preservation: The HMMC was much better at maintaining the variance of load profiles within a pattern ($\sigma_{pro}$), meaning the synthetic data looked as "diverse" as the real data.
  • Total Consumption Accuracy: Metrics for total yearly consumption ($\mu_{total}$ and $\sigma_{total}$) were nearly identical to the raw data. For example, in one test case, the raw data's $\sigma_{total}$ was 5622.8, while the HMMC's was 5523.9; the classic Markov model, by contrast, collapsed to a mere 934.0, failing to capture the spread of the data.
  • Visual Veracity: Qualitative comparisons showed that HMMC-generated daily and weekly patterns were virtually indistinguishable from real-world observations.

Limitations and Future Work

While the HMMC model excels at preserving statistical veracity, the paper notes that the quality of the output is heavily dependent on the initial clustering phase. If the Adaptive K-Means threshold ($\gamma$) is set too high, the patterns become too broad and lose detail; if set too low, the model requires an impractical number of clusters.

Future work might involve exploring how these synthetic profiles perform when used in specific grid simulation stress tests or integrating more complex environmental variables (like weather data) directly into the transition matrices.

Summary updated

Abstract

Electricity users are the major players of the electric systems, and electricity consumption is growing at an extraordinary rate. The research on electricity consumption behaviors is becoming increasingly important to design and deployment of the electric systems. Unfortunately, electricity load profiles are difficult to acquire. Data synthesis is one of the best approaches to solving the lack of data, and the key is the model that preserves the real electricity consumption behaviors. In this paper, we propose a hierarchical multi-matrices Markov Chain (HMMC) model to synthesize scalable electricity load profiles that preserve the real consumption behavior on three time scales: per day, per week, and per year. To promote the research on the electricity consumption behavior, we use the HMMC approach to model two distinctive raw electricity load profiles. One is collected from the resident sector, and the other is collected from the non-resident sectors, including different industries such as education, finance, and manufacturing. The experiments show our model performs much better than the classical Markov Chain model. We publish two trained models online, and researchers can directly use these trained models to synthesize scalable electricity load profiles for further researches.
BibTeX
@misc{huang2018veracity,
  author = {Yunyou Huang and Jianfeng Zhan and Chunjie Luo and Lei Wang and Nana Wang and Daoyi Zheng and Fanda Fan and Rui Ren},
  title = {{A veracity preserving model for synthesizing scalable electricity load profiles}},
  year = {2018},
  eprint = {1802.03500},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.03500},
}

Loading related content...

Figure