Online Learning and Distributed Control for Residential Demand Response

Xin Chen, Yingying Li, Jun Shimada, Na Li

TL;DR

This paper introduces a novel online learning and distributed control framework for managing residential air conditioner (AC) loads during demand response (DR) events. By combining Gaussian Processes for thermal modeling and logistic regression with Thompson Sampling for learning customer opt-out behavior, the system efficiently optimizes energy consumption while respecting customer privacy and adapting to real-time conditions.

What Problem Does This Paper Address?

The increasing integration of renewable energy sources and the growing demand for electricity, especially during peak hours, create significant challenges for grid stability. Demand Response (DR) programs aim to mitigate these issues by incentivizing consumers to reduce their electricity usage during critical periods. A major hurdle in residential DR is the uncertainty surrounding customer behavior, particularly their tendency to "opt-out" of automated load control, which can be influenced by various environmental factors and individual preferences. Existing DR control methods often oversimplify this uncertainty, leading to suboptimal performance and potentially high opt-out rates. This paper tackles the problem of designing real-time, adaptive DR control strategies that can accurately model and predict customer opt-out behavior, thereby maximizing the effectiveness of DR programs.

Methods

The core of this paper lies in its innovative approach to modeling and control. The authors propose a multi-period stochastic optimization framework that integrates two key components:

  • Thermal Dynamics Modeling: Gaussian Processes (GP) are employed to build a non-parametric model of indoor thermal dynamics. This allows for flexible and accurate prediction of how room temperature changes based on AC usage and external factors, without making strong assumptions about the underlying physical model.
  • Customer Opt-Out Behavior Modeling: Logistic Regression is used to model the probability of a customer opting out of DR control. This model incorporates various environmental factors (e.g., indoor temperature, incentive credits, outdoor temperature, electricity price) and individual preferences.
  • Online Learning and Control: The Thompson Sampling (TS) framework is leveraged to create a distributed online DR control algorithm. TS is a Bayesian approach that balances exploration (learning about customer behavior) and exploitation (making optimal control decisions). This allows the system to continuously learn and adapt to customer responses over time.
  • Distributed Implementation: The algorithm is designed for distributed implementation, meaning it can be embedded directly into local household devices (like smart plugs or AC units) or smartphone apps. This approach preserves customer privacy by avoiding the need to centralize sensitive data and reduces communication overhead.

The paper formulates two system-level optimization models: one for minimizing total AC load and another for tracking a regulated power trajectory. These models are then solved using distributed methods, with the TS algorithm driving the online learning and control decisions.

Datasets

The paper utilizes real customer metering data from ThinkEco Inc. for training and testing the thermal dynamics models. This data includes indoor temperature and AC power consumption. Outdoor temperature data was sourced from an open-access meteorological database. The authors do not introduce new benchmark datasets but rather use existing data to evaluate their proposed methods.

Key Insights and Ideas

  • The Power of Adaptive Learning: The most significant insight is the effectiveness of online learning, specifically Thompson Sampling, in handling the inherent uncertainty of customer opt-out behavior. By continuously learning and adapting, the system can achieve better DR outcomes than static or simplified models.
  • Privacy-Preserving Distributed Control: The distributed nature of the algorithm is crucial. It not only reduces communication burdens but also fundamentally protects customer privacy by keeping data local, a critical factor for widespread adoption of DR technologies.
  • GP for Flexible Thermal Modeling: Using Gaussian Processes for thermal dynamics modeling offers a significant advantage over traditional linear models, as it can capture complex, non-linear relationships and adapt to diverse building characteristics without requiring explicit physical model derivation.
  • Balancing Comfort and Savings: The proposed control strategy aims to maintain customer comfort (by keeping opt-out probabilities low) while still achieving substantial energy savings, demonstrating a practical trade-off that is essential for real-world DR programs.

Key Results

  • Improved Prediction Accuracy: The Gaussian Process model for indoor thermal dynamics demonstrated better prediction accuracy compared to a linear model, achieving lower average prediction errors for both short-term (15 minutes) and medium-term (45 minutes) ahead predictions.
  • Efficient Learning and Convergence: Numerical simulations showed that the proposed distributed online DR control algorithm converges to optimal AC control schemes within tens of iterations. The per-event regret decreased dramatically over time, indicating efficient learning of customer behavior.
  • Sublinear Cumulative Regret: The cumulative regret was shown to be sublinear, a key indicator of successful online learning algorithms, meaning the algorithm eventually learns customer behavior well enough to achieve near-optimal performance.
  • Higher Load Reduction: For the SOC-1 model (minimizing total demand), the proposed algorithm achieved a higher average load energy reduction (1.13 kWh per DR event) compared to baseline schemes that simply raised AC setpoints (0.6 kWh and 0.82 kWh).
  • Effective Tracking: For the SOC-2 model (tracking a power trajectory), simulations demonstrated that the algorithm effectively tracks the target power profile while maintaining high customer stay-in probabilities.

Limitations and Future Work

  • Environmental Factor Selection: The paper acknowledges that the selection and definition of environmental factors influencing opt-out behavior are complex and can be tricky in practice. Future work could focus on identifying more significant and effective environmental factors through real user data analysis.
  • Practical DR Experiments: The authors suggest conducting practical DR experiments using the proposed algorithm and analyzing its real-world performance as a crucial next step.
  • Computational Complexity of GP: While GP offers flexibility, its computational complexity scales cubically with the number of observations, which could be a limitation for very large datasets or real-time applications requiring extremely fast updates.
  • Generalizability: While the framework is presented as general, further validation across a wider range of customer demographics, appliance types, and DR program designs would be beneficial.

Summary updated

Abstract

This paper studies the automated control method for regulating air conditioner (AC) loads in incentive-based residential demand response (DR). The critical challenge is that the customer responses to load adjustment are uncertain and unknown in practice. In this paper, we formulate the AC control problem in a DR event as a multi-period stochastic optimization that integrates the indoor thermal dynamics and customer opt-out status transition. Specifically, machine learning techniques including Gaussian process and logistic regression are employed to learn the unknown thermal dynamics model and customer opt-out behavior model, respectively. We consider two typical DR objectives for AC load control: 1) minimizing the total demand, 2) closely tracking a regulated power trajectory. Based on the Thompson sampling framework, we propose an online DR control algorithm to learn customer behaviors and make real-time AC control schemes. This algorithm considers the influence of various environmental factors on customer behaviors and is implemented in a distributed fashion to preserve the privacy of customers. Numerical simulations demonstrate the control optimality and learning efficiency of the proposed algorithm.
BibTeX
@article{chen2021online,
  author = {Xin Chen and Yingying Li and Jun Shimada and Na Li},
  title = {{Online Learning and Distributed Control for Residential Demand Response}},
  journal = {IEEE Trans. Smart Grid},
  year = {2021},
  url = {https://openreview.net/forum?id=Joiq7YZhkTg},
}
Figure