Mean Field Control of Thermostatically Controlled Loads as Piecewise Deterministic Markov Processes

Thomas Le Corre, Adrien Séguret, Ana BušićarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a scalable framework for coordinating thousands of Thermostatically Controlled Loads (TCLs), like water heaters, to help balance the power grid. By modeling these devices as Piecewise Deterministic Markov Processes (PDMPs) and applying Mean Field Control, the authors successfully manage aggregate electricity demand while ensuring individual user comfort through a clever "jump intensity" mechanism.

What Problem Does This Paper Address?

As we integrate more intermittent renewable energy (wind and solar) into the grid, we need ways to adjust electricity demand to match supply. Water heaters and air conditioners are perfect candidates for this "demand response" because they can shift their energy use in time without the user noticing.

However, controlling a population of, say, 100,000 water heaters presents two major hurdles:

  1. The Curse of Dimensionality: You cannot realistically solve an optimization problem with 100,000 individual variables in real-time.
  2. Hybrid Dynamics: A water heater is a "hybrid" system—its temperature changes continuously (physics), but its state is discrete (it’s either ON or OFF).

Existing methods often struggle to balance the complex physics of these devices with the need for a simple, scalable control strategy that doesn't violate "quality-of-service" (i.e., making sure the water is actually hot when you need it).

Methods

The authors combine two powerful mathematical frameworks:

  • Piecewise Deterministic Markov Processes (PDMPs): This models the water heater’s temperature as a smooth, predictable curve (deterministic) that is interrupted by random switches between ON and OFF states (stochastic jumps). This captures the "jittery" nature of real-world appliances.
  • Mean Field Control (MFC): Instead of tracking every single heater, MFC looks at a "representative agent" interacting with the average behavior of the entire crowd. This turns a massive multi-agent problem into a much simpler single-agent problem.

The Secret Sauce: Jump Intensities To keep temperatures within a safe "comfort range" (e.g., 50°C to 65°C), the authors introduced a specific jump intensity ($\hat{\alpha}$). Think of this as a "probabilistic nudge." As the temperature approaches the cold limit, the probability of the heater switching ON spikes dramatically. This ensures safety constraints are met without needing a rigid, computationally expensive "hard" boundary.

The Algorithm: They use a decentralized approach based on a Lagrangian dual formulation. A central planner calculates a "coordination signal" (like a virtual price), and each water heater uses a stochastic gradient descent (the Uzawa algorithm) to decide when to switch, based on its own local temperature and that global signal.

Datasets

The paper does not use a real-world historical dataset. Instead, it evaluates the method using a high-fidelity numerical simulation of Water Heaters (WHs).

  • The simulation models heat loss, heating power, and "drain events" (people taking showers).
  • The initial states (temperature and ON/OFF status) are randomized to reflect a diverse population.

Key Insights and Ideas

  • Soft Boundaries are Better: By using high jump intensities at the edges of the temperature range rather than hard "if-then" rules, the system remains mathematically "smooth" (differentiable). This allows standard optimization tools to work much more efficiently.
  • Desynchronization is a Feature: Because the control is probabilistic, not every heater switches at the exact same moment. This naturally "smears" the power demand, preventing the massive, grid-damaging spikes that occur when thousands of devices are commanded to turn on simultaneously.
  • Decentralization via "Price": The dual formulation shows that you don't need to know the temperature of every house. You just need to broadcast a single coordination signal, and the "physics" of the mean-field ensures the population reacts correctly.

Key Results

The authors tested their approach on two scenarios:

  1. Signal Tracking: The population was tasked with following a specific "smooth" consumption curve. The results showed that as the importance of tracking ($\kappa$) increased, the aggregate consumption of the heaters matched the target signal almost perfectly.
  2. Price Responsiveness: When given a fluctuating electricity price (peak/off-peak), the algorithm successfully pre-heated the water before the expensive period and "coasted" through the high-price window, all while keeping the water within the 50°C–65°C comfort zone.
  3. Smoothing the "Rebound": They demonstrated that by splitting the population into groups with slightly different price schedules, they could eliminate the "rebound effect"—the sudden surge in demand that usually happens the moment a high-price period ends.

Limitations and Future Work

  • The "Large N" Assumption: Mean Field theory assumes there are an infinite number of agents. While it works well for thousands, the approximation might break down for smaller microgrids with only a few dozen houses.
  • Offline vs. Online: The current framework assumes we know the future (like the next 24 hours of prices or water usage) in advance. The authors suggest moving toward an online setting where the algorithm updates its strategy as new information (like a sudden cold snap or a change in wind power) arrives.
  • Complex Constraints: Future work could incorporate "stronger" constraints, such as a hard cap on the total instantaneous power the entire fleet can draw from the grid.

Summary updated

Abstract

This paper presents a mean-field control approach for Piecewise Deterministic Markov Processes (PDMPs), specifically designed for controlling a large number of agents. By modeling the interactions of a large number of agents through an aggregate cost function, the proposed method mitigates the high dimensionality of the problem by focusing on a representative agent. The contribution of this work is the application of a PDMP-based mean-field control framework to the coordination of a large population of Thermostatically Controlled Loads (TCLs). Adapting this framework to TCLs requires incorporating a quality-of-service constraint ensuring that each agent's temperature remains within a specified comfort range. To achieve this, an additional jump intensity is introduced so that agents are very likely to switch between heating and cooling modes when they reach the boundaries of their temperature range. This extension to TCLs is demonstrated through Water Heaters (WHs) control, with a decentralized algorithm based on a dual formulation and stochastic gradient descent. The numerical results obtained illustrate this approach on two examples (signal tracking and taking into account energy price).
BibTeX
@misc{corre2025mean,
  author = {Thomas Le Corre and Adrien Séguret and Ana Bušić},
  title = {{Mean Field Control of Thermostatically Controlled Loads as Piecewise Deterministic Markov Processes}},
  year = {2025},
  eprint = {2511.01500},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.01500},
  howpublished = {arXiv},
}
Figure