A Distributed Dynamic Programming-Based Solution for Load Management in Smart Grids

Wei Zhang, Yinliang Xu, Sisi Li, MengChu Zhou, Wenxin Liu, Ying Xu

Abstract

Load management (LM) is being recognized as an important option for active user participation in the energy market. Traditional LM methods usually require a centralized powerful control center and a two-way communication network between the system operators and energy end-users. The increasing user participation in smart grids may limit their applications. In this paper, a distributed solution for LM in emerging smart grids is proposed. The LM problem is formulated as a constrained optimization problem aiming at maximizing the overall utility of users while meeting the requirement for load reduction requested by the system operator, and is solved by using a distributed dynamic programming (DDP) algorithm. The algorithm is implemented via a distributed framework and thus can deliver a highly desired distributed solution. It avoids the required use of a centralized coordinator or control center, and can achieve satisfactory outcomes for LM. Simulation results with various test systems demonstrate its effectiveness.

Introduction

Traditional Load Management (LM) architectures rely on a centralized "hub-and-spoke" model where a single control center collects data from all users, computes an optimal strategy, and issues commands. As smart grids scale to include millions of participants and volatile renewable energy sources, this model faces significant challenges in scalability, communication overhead, and reliability. A single failure at the central node can compromise the entire system, and the computational burden grows exponentially with the number of users.

Methodology

To address these challenges, this paper proposes a decentralized approach using Distributed Dynamic Programming (DDP). The load management problem is modeled as a variation of the 0-1 knapsack problem, where the objective is to maximize total user utility subject to a global load reduction target.

The proposed distributed framework for load management agents (LMAs).

The system utilizes Load Management Agents (LMAs) to represent individual users or devices. These agents operate within a multi-agent framework and coordinate through a two-stage iterative process:

  1. Information Discovery (ID): Agents exchange local estimates of optimal utility and state with immediate neighbors.
  2. State Update (SU): Agents update their local states based on neighbor data to converge toward the global objective.

This approach supports asynchronous communication, allowing the system to function effectively even with varying network speeds or delays.

Evaluation and Results

The DDP algorithm was tested on several benchmarks, including the IEEE 14-bus system and large-scale synthetic systems with up to 1,062 agents.

Convergence and Accuracy

The algorithm demonstrates robust convergence. As shown in the optimization profiles, both the utility of individual agents and their corresponding load settings stabilize within a few iterations.

Convergence of agent utility and load settings during the optimization process.

Key results include:

  • Near-Optimality: DDP consistently achieves objective values within 10% of the global optimum found by centralized solvers.
  • Scalability: In a 1,062-agent test, the distributed algorithm converged in under 10 seconds, whereas the centralized method required over 100 seconds for computation alone.
  • Performance Index ($I_p$): While centralized methods are efficient for small systems, the distributed approach dominates in large-scale scenarios, with $I_p$ values significantly higher (up to 23.94) for systems with 162+ agents.

Discussion

The distributed approach offers several advantages beyond speed. It enhances resilience, as the network can still find solutions if individual links or agents fail, provided the communication graph remains connected. It also preserves privacy and autonomy; the system operator only broadcasts requirements and incentives, while local agents decide how to adjust their loads. Furthermore, the speed of DDP allows it to track rapid fluctuations in renewable energy, maintaining grid frequency within a tight $\pm 0.05$ Hz margin.

Limitations

Despite its strengths, DDP guarantees a "satisfactory" local optimum rather than a mathematical global optimum due to the non-convex nature of practical power problems. Future work will involve hardware-in-the-loop (HIL) testing to evaluate performance on physical communication hardware and the integration of distributed energy resources (DERs) like home batteries.

Conclusion

The proposed DDP-based distributed solution effectively manages grid loads without a centralized controller. By leveraging local coordination, it provides a scalable, resilient, and privacy-preserving alternative to traditional load management, making it well-suited for the evolving landscape of smart grids.

Summary updated

Abstract

Load management (LM) is being recognized as an important option for active user participation in the energy market. Traditional LM methods usually require a centralized powerful control center and a two-way communication network between the system operators and energy end-users. The increasing user participation in smart grids may limit their applications. In this paper, a distributed solution for LM in emerging smart grids is proposed. The LM problem is formulated as a constrained optimization problem aiming at maximizing the overall utility of users while meeting the requirement for load reduction requested by the system operator, and is solved by using a distributed dynamic programming (DDP) algorithm. The algorithm is implemented via a distributed framework and thus can deliver a highly desired distributed solution. It avoids the required use of a centralized coordinator or control center, and can achieve satisfactory outcomes for LM. Simulation results with various test systems demonstrate its effectiveness.
BibTeX
@article{zhang2016distributed,
  author = {Wei Zhang and Yinliang Xu and Sisi Li and MengChu Zhou and Wenxin Liu and Ying Xu},
  title = {{A Distributed Dynamic Programming-Based Solution for Load Management in Smart Grids}},
  journal = {IEEE Systems Journal},
  year = {2016},
  doi = {10.1109/jsyst.2016.2536141},
}
Figure