Real-time cooperative voltage regulation strategy of distribution network based on approximate dynamic programming

Jingjing Gu, Mingyu Guo, Ruijie Li, Jihe Xu, Yanyi Ye, Zhizhong Hu, Shichang CuiFrontiers in Energy Research 2024 · DOI

Abstract

Large-scale distributed renewable energy in the distribution network can result in reliability issues such as exceeding voltage limits and overloading power lines. Additionally, the rapid growth of electric vehicles has caused a surge in power demand in the distribution network. Therefore, how to guarantee the real-time stability of distribution network voltage under uncertain environment using available resources is an urgent problem to be solved. To this end, this paper proposes a cooperative voltage regulation of an on-load voltage regulator and electric vehicles for a distribution network considering multiple uncertainties. Firstly, an optimization model of the electric vehicle clusters considering the charging location and power is established, and the traditional on-load voltage regulator of distribution networks is also considered along with network operational constraints. Subsequently, a real-time cooperative voltage regulation strategy based on approximate dynamic programming is proposed, which employs segmented linear functions to process the value function to reduce the distribution network voltage offset and to ensure optimization accuracy. Numerical simulation results validate the feasibility and the effectiveness of the proposed cooperative voltage regulation technology involving electric vehicles and on-load tap changer in the distribution network.

Introduction

As modern distribution networks integrate increasing amounts of solar panels, wind turbines, and electric vehicles (EVs), they face a "stability paradox." While renewable energy is essential for sustainability, its volatility causes unpredictable voltage spikes and dips. Simultaneously, the surge in EV charging adds massive, concentrated loads that can strain existing infrastructure.

Traditional tools like On-Load Tap Changers (OLTC)—mechanical regulators that adjust transformer ratios—are often too slow to respond to rapid fluctuations or suffer from mechanical wear if used too frequently. Conversely, managing thousands of individual EVs as separate variables is computationally impossible in real-time. This paper addresses the challenge of coordinating these flexible EV resources with grid hardware to maintain stability despite significant environmental uncertainty.

Methodology: Cooperative Voltage Regulation via ADP

The authors propose a technical approach centered on Approximate Dynamic Programming (ADP). The problem is first reframed as a Markov Decision Process (MDP), where the system makes optimal current decisions while accounting for future impacts. To handle the complexity of the network, EVs are grouped into "clusters" with similar behaviors, converting thousands of variables into a manageable set.

The core innovation lies in using segmented linear functions to approximate the value function—a mathematical "look-ahead" that estimates the future cost of current actions. This allows the model to handle non-linear grid constraints with high efficiency.

Real-time voltage regulation strategy process based on ADP

As shown in the strategy process, the system operates in two stages:

  1. Offline Training: The algorithm "learns" the slopes of the linear functions using historical data and Monte Carlo simulations.
  2. Online Optimization: The grid controller uses these pre-trained slopes to make lightning-fast decisions (averaging 0.026 seconds) based on actual real-time conditions, such as current renewable output and load demand.

Experimental Setup

The strategy was tested using the IEEE-33 Node System, a standard benchmark for distribution networks. The simulation environment included diverse distributed energy resources and EV charging stations strategically placed throughout the network.

The simulation case based on the IEEE-33 node system

To ensure the algorithm's robustness, the researchers generated 200 training scenarios using Monte Carlo sampling. These scenarios modeled wind, solar, and load errors using various statistical distributions (Normal, Beta, and t-location-scale) to ensure the system could handle different types of forecast noise.

Key Results and Insights

The numerical simulations yielded several critical findings:

  • Voltage Stability: The system achieved a 54.6% reduction in voltage deviation compared to using an OLTC alone.
  • High Precision: The ADP method achieved 99.78% accuracy relative to a theoretically perfect global optimization, significantly outperforming "Myopic" strategies and standard Model Predictive Control (MPC).
  • Real-Time Viability: With an average online solution time of 0.026 seconds, the method is highly suitable for hardware implementation where instant decision-making is required.
  • EVs as Grid Assets: The model demonstrates that EVs can be treated as "flexible resources" that modulate charging speeds to act as a high-speed buffer for voltage swings, rather than just being a burden on the grid.

Conclusion

This paper presents a scalable and efficient solution for real-time voltage regulation. By leveraging ADP and segmented linear functions, the proposed strategy effectively coordinates traditional grid hardware with modern EV clusters. While the results are promising, future work will need to address the economic incentive structures required to compensate EV owners for grid participation and the long-term impact of such regulation on battery health.

Summary updated

Abstract

Large-scale distributed renewable energy in the distribution network can result in reliability issues such as exceeding voltage limits and overloading power lines. Additionally, the rapid growth of electric vehicles has caused a surge in power demand in the distribution network. Therefore, how to guarantee the real-time stability of distribution network voltage under uncertain environment using available resources is an urgent problem to be solved. To this end, this paper proposes a cooperative voltage regulation of an on-load voltage regulator and electric vehicles for a distribution network considering multiple uncertainties. Firstly, an optimization model of the electric vehicle clusters considering the charging location and power is established, and the traditional on-load voltage regulator of distribution networks is also considered along with network operational constraints. Subsequently, a real-time cooperative voltage regulation strategy based on approximate dynamic programming is proposed, which employs segmented linear functions to process the value function to reduce the distribution network voltage offset and to ensure optimization accuracy. Numerical simulation results validate the feasibility and the effectiveness of the proposed cooperative voltage regulation technology involving electric vehicles and on-load tap changer in the distribution network.
BibTeX
@inproceedings{gu2024real,
  author = {Jingjing Gu and Mingyu Guo and Ruijie Li and Jihe Xu and Yanyi Ye and Zhizhong Hu and Shichang Cui},
  title = {{Real-time cooperative voltage regulation strategy of distribution network based on approximate dynamic programming}},
  booktitle = {Frontiers in Energy Research},
  year = {2024},
  doi = {10.3389/fenrg.2024.1512832},
}
Figure