Uncertainty-Aware Model-Based Multi-Agent Deep Reinforcement Learning for Robust Active Voltage Control

Zhaoyang Liu, Liang Ma, Ke Wang, Junnan Zhang, Chenyi Si, Jun Yi, Chaoxu MuIEEE Transactions on Circuits and Systems I Regular Papers 2025 · DOI

Abstract

The large-scale injection of new energy systems into active distribution networks (ADNs) has caused voltage violations, challenging the stable operation of power grids. Recently, deep reinforcement learning (DRL) has emerged with great advantages in replacing traditional optimization methods for voltage regulation in ADNs. However, existing DRL studies face sampling inefficiency and overlook the robustness issue resulting from the uncertainties brought by renewable energy systems in ADNs, which seriously affects the application of DRL in the real world. In this paper, a novel uncertainty-aware model-based multi-agent deep reinforcement learning (MADRL) framework is proposed for robust active voltage control (AVC). First, a probabilistic ensemble of neural networks with different initializations is designed for uncertainties in environment model learning, and a hybrid data augmentation method is proposed to improve the learning efficiency and final performance of MADRL. Then, a multi-agent distributional soft actor-critic (MADSAC) framework is developed for robust voltage regulation by tackling various uncertainties in the ADN environment. Simulations are performed on the IEEE 33-bus distribution network and IEEE 141-bus distribution network to validate that the proposed model-based MADSAC algorithm can significantly improve sampling efficiency, robustness and performance in AVC.
BibTeX
@article{liu2025uncertainty,
  author = {Zhaoyang Liu and Liang Ma and Ke Wang and Junnan Zhang and Chenyi Si and Jun Yi and Chaoxu Mu},
  title = {{Uncertainty-Aware Model-Based Multi-Agent Deep Reinforcement Learning for Robust Active Voltage Control}},
  journal = {IEEE Transactions on Circuits and Systems I Regular Papers},
  year = {2025},
  doi = {10.1109/tcsi.2025.3588231},
}

Loading related content...

Figure