Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

Ying Zhang, Xinan Wang, Jianhui Wang, Yingchen Zhang

Research output: Contribution to journalArticlepeer-review

149 Scopus Citations


This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (DRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions in the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the agents. A delicately designed reward function guides these agents to interact with the distribution system, in the direction of reinforcing voltage regulation and power loss reduction simultaneously. The forward-backward sweep method for radial three-phase distribution systems provides accurate power flow results within a few iterations to the DRL environment. The proposed method realizes the dual goals for VVO. We test this algorithm on the unbalanced IEEE 13-bus and 123-bus systems. Numerical simulations validate the excellent performance of this method in voltage regulation and power loss reduction.
Original languageAmerican English
Pages (from-to)361-371
Number of pages11
JournalIEEE Transactions on Smart Grid
Issue number1
StatePublished - 2021

NREL Publication Number

  • NREL/JA-5D00-78897


  • artificial intelligence
  • deep reinforcement learning
  • smart inverter
  • unbalanced distribution systems
  • volt-VAR optimization
  • voltage regulation


Dive into the research topics of 'Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems'. Together they form a unique fingerprint.

Cite this