Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Di Cao, Junbo Zhao, Weihao Hu, Nanpeng Yu, Fei Ding, Qi Huang, Zhe Chen

Research output: Contribution to journalArticlepeer-review

51 Scopus Citations


Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several sub-networks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each sub-network is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33- and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

Original languageAmerican English
Pages (from-to)149-165
Number of pages17
JournalIEEE Transactions on Smart Grid
Issue number1
StatePublished - 1 Jan 2022

Bibliographical note

Publisher Copyright:
© 2010-2012 IEEE.

NREL Publication Number

  • NREL/JA-5D00-81108


  • Active distribution systems
  • coordinated control
  • deep reinforcement learning
  • PV inverters
  • Volt-Var optimization
  • voltage regulation


Dive into the research topics of 'Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems'. Together they form a unique fingerprint.

Cite this