Data-Driven Optimal Control Strategy for Virtual Synchronous Generator via Deep Reinforcement Learning Approach

Yushuai Li, Wei Gao, Weihang Yan, Shuo Huang, Rui Wang, Vahan Gevorgian, David Gao

Research output: Contribution to journalArticlepeer-review

76 Scopus Citations

Abstract

This paper aims at developing a data-driven optimal control strategy for virtual synchronous generator (VSG) in the scenario where no expert knowledge or requirement for system model is available. Firstly, the optimal and adaptive control problem for VSG is transformed into a reinforcement learning task. Specifically, the control variables, i.e., virtual inertia and damping factor, are defined as the actions. Meanwhile, the active power output, angular frequency and its derivative are considered as the observations. Moreover, the reward mechanism is designed based on three preset characteristic functions to quantify the control targets: (1) maintaining the deviation of angular frequency within special limits; (2) preserving well-damped oscillations for both the angular frequency and active power output; (3) obtaining slow frequency drop in the transient process. Next, to maximize the cumulative rewards, a decentralized deep policy gradient algorithm, which features model-free and faster convergence, is developed and employed to find the optimal control policy. With this effort, a data-driven adaptive VSG controller can be obtained. By using the proposed controller, the inverter-based distributed generator can adaptively adjust its control variables based on current observations to fulfill the expected targets in model-free fashion. Finally, simulation results validate the feasibility and effectiveness of the proposed approach.

Original languageAmerican English
Article number9335702
Pages (from-to)919-929
Number of pages11
JournalJournal of Modern Power Systems and Clean Energy
Volume9
Issue number4
DOIs
StatePublished - Jul 2021

Bibliographical note

Publisher Copyright:
© 2013 State Grid Electric Power Research Institute.

NREL Publication Number

  • NREL/JA-5D00-76712

Keywords

  • Adaptive control
  • deep learning
  • reinforcement learning
  • virtual synchronous generator (VSG)

Fingerprint

Dive into the research topics of 'Data-Driven Optimal Control Strategy for Virtual Synchronous Generator via Deep Reinforcement Learning Approach'. Together they form a unique fingerprint.

Cite this