PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems

David Biagioni, Xiangyu Zhang, Dylan Wald, Deepthi Vaidhynathan, Rohit Chintala, Jennifer King, Ahmed S. Zamzam

Research output: Contribution to conferencePaperpeer-review

5 Scopus Citations

Abstract

We present the PowerGridworld open source software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-Agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-Agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld helps to fill this gap. To highlight PowerGridworld's key features, we present two case studies and demonstrate learning MARL policies using both OpenAI's multi-Agent deep deterministic policy gradient (MADDPG) and RL-Lib's proximal policy optimization (PPO) algorithms. In both cases, at least some subset of agents incorporates elements of the power flow solution at each time step as part of their reward (negative cost) structures.

Original languageAmerican English
Pages565-570
Number of pages6
DOIs
StatePublished - 28 Jun 2022
Evente-Energy '22: Thirteenth ACM International Conference on Future Energy Systems -
Duration: 28 Jun 20221 Jul 2022

Conference

Conferencee-Energy '22: Thirteenth ACM International Conference on Future Energy Systems
Period28/06/221/07/22

Bibliographical note

Publisher Copyright:
© 2022 ACM.

NREL Publication Number

  • NREL/CP-2C00-83699

Keywords

  • deep learning
  • multi-Agent systems
  • OpenAI gym
  • power systems
  • reinforcement learning

Fingerprint

Dive into the research topics of 'PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems'. Together they form a unique fingerprint.

Cite this