A Parallel Hub-and-Spoke System for Large-Scale Scenario-Based Optimization Under Uncertainty

Bernard Knueven, David Mildebrath, Christopher Muir, John Siirola, Jean-Paul Watson, David Woodruff

Research output: Contribution to journalArticlepeer-review

2 Scopus Citations

Abstract

Practical solution of stochastic programming problems generally requires the use of parallel computing resources. Here, we describe the open source package mpi-sppy, in which efficient and scalable parallelization is a central feature. We report computational experiments that demonstrate the ability to solve very large stochastic programming problems—including mixed-integer variants—in minutes of wall clock time, efficiently leveraging significant parallel computing resources. We report results for the largest publicly available instances of stochastic mixed-integer unit commitment problems, solving to provably tight optimality gaps. In addition, we introduce a novel software architecture that facilitates combinations of methods for accelerating convergence that can be combined in plug-and-play manner. The mpi-sppy package is written in Python, leverages the widely used Pyomo (http://www.pyomo.org) library for modeling mathematical programs, builds on existing MPI implementations to ensure efficiency and scalability, and is available via http://github.com/Pyomo/mpi-sppy .

Original languageAmerican English
Pages (from-to)591-619
Number of pages29
JournalMathematical Programming Computation
Volume15
Issue number4
DOIs
StatePublished - 2023

Bibliographical note

Publisher Copyright:
© 2023, Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society.

NREL Publication Number

  • NREL/JA-2C00-84450

Keywords

  • Decomposition strategies
  • Parallel computing
  • Progressive hedging
  • Stochastic programming

Fingerprint

Dive into the research topics of 'A Parallel Hub-and-Spoke System for Large-Scale Scenario-Based Optimization Under Uncertainty'. Together they form a unique fingerprint.

Cite this