Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures

Juliane Mueller, Vincent Dumont, Xiangyang Ju

Research output: NRELPresentation


Deep Learning (DL) models are increasingly used throughout the sciences. However, their performance and usefulness depend greatly on their architecture which is defined by hyperparameters such as the number of nodes, layers, the learning rate, etc. Tuning these hyperparameters is time-consuming because evaluating their performance requires a lengthy training step. Stochastic optimizers used in training lead to performance variability and potentially prediction reliability issues. In this talk, we will describe an automated optimization method based on surrogate models and active learning strategies for tuning DL model architectures. We take into account the prediction variability with the goal to identify architectures that make reliable and robust predictions. We demonstrate our developments on an application arising in particle physics.
Original languageAmerican English
Number of pages19
StatePublished - 2022

Publication series

NamePresented at the 2022 INFORMS Annual Meeting, 16-19 October 2022, Indianapolis, Indiana

NREL Publication Number

  • NREL/PR-2C00-84341


  • deep learning
  • high energy physics
  • hyperparameter optimization
  • surrogate modeling


Dive into the research topics of 'Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures'. Together they form a unique fingerprint.

Cite this