Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression

Stephen C. Billups, Jeffrey Larson, Peter Graf

Research output: Contribution to journalArticlepeer-review

24 Scopus Citations


We propose a derivative-free algorithm for optimizing computationally expensive functions with computational error. The algorithm is based on the trust region regression method by Conn, Scheinberg, and Vicente [A. R. Conn, K. Scheinberg, and L. N. Vicente, IMA J. Numer. Anal., 28 (2008), pp. 721-748] but uses weighted regression to obtain more accurate model functions at each trust region iteration. A heuristic weighting scheme is proposed that simultaneously handles (i) differing levels of uncertainty in function evaluations and (ii) errors induced by poor model fidelity. We also extend the theory of Ë-poisedness and strong Ë-poisedness to weighted regression. We report computational results comparing interpolation, regression, and weighted regression methods on a collection of benchmark problems. Weighted regression appears to outperform interpolation and regression models on nondifferentiable functions and functions with deterministic noise.

Original languageAmerican English
Pages (from-to)27-53
Number of pages27
JournalSIAM Journal on Optimization
Issue number1
StatePublished - 2013

NREL Publication Number

  • NREL/JA-2C00-56795


  • Derivative-free optimization
  • Noisy function evaluations
  • Weighted regression models


Dive into the research topics of 'Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression'. Together they form a unique fingerprint.

Cite this