Handling Iterative Solvers in an Algorithmic Differentiation Framework Using Implicit Methods

Differentiable programming is a powerful concept as it enables the seemly propagation of gradients through functions, algorithms, and/or whole physics simulations. These gradients are useful for a wide variety of applications, including sensitivity studies and machine learning, but one of particular interest is optimization. Gradient-based optimization, enabled through automatic/algorithmic differentiation (AD), can be used on predictive physical models to efficiently optimize a set of design variables. AD methods are a particularly promising approach to complex physics simulations because they can be shown to scale well with an increasing number of design variables; however, care must be taken when coupling between different models or different states of a single model.