Automatic differentiation of characterizing sequences

摘要

Gradient and Hessian matrix of a real function of several variables play an important role in many numerical methods, especially in nonlinear optimization. But little effort has been devoted to the computation of these entities so far. “The Hessian is not available”, this statement used to be an axiom in the optimization folklore for decades. Nevertheless, we will show how to obtain gradient and Hessian matrix of an explicitly given function “automatically” in an easy and straightforward manner. No quotients of differences are used. No manipulation of symbols is involved, we deal with numbers, not with formulas. Furthermore, we show how to apply automatic differentiation in special situations, arising in nonlinear optimization and in the multivariate Halley-method.