Calibration by optimization without using derivatives

Abstract

Applications in engineering frequently require the adjustment of certain parameters. While the mathematical laws that determine these parameters often are well understood, due to time limitations in every day industrial life, it is typically not feasible to derive an explicit computational procedure for adjusting the parameters based on some given measurement data. This paper aims at showing that in such situations, direct optimization offers a very simple approach that can be of great help. More precisely, we present a numerical implementation for the local minimization of a smooth function $f:{\mathbb R}^n\rightarrow {\mathbb R}$ subject to upper and lower bounds without relying on the knowledge of the derivative of f. In contrast to other direct optimization approaches the algorithm assumes that the function evaluations are fairly cheap and that the rounding errors associated with the function evaluations are small. As an illustration, this algorithm is applied to approximate the solution of a calibration problem arising from an engineering application. The algorithm uses a Quasi-Newton trust region approach adjusting the trust region radius with a line search. The line search is based on a spline function which minimizes a weighted least squares sum of the jumps in its third derivative. The approximate gradients used in the Quasi-Newton approach are computed by central finite differences. A new randomized basis approach is considered to generate finite difference approximations of the gradient which also allow for a curvature correction of the Hessian in addition to the Quasi-Newton update. These concepts are combined with an active set strategy. The implementation is public domain; numerical experiments indicate that the algorithm is well suitable for the calibration problem of measuring instruments that prompted this research. Further preliminary numerical results suggest that an approximate local minimizer of a smooth non-convex function f depending on $n\le 300$ variables can be computed with a number of iterations that grows moderately with n.