# Calibration by optimization without using derivatives

- 212 Downloads

## Abstract

Applications in engineering frequently require the adjustment of certain parameters. While the mathematical laws that determine these parameters often are well understood, due to time limitations in every day industrial life, it is typically not feasible to derive an explicit computational procedure for adjusting the parameters based on some given measurement data. This paper aims at showing that in such situations, direct optimization offers a very simple approach that can be of great help. More precisely, we present a numerical implementation for the local minimization of a smooth function \(f:{\mathbb R}^n\rightarrow {\mathbb R}\) subject to upper and lower bounds without relying on the knowledge of the derivative of *f*. In contrast to other direct optimization approaches the algorithm assumes that the function evaluations are fairly cheap and that the rounding errors associated with the function evaluations are small. As an illustration, this algorithm is applied to approximate the solution of a calibration problem arising from an engineering application. The algorithm uses a Quasi-Newton trust region approach adjusting the trust region radius with a line search. The line search is based on a spline function which minimizes a weighted least squares sum of the jumps in its third derivative. The approximate gradients used in the Quasi-Newton approach are computed by central finite differences. A new randomized basis approach is considered to generate finite difference approximations of the gradient which also allow for a curvature correction of the Hessian in addition to the Quasi-Newton update. These concepts are combined with an active set strategy. The implementation is public domain; numerical experiments indicate that the algorithm is well suitable for the calibration problem of measuring instruments that prompted this research. Further preliminary numerical results suggest that an approximate local minimizer of a smooth non-convex function *f* depending on \(n\le 300 \) variables can be computed with a number of iterations that grows moderately with *n*.

## Keywords

Calibration of measuring instruments Minimization without derivatives Direct search Quadratic model## Notes

### Acknowledgments

The authors would like to thank Andrew Conn, Roland Freund, and Arnold Neumaier for helpful criticism and an unknown referee for comments that helped to improve this paper.

## References

- Anderson EJ, Ferris MC (2001) A direct search algorithm for optimization of expensive functions by surrogates. SIAM J Optim 11:837–857MathSciNetCrossRefMATHGoogle Scholar
- Audet C (2014) A survey on direct search methods for Blackbox optimization and their applications. In: Pardalos PM, Rassias TM (eds) Mathematics Without Boundaries. Springer, New York, pp 31–56Google Scholar
- Audet C, Dennis JE Jr (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(1):188–217MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Ianni A, Le Digabel S, Tribes C (2014) Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J Optim 24(2):621–642MathSciNetCrossRefMATHGoogle Scholar
- Bingham D (2005) Virtual Library of Simulation Experiments: Test Functions and Datasets, Optimization Test Problems. http://www.sfu.ca/\(\tilde{\ \ }\)ssurjano/optimization.htmlGoogle Scholar
- BIPB - JCGM 200:2012 (2012) International Vocabulary of Metrology—Basic and General Concepts and Associated Terms (VIM), vol. 28, 3rd edn. Springer, Berlin, HeidelbergGoogle Scholar
- Booker AJ, Dennis JE Jr, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17:1–13CrossRefGoogle Scholar
- Botsaris CA, Jacobson DH (1976) A Newton-type curvilinear search method for optimization. J Math Anal Appl 54(1):217–229MathSciNetCrossRefMATHGoogle Scholar
- Conn AR, Scheinberg K, Toint PL (1997a) On the convergence of derivative-free methods for unconstrained optimization. In: Powell MJD, Buhmann MD, Iserles A (eds) Approximation Theory and Optimization. Cambridge University Press, Cambridge, pp 83–108Google Scholar
- Conn AR, Scheinberg K, Toint PL (1997b) Recent progress in unconstrained nonlinear optimization without derivatives. Math Program 79:397–414MathSciNetMATHGoogle Scholar
- Conn AR, Scheinberg K, Vicente LN (2009a) Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J Optim 20:387–415MathSciNetCrossRefMATHGoogle Scholar
- Conn AR, Scheinberg K, Vicente LN (2009b) Introduction to Derivative-Free Optimization, MPS-SIAM Series on Optimization. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
- Csendes T (1988) Nonlinear parameter estimation by global optimization—efficiency and reliability. Acta Cybern 8:361–370MathSciNetMATHGoogle Scholar
- Csendes T, Pal L, Oscar J, Sendin H, Banga JR (2008) The GLOBAL Optimization Method Revisited, Report. Institute of Informatics, University of Szeged, HungaryMATHGoogle Scholar
- Custodio AL, Vicente LN (2007) Using sampling and simplex derivatives in pattern search methods. SIAM J Optim 18:537–555MathSciNetCrossRefMATHGoogle Scholar
- Custodio AL, Rocha H, Vicente LN (2010) Incorporating minimum Frobenius norm models in direct search. Comput Optim Appl 46:265–278MathSciNetCrossRefMATHGoogle Scholar
- Dennis JE Jr, Echebest N, Guardarucci MT, Martinez JM, Scolnik HD, Vacchino C (1991) A curvilinear search using tridiagonal secant updates for unconstrained optimization. SIAM J Optim 1(3):333–357MathSciNetCrossRefMATHGoogle Scholar
- Elster C, Neumaier A (1995) A grid algorithm for bound constrained optimization of noisy functions. IMA J Numer Anal 15:585–608MathSciNetCrossRefMATHGoogle Scholar
- Goldstein H, Poole C, Safko J (2002) Classical Mechanics, 3rd edn. Addison-Wesley, San Fransisco, pp 150–154MATHGoogle Scholar
- Golub GH, Van Loan CF (1993) Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore/LondonMATHGoogle Scholar
- Hansen N (2006) The CMA Evolution Strategy: A Comparing Review. In: Lozano JA, Larraga P, Inza I, Bengoetxea E (eds) Towards a New Evolutionary Computation. Advances in Estimation of Distribution Algorithms. Springer, Heidelberg, pp 75–102CrossRefGoogle Scholar
- Hansen N, Niederberger ASP, Guzzella L, Koumoutsakos P (2009) A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans Evol Comput 13(1):180–197CrossRefGoogle Scholar
- Huyer W, Neumaier A (2008) Snobfit—stable noisy optimization by branch and fit. ACM Trans Math Softw 35:25 (Article 9)MathSciNetCrossRefGoogle Scholar
- Jarre F (2015) MWD, Smooth Minimization Without using Derivatives, a Matlab Collection. http://www.opt.uni-duesseldorf.de/en/forschung-fs.htmlGoogle Scholar
- Jarre F, Stoer J (2004) Optimierung. Springer, Berlin/Heidelberg/New YorkCrossRefGoogle Scholar
- Kiefer J (1953) Sequential minimax search for a maximum. Proc Am Math Soc 4(3):502–506MathSciNetCrossRefMATHGoogle Scholar
- Lewis RM, Torczon V, Trosset MW (2000) Direct search methods: then and now. J Comput Appl Math 124(1–2):191–207MathSciNetCrossRefMATHGoogle Scholar
- Li RC (2008) On Meinardus’ examples for the conjugate gradient method. Math Comput 77(261):335–352MathSciNetCrossRefMATHGoogle Scholar
- Powell MDJ (1970) A new algorithm for unconstrained optimization. In: Rosen JB, Mangasarian OL, Ritter K (eds) Nonlinear Programming. Academic Press, New York, pp 31–65CrossRefGoogle Scholar
- Powell MDJ (1998) Direct search algorithms for optimization calculations. Acta Numer 7:287–336MathSciNetCrossRefMATHGoogle Scholar
- Rios LM, Sahinidis NV (2013) Derivative-free optimization: a review of algorithms and comparison of software implementations. J Glob Optim 56:1247–1293MathSciNetCrossRefMATHGoogle Scholar
- Schmidt M (2012) minFunc: Unconstrained Differentiable Multivariate Optimization in Matlab. http://www.cs.ubc.ca/ schmidtm/Software/minFunc.htmlGoogle Scholar
- Stoer J, Bulirsch R (2002) “Introduction to Numerical Analysis”, Third Edition, Texts in Applied Mathematics. Springer, BerlinMATHGoogle Scholar
- Vaz AIF, Vicente LN (2007) A particle swarm pattern search method for bound constrained global optimization. J Glob Optim 39:197–219MathSciNetCrossRefMATHGoogle Scholar