Solution of Nonlinear Equations and Optimization
One of the most widely-encountered specialized optimization problems is the linear programming problem and related problems in network optimization. Griva, Nash, and Sofer (2008) describe methods for such problems.
Stochastic optimization is discussed in some detail by Spall (2004). De Jong (2006) describes the basic ideas of evolutionary computation and how the methods can be used in a variety of optimization problems.
Many of the relevant details of numerical optimization for statistical applications are discussed by Rustagi (1994) and by Gentle (2009). The EM method for optimization is covered in some detail by Ng, Krishnan, and McLachlan (2004). The EM method itself was first described and analyzed systematically by Dempster, Laird, and Rubin (1977).
Most of the comprehensive scientific software packages such as the IMSL Libraries, Matlab, and R have functions or separate modules for solution of systems of nonlinear equations and for optimization.
The R function uniroot (which is zbrent in the IMSL Libraries) is based on an algorithm of Richard Brent that uses a combination of linear interpolation, inverse quadratic interpolation, and bisection to find a root of a univariate function in an interval whose endpoints evaluate to values with different signs. The R function polyroot (which is zpolrc or zpolcc in the IMSL Libraries) is based on the Traub-Jenkins algorithm to find the roots of a univariate polynomial.
KeywordsObjective Function Simulated Annealing Nonlinear Equation Descent Direction Secant Method
Unable to display preview. Download preview PDF.