Newton-Type Methods for Nonlinear Least Squares Using Restricted Second Order Information

Summary

In the paper, a special approximated Newton method for minimizing a sum of squares $f\left( \mathfrak{x} \right) = \frac{1} {2}\left\| {f\left( \mathfrak{x} \right)} \right\|^2 = \frac{1} {2}\Sigma _{i = 1}^m \left[ {F_i \left( \mathfrak{x} \right)} \right]^2$ is introduced. In this Restricted Newton method, the Hessian $H = G + S$ of f where $G = \left( {F'} \right)^T F',S = F \circ F''$ , is approximated by $A_{RN} = G + B$ where $B = Z_2 Z_2^T SZ_2 Z_2^T$ is the restriction of the second order term S on the subspace im Z2 spanned by the eigenvectors of the Gauss-Newton matrix G which belong to the q smallest eigenvalues of G. Some properties of this approximation are derived, and a related trust region method is tested on hand of some test functions from the literature.