# Non-linear least squares

• L. E. Scales
Chapter
Part of the Macmillan Computer Science Series book series (COMPSS)

## Abstract

The methods of chapter 3 have been designed to be effective on all sufficiently smooth objective functions. However it is sometimes possible to devise methods that are more efficient should the objective function have a special form. By far the most important of these forms encountered in practice is a sum of the squares of other non-linear functions:
$$F{\text{(x) = }}\sum\limits_{i = 1}^m {f_i^2{\text{(x)}}}$$
(4.1.1)
The minimization of functions of this kind is called non-linear least squares. Note immediately that the objective function can never take a negative value in these problems. It is convenient to gather the functions f i together in vector form
$${\text{f(x) = }}\left[ {\begin{array}{*{20}{c}} {\begin{array}{*{20}{c}} {{f_1}} \\ {{f_2}} \\ {\begin{array}{*{20}{c}} . \\ . \end{array}} \\ . \end{array}} \\ {{f_m}} \end{array}} \right]$$
(4.1.2)
When we can write
$$F{\text{(x)=}}{{\text{f}}^T}{\text{(x)f(x)}}$$
(4.1.3)
It is the aim of this chapter to discuss some of the most effective methods currently available for minimizing functions of the form (4.1.3). But first let us see some ways in which such functions arise in practice.