• Allen Mclntosh
Chapter
Part of the Lecture Notes in Statistics book series (LNS, volume 10)

## Abstract

Let ø(β) be a function mapping R r into R. Here and in the next two sections, we suppose that ø(β) has a unique minimizer on R r , which we will denote β. When a solution in closed form is impossible or impractical, iterative methods are usually used to find β. Starting with an initial approximation β(0), an iterative method attempts to construct a sequence β(0), β(1), β(2), … that converges to β. Given β(k) a particular approximation to β, a better approximation may be computed as β(k+1) = β(k)(k)k = 0,1, … (3.1.1) where the search direction p(k) is a non-zero vector in R r and the step length α(k) is chosen to produce a reasonable decrease in ø. The procedure of choosing α(k) is called a line search; it is said to be exact if α(k) minimizes ø(α) = α (β(k)+αP(k)) (3.1.2) We will adopt the notation
$$g\left( \beta \right) = \left[ {{{\partial \phi } \over {\partial \beta }}} \right] = {\left[ {{{\partial \phi } \over {\partial {\beta _1}}},{{\partial \phi } \over {\partial {\beta _2}}}, \cdots ,{{\partial \phi } \over {\partial \beta r}}} \right]^t}$$
g(k) = g(k)) and
$$G\left( \beta \right) = {\partial \over {\partial \beta }}{\left[ {{{\partial \phi } \over {\partial \beta }}} \right]^t} = {\left[ {{{{\partial ^2}\phi } \over {\partial {\beta _i}\partial {\beta _j}}}} \right]_{r \times r}}$$
G(k) = G(β(k)) for the first and second partial derivatives of ø with respect to β. (We assume that these exist everywhere in R r .)

## Keywords

Search Direction Line Search Steep Descent Minimal Polynomial Unique Minimizer
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.