Iterative Optimization

Abstract

In this chapter we introduce iterative search methods for minimizing cost functions, and in particular, the $J_{\mathrm MSE}$ function. We focus on the methods of Steepest Descent and Newton-Raphson, which belong to the family of deterministic gradient algorithms. Although these methods still require knowledge of the second order statistics as does the Wiener filter, they find this solution iteratively. We also study the convergence of both algorithms and include simulation results to provide more insights on their performance. Understanding their functioning and convergence properties is very important as they will be the basis for the development of stochastic gradient adaptive filters in the next chapter.