Brief Review of Some Optimization Methods

Abstract

In this book we have discussed many data analysis methods that use some form of optimization. Many data analysis algorithms are essentially optimization algorithms. In this appendix we briefly review the basics of the optimization methods used in this book: optimization with derivatives, gradient descent, and Lagrange optimization. For a more comprehensive overview of optimization methods the reader is referred to the literature, for example [1, 2, 3].