Optimization

Abstract

Numerous computational problems are appropriately modeled in terms of an optimal property. Indeed, we have already seen two examples in linear regression and principal component analysis, both of which describe a computational and modeling intent in terms of an optimal quality. Any earnest taxonomy of the field of optimization would demonstrate a wide girth and a sizable depth of application, solution procedure, and mathematical analysis. In this chapter we restrict our attention to the general themes that underlie many of the algorithms used to solve optimization problems. The chapter is divided into three parts, those being unconstrained, constrained, and global optimization. The algorithms contained herein provide a suite of computational procedures to solve a variety of optimization problems.