Algorithms of Nonsmooth Optimization
In our applications we have to deal with two types of nonsmooth problems: the minimization of a nonsmooth functional f and the solution of nonsmooth equations. This chapter presents in short two basic algorithms which can cope with the difficulty caused by the nondifferentiability. These two codes are working horses in the numerical part of this book.
KeywordsLine Search Convergence Analysis Trust Region Cluster Point Equilibrium Constraint
Unable to display preview. Download preview PDF.
- For a general description of the subgradient methods, we refer to the monograph by Shor, 1985.Google Scholar
- The cutting plane method is independently due to Cheney and Goldstein, 1959 and Kelley, 1960. The step from the pure cutting plane model to the bundle concept was done by Lemaréchal, 1974; Lemaréchal, 1975 and Wolfe, 1975, and the algorithmic realization M1FC1 is due to Lemaréchal and Imbert, 1985. The idea to combine the bundle concept with the trust region technique was around for some time before it was studied in detail in Schramm, 1989 and Schramm and Zowe, 1992 and implemented as code BT; see Schramm and Zowe, 1991.Google Scholar
- Closely related concepts like the proximal point idea were studied, e.g., by Kiwiel, 1990. Recent extensions of bundle methods which try to improve the numerical behaviour by incorporating second order information are due to Lemaréchal and Sagastizábal, 1997; Lukšan and Vlček, 1996; Mifflin, 1996 and Mifflin et al., 1996.Google Scholar
- The monographs Kiwiel, 1985 and Hiriart-Urruty and Lemaréchal, 1993 are excellent references for a complete and detailed setting of the bundle concept and related algorithmic ideas.Google Scholar
- Starting with a technical report by Josephy, 1979, much attention has been paid to suitable modifications of the Newton’s method to the solution of nonsmooth equations. In the early papers (e.g. Pang, 1990b), the Newton iteration was constructed by the so-called B—derivative; cf. (3.66). In Kummer, 1992, another possibilities were proposed, among other also (3.69). This variant of the Newton’s method was then studied in many papers (e.g. Qi and Sun, 1993; Qi, 1993) and led to effective implementations for both NCPs (DeLuca et al., 1996) and VIs (Facchinei et al., 1995).Google Scholar