Nonsmooth Optimization Methods

  • Jaroslav Haslinger
  • Markku Miettinen
  • Panagiotis D. Panagiotopoulos
Part of the Nonconvex Optimization and Its Applications book series (NOIA, volume 35)


From the previous chapters we know that after the discretization, elliptic and parabolic hemivariational inequalities can be transformed into substationary point type problems for locally Lipschitz superpotentials and as such will be solved. There is a class of mathematical programming methods especially developed for this type of problems. The aim of this chapter is to give an overview of nonsmooth optimization techniques with special emphasis on the first and the second order bundle methods. We present their basic ideas in the convex case and necessary modifications for nonconvex optimization. We shall use them in the next chapter for the numerical realization of several model examples. Let us mention also that bundle type methods can be applied to a large class of locally Lipschitz superpotentials without any special requirement on their properties (such as the difference of two convex functions, e.g.).


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Auslender, A. (1987). Numerical methods for nondifferentiable convex optimization. Mathematical Programming Study, 30: 102–126.MathSciNetzbMATHCrossRefGoogle Scholar
  2. Bihain, A. (1984). Optimization of upper semidifferentiable functions. Journal of Optimization Theory and Applications, 4: 545–568.MathSciNetCrossRefGoogle Scholar
  3. Bonnans, J. F., Gilbert, J. C., Lemaréchal, C., and Sagastizâbal, C. (1995). A family of variable metric proximal methods. Mathematical Programming, 68: 15–47.MathSciNetzbMATHGoogle Scholar
  4. Brännlund, U. (1994). A descent method with relaxation type step. In Henry, J. and Yvon, J. P., editors, Lecture Notes in Control and Information Sciences, volume 197, pages 177–186, New York. Springer-Verlag.Google Scholar
  5. Brännlund, U., Kiwiel, K. C., and Lindberg, P. O. (1996). Preliminary computational experience with a descent level method for convex nondifferentiable optimization. In zal, J. D. and Fidler, J., editors, System Modelling and Optimization, pages 387–394, London. Chapman and Hall.Google Scholar
  6. Cheney, E. W. and Goldstein, A. A. (1959). Newton’s method for convex programming and tchebycheff approximation. Numerische Mathematik, 1: 253–268.MathSciNetzbMATHCrossRefGoogle Scholar
  7. Fletcher, R. (1987). Practical Methods of Optimization. John Wiley, Chichester.zbMATHGoogle Scholar
  8. Gaudioso, M. and Monaco, M. F. (1982). A bundle type approach to the unconstrained minimization of convex nonsmooth functions. Mathematical Programming, 23: 216–226.MathSciNetzbMATHCrossRefGoogle Scholar
  9. Gaudioso, M. and Monaco, M. F. (1988). Some techniques for finding the search direction in nonsmooth minimization problems. Technical Report 75, Dipartimento di Sistemi, Universita’ della Calabria.Google Scholar
  10. Gaudioso, M. and Monaco, M. F. (1992). Variants to the cutting plane approach for convex nondifferentiable optimization. Optimization, 25: 65–75.MathSciNetzbMATHCrossRefGoogle Scholar
  11. Kelley, J. E. (1960). The cutting plane method for solving convex programs SIAM J, 8: 703–712.MathSciNetGoogle Scholar
  12. Kiwiel, K. C. (1985). Methods of Descent for Nondifferentiable Optimization. Springer-Verlag, Berlin.zbMATHGoogle Scholar
  13. Kiwiel, K. C. (1990). Proximity control in bundle methods for convex nondifferentiable optimization. Mathematical Programming, 46: 105–122.MathSciNetzbMATHCrossRefGoogle Scholar
  14. Kiwiel, K. C. (1991). A tilted cutting plane proximal bundle method for convex nondifferentiable optimization. Operations Research Letters, 10: 75–81.MathSciNetzbMATHCrossRefGoogle Scholar
  15. Kiwiel, K. C. (1995). Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequlities. Mathematical Programming, 69: 89–109.MathSciNetzbMATHGoogle Scholar
  16. Kiwiel, K. C. (1996). Restricted step and levenberg-marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM Journal on Optimization, 6: 227–249.MathSciNetzbMATHCrossRefGoogle Scholar
  17. Lemaréchal, C. (1975). An extension of davidon methods to non differentiable problems. Mathematical Programming Study, 31: 95–109.CrossRefGoogle Scholar
  18. Lemaréchal, C. (1976). Combining kelley’s and conjucate gradient methods. In Abstracts of IX International Symposium on Mathematical Programming, Budapest, Hungary.Google Scholar
  19. Lemaréchal, C. (1978). Nonsmooth optimization and descent methods. Technical report, IIASA-report, Laxemburg, Austria.Google Scholar
  20. Lemaréchal, C. (1982). Numerical experiments in nonsmooth optimization. In Nurminski, E. A., editor, Progress in Nondifferentiable Optimization, ITASAreport, pages 61–84, Laxemburg, Austria.Google Scholar
  21. Lemaréchal, C. (1989). Nondifferentiable optimization. In Nemhauser, G. L., Kan, A. H. G. R., and Todd, M. J., editors, Optimization, pages 529–572, Amsterdam. North-Holland.CrossRefGoogle Scholar
  22. Lemaréchal, C., Nemirovskii, A., and Nesterov, Y. (1995). New variants of bundle methods. Mathematical Programming, 69: 111–147.MathSciNetzbMATHCrossRefGoogle Scholar
  23. Lemaréchal, C. and Sagastizdbal, C. (1994). An approach to variable metric bundle methods. In Henry, J. and Yvon, J. P., editors, Lecture Notes in Control and Information Sciences, volume 197, pages 144–162, New York. Springer-Verlag.Google Scholar
  24. Lemaréchal, C., Strodiot, J.-J., and Bihain, A. (1981). On a bundle algorithm for nonsmooth optimization. In Mangasarian, O. L., Mayer, R. R., and Robinson, S. M., editors, Nonlinear Programming, volume 4, pages 245–281, New York. Academic Press.Google Scholar
  25. Levenberg, K. (1944). A method for the solution of certain nonlinear problems in least squares. Quart. Appl. Math., 2: 164–166.MathSciNetzbMATHGoogle Scholar
  26. LukSan, L. and Vlcek, J. (1995). A bundle-newton method for nonsmooth unconstrained minimization. Technical report 654, Institute of Computer Science, Academy of Sciences of the Czech Republic.Google Scholar
  27. Mäkelä, M. M. (1989). Methods and algorithms for nonsmooth optimization. Reports on Applied Mathematics and Computing 2, University of Jyväskylä, Department of Mathematics.Google Scholar
  28. Mäkelä, M. M., Miettinen, M., Luksan, L., and Vlcek, J. (1999). Comparing nonsmooth nonconvex bundle methods in solving hemivariational inequalities. J. Global Optim., 14: 117–135.MathSciNetzbMATHCrossRefGoogle Scholar
  29. Mäkelä, M. M. and Neittaanmäki, P. (1992). Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., Singapore.zbMATHGoogle Scholar
  30. Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics, 11: 431–441.MathSciNetzbMATHCrossRefGoogle Scholar
  31. Mifflin, R. (1977). An algorithm for constrained optimization with semismooth functions. Mathematics of Operations Research, 2: 191–207.MathSciNetzbMATHCrossRefGoogle Scholar
  32. Mifflin, R. (1982). A modification and an extension of lemaréchal’s algorithm for nonsmooth minimization. Mathematical Programming Study, 17: 77–90.MathSciNetzbMATHCrossRefGoogle Scholar
  33. Mifflin, R. (1996). A quasi-second-order proximal bundle algorithm. Mathematical Programming, 73: 51–72.MathSciNetzbMATHGoogle Scholar
  34. Moreau, J. J. (1965). Proximité et dualité dans un espace hilbertien. Bulletin de la Société Mathématique de France, 93: 273–299.MathSciNetzbMATHGoogle Scholar
  35. Outrata, J., Kocvara, M., and Zowe, J. (1998). Nonsmooth Approach to Optimization Problems with Equilibrium Constraints. Theory, Applications and Numerical Results, volume 28 of Nonconvex Optimization and its Applications. Kluwer Academic Publishers, Dordrecht.Google Scholar
  36. Polak, E., Mayne, D. Q., and Wardi, Y. (1983). On the extension of constrained optimization algorithms from differentiable to nondifferentiable problems. SIAM Journal on Optimal Control and Optimization, 21: 179–203.MathSciNetzbMATHCrossRefGoogle Scholar
  37. Rockafellar, R. T. (1976). Monotone operators and the proximal point algo- rithm. SIAM Journal on Optimal Control and Optimization, 14: 877–898.MathSciNetzbMATHCrossRefGoogle Scholar
  38. Schramm, H. and Zowe, J. (1992). A version of the bundle idea for minimizing a nonsmooth functions: Conceptual idea, convergence analysis, numerical results. SIAM Journal on Optimization, 2: 121–152.MathSciNetzbMATHCrossRefGoogle Scholar
  39. Shor, N. Z. (1985). Minimization Methods for Non-differentiable Functions. Springer-Verlag, Berlin.zbMATHCrossRefGoogle Scholar
  40. Strodiot, J.-J., Nguyen, V. H., and Heukemes, N. (1983). e-optimal solutions in nondifferentiable convex programming and some related questions. Mathematical Programming, 25: 307–328.Google Scholar
  41. Tarasov, V. N. and Popova, N. K. (1984). A modification of the cutting-plane method with accelerated convergence. In Demyanov, V. F. and Pallaschke, D., editors, Nondifferentiable Optimization: Motivations and Applications, pages 284–290, Berlin. Springer-Verlag.Google Scholar
  42. Uryas’ev, S. P. (1991). New variable metric algorithms for nondifferentiable optimization problems. Journal of Optimization Theory and Applications, 71: 359–388.MathSciNetCrossRefGoogle Scholar
  43. Wolfe, P. (1975). A method of conjugate subgradients for minimizing nondifferentiable functions. Mathematical Programming Study, 3: 145–173.MathSciNetCrossRefGoogle Scholar
  44. Yosida, K. (1964). Functional Analysis. Springer-Verlag.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1999

Authors and Affiliations

  • Jaroslav Haslinger
    • 1
  • Markku Miettinen
    • 2
  • Panagiotis D. Panagiotopoulos
    • 3
  1. 1.Charles UniversityCzech Republic
  2. 2.University of JyväskyläFinland
  3. 3.Aristotle UniversityGreece

Personalised recommendations