Skip to main content
SpringerLink
Log in

Subgradient and Bundle Methods for Nonsmooth Optimization

  • Chapter
Numerical Methods for Differential Equations, Optimization, and Technological Problems

Part of the book series: Computational Methods in Applied Sciences ((COMPUTMETHODS,volume 27))

Abstract

The nonsmooth optimization methods can mainly be divided into two groups: subgradient and bundle methods. Usually, when developing new algorithms and testing them, the comparison is made between similar kinds of methods. The goal of this work is to test and compare different bundle and subgradient methods as well as some hybrids of these two and/or some others. The test set included a large amount of different unconstrained nonsmooth minimization problems, e.g., convex and nonconvex problems, piecewise linear and quadratic problems, and problems with different sizes. Rather than foreground some method over the others, our aim is to get some insight on which method is suitable for certain types of problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Äyrämö S (2006) Knowledge mining using robust clustering. PhD thesis, University of Jyväskylä

    Google Scholar 

  2. Bagirov AM, Ganjehlou AN (2010) A quasisecant method for minimizing nonsmooth functions. Optim Methods Softw 25(1):3–18

    Article  MathSciNet  MATH  Google Scholar 

  3. Bagirov AM, Karasözen B, Sezer M (2008) Discrete gradient method: Derivative-free method for nonsmooth optimization. J Optim Theory Appl 137(2):317–334

    Article  MathSciNet  MATH  Google Scholar 

  4. Beck A, Teboulle M (2003) Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper Res Lett 31(3):167–175

    Article  MathSciNet  MATH  Google Scholar 

  5. Ben-Tal A, Nemirovski A (2005) Non-Euclidean restricted memory level method for large-scale convex optimization. Math Program 102(3):407–456

    Article  MathSciNet  MATH  Google Scholar 

  6. Bihain A (1984) Optimization of upper semidifferentiable functions. J Optim Theory Appl 44(4):545–568

    Article  MathSciNet  MATH  Google Scholar 

  7. Bradley PS, Fayyad UM, Mangasarian OL (1999) Mathematical programming for data mining: Formulations and challenges. INFORMS J Comput 11(3):217–238

    Article  MathSciNet  MATH  Google Scholar 

  8. Burke JV, Lewis AS, Overton ML (2005) A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J Optim 15(3):751–779

    Article  MathSciNet  MATH  Google Scholar 

  9. Byrd RH, Nocedal J, Schnabel RB (1994) Representations of quasi-Newton matrices and their use in limited memory methods. Math Program 63(2):129–156

    Article  MathSciNet  MATH  Google Scholar 

  10. Clarke FH (1983) Optimization and nonsmooth analysis. Wiley, New York

    MATH  Google Scholar 

  11. Clarke FH, Ledyaev YS, Stern RJ, Wolenski PR (1998) Nonsmooth analysis and control theory. Springer, New York

    MATH  Google Scholar 

  12. Demyanov VF, Bagirov AM, Rubinov AM (2002) A method of truncated codifferential with application to some problems of cluster analysis. J Glob Optim 23(1):63–80

    Article  MathSciNet  MATH  Google Scholar 

  13. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91(2):201–213

    Article  MathSciNet  MATH  Google Scholar 

  14. Gaudioso M, Monaco MF (1992) Variants to the cutting plane approach for convex nondifferentiable optimization. Optimization 25(1):65–75

    Article  MathSciNet  MATH  Google Scholar 

  15. Haarala M, Miettinen K, Mäkelä MM (2004) New limited memory bundle method for large-scale nonsmooth optimization. Optim Methods Softw 19(6):673–692

    Article  MathSciNet  MATH  Google Scholar 

  16. Haarala N, Miettinen K, Mäkelä MM (2007) Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math Program 109(1):181–205

    Article  MathSciNet  MATH  Google Scholar 

  17. Haslinger J, Neittaanmäki P (1996) Finite element approximation for optimal shape, material and topology design, 2nd edn. Wiley, Chichester

    MATH  Google Scholar 

  18. Hiriart-Urruty J-B, Lemaréchal C (1993) Convex analysis and minimization algorithms. II. Advanced theory and bundle methods. Springer, Berlin

    MATH  Google Scholar 

  19. Kappel F, Kuntsevich AV (2000) An implementation of Shor’s r-algorithm. Comput Optim Appl 15(2):193–205

    Article  MathSciNet  MATH  Google Scholar 

  20. Kärkkäinen T, Heikkola E (2004) Robust formulations for training multilayer perceptrons. Neural Comput 16(4):837–862

    Article  MATH  Google Scholar 

  21. Karmitsa N, Bagirov A, Mäkelä MM (2009) Empirical and theoretical comparisons of several nonsmooth minimization methods and software. TUCS Technical Report 959, Turku Centre for Computer Science, Turku. Available online http://tucs.fi/publications/insight.php?id=tKaBaMa09a

  22. Karmitsa N, Bagirov A, Mäkelä MM (2012) Comparing different nonsmooth minimization methods and software. Optim Methods Softw 27(1):131–153. doi:10.1080/10556788.2010.526116

    Article  MathSciNet  MATH  Google Scholar 

  23. Kiwiel KC (1985) Methods of descent for nondifferentiable optimization. Lecture notes in mathematics, vol 1133. Springer, Berlin

    MATH  Google Scholar 

  24. Kiwiel KC (1990) Proximity control in bundle methods for convex nondifferentiable minimization. Math Program 46(1):105–122

    Article  MathSciNet  MATH  Google Scholar 

  25. Kuntsevich A, Kappel F (1997) SolvOpt – the solver for local nonlinear optimization problems. Graz. http://www.uni-graz.at/imawww/kuntsevich/solvopt/

  26. Lemaréchal C (1989) Nondifferentiable optimization. In: Nemhauser GL, Rinnooy Kan AHG, Todd MJ (eds) Optimization. North-Holland, Amsterdam, pp 529–572

    Chapter  Google Scholar 

  27. Lukšan L (1984) Dual method for solving a special problem of quadratic programming as a subproblem at linearly constrained nonlinear minimax approximation. Kybernetika 20(6):445–457

    MathSciNet  MATH  Google Scholar 

  28. Lukšan L, Tu̇ma M, Šiška M, Vlček J, Ramešová N (2002) Ufo 2002: Interactive system for universal functional optimization. Technical report V-883, Academy of Sciences of the Czech Republic, Prague

    Google Scholar 

  29. Lukšan L, Vlček J (1998) A bundle-Newton method for nonsmooth unconstrained minimization. Math Program 83(3):373–391

    Article  MATH  Google Scholar 

  30. Lukšan L, Vlček J (2000) NDA: Algorithms for nondifferentiable optimization. Technical report V-797, Academy of Sciences of the Czech Republic, Prague

    Google Scholar 

  31. Lukšan L, Vlček J (2000) Test problems for nonsmooth unconstrained and linearly constrained optimization. Technical report V-798, Academy of Sciences of the Czech Republic, Prague

    Google Scholar 

  32. Mäkelä MM (2002) Survey of bundle methods for nonsmooth optimization. Optim Methods Softw 17(1):1–29

    Article  MathSciNet  MATH  Google Scholar 

  33. Mäkelä MM (2003) Multiobjective proximal bundle method for nonconvex nonsmooth optimization: Fortran subroutine MPBNGC 2.0. Reports of the Department of Mathematical Information Technology, Series B, Scientific Computing B13/2003, University of Jyväskylä, Jyväskylä

    Google Scholar 

  34. Mäkelä MM, Miettinen M, Lukšan L, Vlček J (1999) Comparing nonsmooth nonconvex bundle methods in solving hemivariational inequalities. J Glob Optim 14(2):117–135

    Article  MATH  Google Scholar 

  35. Mäkelä MM, Neittaanmäki P (1992) Nonsmooth optimization: Analysis and algorithms with applications to optimal control. World Scientific, River Edge

    MATH  Google Scholar 

  36. Mistakidis ES, Stavroulakis GE (1998) Nonconvex optimization in mechanics. Algorithms, heuristics and engineering applications by the FEM. Kluwer, Dordrecht

    Google Scholar 

  37. Moreau JJ, Panagiotopoulos PD, Strang G (eds) (1988) Topics in nonsmooth mechanics. Birkhäuser, Basel

    MATH  Google Scholar 

  38. Outrata J, Kočvara M, Zowe J (1998) Nonsmooth approach to optimization problems with equilibrium constraints. Theory, applications and numerical results. Kluwer, Dordrecht

    MATH  Google Scholar 

  39. Robinson SM (1999) Linear convergence of epsilon-subgradient descent methods for a class of convex functions. Math Program 86(1):41–50

    Article  MathSciNet  MATH  Google Scholar 

  40. Sagastizábal C, Solodov M (2005) An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J Optim 16(1):146–169

    Article  MathSciNet  MATH  Google Scholar 

  41. Schramm H, Zowe J (1992) A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results. SIAM J Optim 2(1):121–152

    Article  MathSciNet  MATH  Google Scholar 

  42. Shor NZ (1985) Minimization methods for non-differentiable functions. Springer, Berlin

    Book  MATH  Google Scholar 

  43. Uryasev SP (1991) New variable-metric algorithms for nondifferentiable optimization problems. J Optim Theory Appl 71(2):359–388

    Article  MathSciNet  Google Scholar 

  44. Vlček J, Lukšan L (2001) Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J Optim Theory Appl 111(2):407–430

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge professors A. Kuntsevich and F. Kappel for providing Shor’s r-algorithm in their web-page as well as professors L. Lukšan and J. Vlček for providing the bundle-Newton algorithm. The work was financially supported by the University of Turku (Finland) and the University of Ballarat (Australia) and the Australian Research Council.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Turku, 20014, Turku, Finland

    Marko M. Mäkelä & Napsu Karmitsa

  2. Centre for Informatics and Applied Optimization, School of Science, Information Technology and Engineering, University of Ballarat, University Drive, Mount Helen, PO Box 663, Ballarat, VIC, 3353, Australia

    Adil Bagirov

Authors
  1. Marko M. Mäkelä
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Napsu Karmitsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adil Bagirov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marko M. Mäkelä .

Editor information

Editors and Affiliations

  1. Mathematical Information Technology, University of Jyväskylä, Mattilanniemi 2, Jyväskylä, 40100, Finland

    Sergey Repin

  2. Mathematical Information Technology, University of Jyväskylä, Mattilanniemi 2, Jyväskylä, 40100, Finland

    Timo Tiihonen

  3. Mathematical Information Technology, University of Jyväskylä, Mattilanniemi 2, Jyväskylä, 40100, Finland

    Tero Tuovinen

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Mäkelä, M.M., Karmitsa, N., Bagirov, A. (2013). Subgradient and Bundle Methods for Nonsmooth Optimization. In: Repin, S., Tiihonen, T., Tuovinen, T. (eds) Numerical Methods for Differential Equations, Optimization, and Technological Problems. Computational Methods in Applied Sciences, vol 27. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-5288-7_15

Download citation

  • DOI: https://doi.org/10.1007/978-94-007-5288-7_15

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-007-5287-0

  • Online ISBN: 978-94-007-5288-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation