Skip to main content
SpringerLink
Log in

Nonsmooth Optimization Methods

  • Chapter
  • First Online:
Partitional Clustering via Nonsmooth Optimization

Part of the book series: Unsupervised and Semi-Supervised Learning ((UNSESUL))

  • 775 Accesses

Abstract

This chapter is devoted to the description of some general methods of nonsmooth optimization. The first two—the standard subgradient and the proximal bundle methods—form the basis for numerical nonsmooth optimization. In addition, we will focus on the methods that will be utilized in the clustering algorithms given in Part II of this book. They are the limited memory bundle method, DC diagonal bundle method, nonsmooth DC method, DC algorithm, discrete gradient method, and the method based on the smoothing techniques. For each of these methods we present a flowchart, give some clarifying explanations, and study convergence properties.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer, Berlin (2017)

    MATH  Google Scholar 

  2. Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137, 317–334 (2008)

    MathSciNet  MATH  Google Scholar 

  3. Bagirov, A.M., Al Nuaimat, A., Sultanova, N.: Hyberpolic smoothing function methodfor minimax problems. Optimization 62(6), 759–784 (2013)

    MathSciNet  MATH  Google Scholar 

  4. Bagirov, A.M., Taheri, S., Ugon, J.: Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems. Pattern Recogn. 53, 12–24 (2016)

    MATH  Google Scholar 

  5. Bertsekas., D.P.: Convex Optimization Algorithms, 2nd edn. Athena Scientific, Belmont, MA (2015)

    Google Scholar 

  6. Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)

    MathSciNet  MATH  Google Scholar 

  7. Gaudioso, M., Gorgone, E.: Gradient set splitting in nonconvex nonsmooth numerical optimization. Optim. Methods Softw. 25, 59–74 (2010)

    MathSciNet  MATH  Google Scholar 

  8. Haarala, M.: Large-Scale Nonsmooth Optimization: Variable Metric Bundle Method with Limited Memory. PhD thesis, Department of Mathematical Information Technology, University of Jyväskylä (2004)

    Google Scholar 

  9. Haarala, N., Miettinen, K., Mäkelä, M.M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109(1), 181–205 (2007)

    MathSciNet  MATH  Google Scholar 

  10. Hiriart-Urruty, J.-B., Lemarechal, C.: Convex Analysis and Minimization Algorithms I and II. Springer, Heidelberg (1993)

    MATH  Google Scholar 

  11. Karmitsa, N.: Diagonal bundle method for nonsmooth sparse optimization. J. Optim. Theory Appl. 166(3), 889–905 (2015)

    MathSciNet  MATH  Google Scholar 

  12. Karmitsa, N., Mäkelä, M.M, Ali, M.M.: Limited memory interior point bundle method for large inequality constrained nonsmooth minimization. Appl. Math. Comput. 198(1), 382–400 (2008)

    MathSciNet  MATH  Google Scholar 

  13. Karmitsa, N., Bagirov, A.M., Mäkelä, M. M.: Comparing different nonsmooth optimization methods and software. Optim. Methods Softw. 27(1), 131–153 (2012)

    MathSciNet  MATH  Google Scholar 

  14. Karmitsa, N., Bagirov, A.M., Taheri, S.: New diagonal bundle method for clustering problems in large data sets. Eur. J. Oper. Res. 263(2), 367–379 (2017)

    MathSciNet  MATH  Google Scholar 

  15. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics, vol. 1133. Springer, Berlin (1985)

    MATH  Google Scholar 

  16. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46(1–3), 105–122 (1990)

    MathSciNet  MATH  Google Scholar 

  17. Kiwiel, K.C.: Improved convergence result for the discrete gradient and secant methods for nonsmooth optimization. J. Optim. Theory Appl. 144(1), 69–75 (2010)

    MathSciNet  MATH  Google Scholar 

  18. Kogan, J.: Introduction to Clustering Large and High-Dimensional Data. Cambridge University Press, Cambridge (2007)

    MATH  Google Scholar 

  19. Lemaréchal, C., Strodiot, J.-J., Bihain, A.: On a bundle algorithm for nonsmooth optimization. In: Mangasarian, O.L., Mayer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, pp. 245–281. Academic Press, New York (1981)

    Google Scholar 

  20. Le Thi, H.A., Pham Dinh, T.: Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. J. Glob. Optim. 11(3), 253–285 (1997)

    Google Scholar 

  21. Le Thi, H.A., Pham Dinh, T.: The DC (differnece of convex functions) programming and DCA revised with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1–4), 23–46 (2005)

    MathSciNet  MATH  Google Scholar 

  22. Lukšan, L., Vlček, J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optim. Theory Appl. 102(3), 593–613 (1999)

    MathSciNet  MATH  Google Scholar 

  23. Mäkelä., M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17(1), 1–29 (2002)

    MathSciNet  MATH  Google Scholar 

  24. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific, Singapore (1992)

    MATH  Google Scholar 

  25. Mifflin, R.: A modification and an extension of Lemaréchal’s algorithm for nonsmooth minimization. Math. Program. Stud. 17, 77–90 (1982)

    MATH  Google Scholar 

  26. Robinson, S.M.: Linear convergence of epsilon-subgradient descent methods for a class of convex functions. Math. Program. 86(1), 41–50 (1999)

    MathSciNet  MATH  Google Scholar 

  27. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2(1), 121–152 (1992)

    MathSciNet  MATH  Google Scholar 

  28. Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer, Berlin (1985)

    MATH  Google Scholar 

  29. Sriperumbudur, B.K., Lanckriet, R.G.: On the convergence of the concave-convex procedure. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Proceedings of the 22nd International Conference on Neural Information Processing Systems, pp. 1759–1767. Curran Associates Inc., Red Hook (2009)

    Google Scholar 

  30. Tao, P.D.: Duality in d.c. (difference of convex functions) optimization. Subgradient methods. In: Hoffmann, K.H., et al. (ed.) Trends in Mathematical Optimization. International Series of Numer Math., vol. 84. Birkhauser, Basel (1988)

    Google Scholar 

  31. Torzcon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)

    MathSciNet  Google Scholar 

  32. Uryasév, S.P.: New variable metric algorithms for nondifferentiable optimization problems. J. Optim. Theory Appl. 71(2), 359–388 (1991)

    MathSciNet  Google Scholar 

  33. Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 111(2), 407–430 (2001)

    MathSciNet  MATH  Google Scholar 

  34. Wolfe, P.: A method of conjugate subgradients for minimizing nondifferentiable functions. In: Balinski, M.L., Wolfe, P. (eds.) Nondifferentiable Optimization, pp. 145–173. Springer, Heidelberg (1975)

    MATH  Google Scholar 

  35. Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15(4), 915–936 (2003)

    MATH  Google Scholar 

  36. Haarala, M., Miettinen, K., Mäkelä, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19(6), 673–692 (2004)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Science, Engineering & Information Technology, Federation University Australia, Ballarat, VIC, Australia

    Adil M. Bagirov & Sona Taheri

  2. Department of Mathematics and Statistics, University of Turku, Turku, Finland

    Napsu Karmitsa

Authors
  1. Adil M. Bagirov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Napsu Karmitsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sona Taheri
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

M. Bagirov, A., Karmitsa, N., Taheri, S. (2020). Nonsmooth Optimization Methods. In: Partitional Clustering via Nonsmooth Optimization. Unsupervised and Semi-Supervised Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-37826-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37826-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37825-7

  • Online ISBN: 978-3-030-37826-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation