An effective adaptive trust region algorithm for nonsmooth minimization

Article
First Online:
Received:
  • 31 Downloads

Abstract

In this paper, an adaptive trust region algorithm that uses Moreau–Yosida regularization is proposed for solving nonsmooth unconstrained optimization problems. The proposed algorithm combines a modified secant equation with the BFGS update formula and an adaptive trust region radius, and the new trust region radius utilizes not only the function information but also the gradient information. The global convergence and the local superlinear convergence of the proposed algorithm are proven under suitable conditions. Finally, the preliminary results from comparing the proposed algorithm with some existing algorithms using numerical experiments reveal that the proposed algorithm is quite promising for solving nonsmooth unconstrained optimization problems.

Keywords

Nonsmooth problems Moreau–Yosida regularization Trust region algorithm Global convergence Superlinear convergence 

Mathematical subject classification

65K05 90C26 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The authors would like to thank the referees and the editor for their valuable comments that greatly improved this paper. This work is supported by the National Natural Science Foundation of China (11661009 and 11261006), the Guangxi Science Fund for Distinguished Young Scholars (No. 2015GXNSFGA139001), the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046), and the Basic Ability Promotion Project of Guangxi Young and Middle-aged Teachers (No. 2017KY0019).

References

  1. 1.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing, Singapore (1992)CrossRefMATHGoogle Scholar
  2. 2.
    Bagirov, A., Karmitsa, N., Mäkelä, M.M.: Introduction to Nonsmooth Optimization: Theory, Practice and Software. Springer, Berlin (2014)CrossRefMATHGoogle Scholar
  3. 3.
    Li, J., Li, X., Yang, B., Sun, X.: Segmentation-based image copy-move forgery detection scheme. IEEE Trans. Inf. Forensics Secur. 10, 507–518 (2015)CrossRefGoogle Scholar
  4. 4.
    Luks̆an, L., Vlc̆ek, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83, 373–391 (1998)MathSciNetGoogle Scholar
  5. 5.
    Mäkelä, M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17, 1–29 (2002)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer, Berlin (2012)Google Scholar
  7. 7.
    Polak, E., Royset, J.O.: Algorithms for finite and semi-infinite min-max-min problems using adaptive smoothing techniques. J. Optim. Theory Appl. 119, 421–457 (2003)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Grapiglia, G.N., Yuan, J., Yuan, Y.: A derivative-free trust-region algorithm for composite nonsmooth optimization. Comput. Appl. Math. 32, 1–25 (2014)MATHGoogle Scholar
  9. 9.
    Fukushima, M., Qi, L.: A globally and superlinearly convergent algorithm for nonsmooth convex minimization. SIAM J. Optim. 6, 1106–1120 (1996)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Sagara, N., Fukushima, M.: A trust region method for nonsmooth convex optimization. J. Ind. Manag. Optim. 1, 171–180 (2005)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Zhang, L.: A new trust region algorithm for nonsmooth convex minimization. Appl. Math. Comput. 193, 135–142 (2007)MathSciNetMATHGoogle Scholar
  12. 12.
    Yuan, G., Wei, Z.: The Barzilai and Borwein gradient method with nonmonotone line search for nonsmooth convex optimization problems. Math. Model. Anal. 17, 203–216 (2012)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Yuan, G., Wei, Z., Wang, Z.: Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization. Comput. Optim. Appl. 54, 45–64 (2013)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Cui, Z., Yuan, G., Sheng, Z., Liu, W., Wang, X., Duan, X.: A modified BFGS formula using a trust region model for nonsmooth convex minimizations. PLoS ONE 10, 1–15 (2015)Google Scholar
  15. 15.
    Ou, Y., Lin, H.: An ODE-like nonmonotone method for nonsmooth convex optimization. J. Appl. Math. Comput. 52, 1–21 (2015)MathSciNetGoogle Scholar
  16. 16.
    Yuan, G., Meng, Z., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129–152 (2016)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Yuan, G., Sheng, Z., Liu, W.: The modified HZ conjugate gradient algorithm for large-scale nonsmooth optimization. PLoS ONE 11, 1–15 (2016)Google Scholar
  18. 18.
    Yuan, G., Sheng, Z.: Nonsmooth Optimization Algorithms. Press of Science, Beijing (2017)Google Scholar
  19. 19.
    Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Akbari, Z., Yousefpour, R., Peyghami, M.R.: A new nonsmooth trust region algorithm for locally Lipschitz unconstrained optimization problems. J. Optim. Theory Appl. 164, 733–754 (2015)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–65. Academic Press (1970).Google Scholar
  22. 22.
    Zhang, X., Zhang, J., Liao, L.: An adaptive trust region method and its convergence. Sci. China Ser. A Math. 45, 620–631 (2002)MathSciNetMATHGoogle Scholar
  23. 23.
    Cui, Z., Wu, B.: A new modified nonmonotone adaptive trust region method for unconstrained optimization. Comput. Optim. Appl. 53, 795–806 (2012)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Li, G.: A trust region method with automatic determination of the trust region radius. Chin. J. Eng. Math. 23, 843–848 (2006)MathSciNetMATHGoogle Scholar
  25. 25.
    Shi, Z.J., Guo, J.H.: A new trust region method for unconstrained optimization. J. Comput. Appl. Math. 213, 509–520 (2008)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Shi, Z., Wang, S.: Nonmonotone adaptive trust region method. Eur. J. Oper. Res. 208, 28–36 (2011)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Cui, Z., Wu, B.: A new trust region method with adaptive radius for unconstrained optimization. Optim. Methods Softw. 27, 419–429 (2012)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Zhou, Q., Hang, D.: Nonmonotone adaptive trust region method with line search based on new diagonal updating. Appl. Numer. Math. 91, 75–88 (2015)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Kamandi, A., Amini, K., Ahookhosh, M.: An improved adaptive trust-region algorithm. Optim. Lett. 11, 1–15 (2016)MathSciNetMATHGoogle Scholar
  30. 30.
    Esmaeili, H., Kimiaei, M.: A new adaptive trust-region method for system of nonlinear equations. Appl. Math. Model. 38, 3003–3015 (2014)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Kimiaei, M., Esmaeili, H.: A trust-region approach with novel filter adaptive radius for system of nonlinear equations. Numer. Algoritm. 73, 1–18 (2016)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Bangkok (1983)MATHGoogle Scholar
  34. 34.
    Pang, J.S., Qi, L.: A globally convergent Newton method for convex \(SC^1\) minimization problems. J. Optim. Theory Appl. 85, 633–648 (1995)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods, vol. 306 of Grundlehren der Mathematischen Wissenschaften. Springer, Berlin (1993)CrossRefMATHGoogle Scholar
  36. 36.
    Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Fukushima, M.: A descent algorithm for nonsmooth convex optimization. Math. Program. 30, 163–175 (1984)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Auslender, A.: Numerical Methods for Nondifferentiable Convex Optimization. Nonlinear Analysis and Optimization, pp. 102–126. Springer, Berlin (1987)MATHGoogle Scholar
  39. 39.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62, 261–275 (1993)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Rauf, A.I., Fukushima, M.: Globally convergent BFGS method for nonsmooth convex optimization. J. Optim. Theory Appl. 104, 539–558 (2000)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Wei, Z., Li, G., Qi, L.: New quasi-Newton methods for unconstrained optimization problems. Appl. Math. Comput. 175, 1156–1188 (2006)MathSciNetMATHGoogle Scholar
  42. 42.
    Yuan, G., Wei, Z., Li, G.: A modified Polak–Ribière–Polyak conjugate gradient algorithm for nonsmooth convex programs. J. Comput. Appl. Math. 255, 86–96 (2014)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Yuan, G., Wei, Z., Lu, X.: A BFGS trust-region method for nonlinear equations. Computing 92, 317–333 (2011)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Powell, M.J.D.: Convergence properties of a class of minimization algorithms. Nonlinear Program. 2, 1–27 (1975)MATHGoogle Scholar
  45. 45.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. SIAM, Bangkok (2000)CrossRefMATHGoogle Scholar
  46. 46.
    Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Di Pillo, G., Grippo, L., Lucidi, S.: A smooth method for the finite minimax problem. Math. Program. 60, 187–214 (1993)MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Bagirov, A.: A method for minimization of quasidifferentiable functions. Optim. Methods Softw. 17, 31–60 (2002)MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Mathematics and Information ScienceGuangxi UniversityNanningPeople’s Republic of China

Personalised recommendations