Family weak conjugate gradient algorithms and their convergence analysis for nonconvex functions

  • Gonglin Yuan
  • Xiaoliang Wang
  • Zhou ShengEmail author
Original Paper


It is well-known that conjugate gradient algorithms are widely applied in many practical fields, for instance, engineering problems and finance models, as they are straightforward and characterized by a simple structure and low storage. However, challenging problems remain, such as the convergence of the PRP algorithms for nonconvexity under an inexact line search, obtaining a sufficient descent for all conjugate gradient methods, and other theory properties regarding global convergence and the trust region feature for nonconvex functions. This paper studies family conjugate gradient formulas based on the six classic formulas, PRP, HS, CD, FR, LS, and DY, where the family conjugate gradient algorithms have better theory properties than those of the formulas by themselves. Furthermore, this technique of the presented conjugate gradient formulas can be extended to any two-term conjugate gradient formula. This paper designs family conjugate gradient algorithms for nonconvex functions, which have the following features without other conditions: (i) the sufficient descent property holds, (ii) the trust region feature is true, and (iii) the global convergence holds under normal assumptions. Numerical results show that the given conjugate gradient algorithms are competitive with those of normal methods.


Nonconvex functions Sufficient descent Trust region Inexact line search Global convergence 

Mathematics Subject Classification (2010)




The authors would like to thank the editor and the referee for their valuable comments which greatly improve this manuscript.

Funding information

This work was supported by the National Natural Science Fund of China (Grant No. 11661009), the Guangxi Natural Science Fund for Distinguished Young Scholars (No. 2015GXNSFGA139001), and the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046).


  1. 1.
    Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Dai, Y.: Analysis of Conjugate Gradient Methods, Ph.D. Thesis, Institute of Computational Mathe- matics and Scientific/Engineering Computing, Chese Academy of Sciences (1997)Google Scholar
  3. 3.
    Dai, Y.: Convergence properties of the BFGS algoritm. SIAM J. Optim. 13, 693–701 (2003)CrossRefzbMATHGoogle Scholar
  4. 4.
    Dai, Y., Liao, L.: New conjugacy conditions and related nonlinear conjugate gradient methods. App. Math. Optim. 43, 87–101 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Dai, Y., Yuan, Y.: A nonlinear conjugate gradient with a strong global convergence properties. SIAM J. Optim. 10, 177–182 (2000)CrossRefzbMATHGoogle Scholar
  6. 6.
    Dai, Y., Yuan, Y.: Nonlinear conjugate gradient methods, Shanghai Scientific and Technical Publishers (1998)Google Scholar
  7. 7.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Fan, J., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. In: Li, D. (ed.) Proceedings of the 5th International Conference on Optimization: Techniques and Applications (December 2001, Hongkong), pp 786–794 (2001)Google Scholar
  9. 9.
    Fletcher, R.: Practical methods of optimization, 2nd. Wiley, New York (1987)zbMATHGoogle Scholar
  10. 10.
    Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2, 21–42 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Grippo, L., Lucidi, S.: A globally convergent version of the Polak-Ribière-Polyak conjugate gradient method. Math. Program. 78, 375–391 (1997)zbMATHGoogle Scholar
  13. 13.
    Hager, W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Hager, W., Zhang, H.: Algorithm 851: a conjugate gradient method with guaranteed descent. ACM Trans. Math. Soft. 32, 113–137 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hestenes, M.R., Stiefel, E.: Method of conjugate gradient for solving linear equations. J. Res. Nation. Bur. Stand. 49, 409–436 (1952)CrossRefzbMATHGoogle Scholar
  16. 16.
    Levenberg, K.: A method for the solution of certain nonlinear problem in least squares. Quart. Appl. Math. 2, 164–168 (1944)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Li, D.H., Tian, B.S.: N-step quadratic convergence of the MPRP method with a restart strategy. J. Comput. Appl. Math. 235, 4978–4990 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Li, X., Wang, S., Jin, Z., Pham, H.: A conjugate gradient algorithm under Yuan-Wei-Lu line search technique for large-scale minimization optimization models, vol. 2018 (2018)Google Scholar
  19. 19.
    Liu, Y., Storey, C.: Effcient generalized conjugate gradient algorithms part 1: theory. J. Optim. Theo. Appl. 69, 129–137 (1991)CrossRefzbMATHGoogle Scholar
  20. 20.
    Martinet, B.: Régularisation d’inéquations variationelles par approxiamations succcessives. Rev. Fr. Inform. Rech. Oper. 4, 154–158 (1970)Google Scholar
  21. 21.
    Nocedal, J., Wright, S.J.: Numerical optimization. Springer, New York (1999)CrossRefzbMATHGoogle Scholar
  22. 22.
    Polak, E.: The conjugate gradient method in extreme problems. Comput. Math. Mathem. Phy. 9, 94–112 (1969)CrossRefGoogle Scholar
  23. 23.
    Polak, E., Ribière, G.: Note sur la convergence de directions conjugees. Rev. Fran. Inf. Rech. Opérat. 3, 35–43 (1969)zbMATHGoogle Scholar
  24. 24.
    Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, Q.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear programming, vol. 2, pp 1–27. Academic Press, New York (1974)Google Scholar
  25. 25.
    Powell, M.J.D.: Nonconvex minimization calculations and the conjugate gradient method, Lecture Notes in Mathematics, vol. 1066, pp 122–141. Springer, Berlin (1984)Google Scholar
  26. 26.
    Powell, M.J.D.: Convergence properties of algorithm for nonlinear optimization. SIAM Rev. 28, 487–500 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Sheng, Z., Ouyang, A., Liu, L., et al.: A novel parameter estimation method for Muskingum model using new Newton-type trust region algorithm. Math. Probl. Eng. 2014, 1–7 (2014). Art. ID 634852MathSciNetzbMATHGoogle Scholar
  28. 28.
    Sheng, Z., Yuan, G.: An effective adaptive trust region algorithm for nonsmooth minimization. Comput. Optim. Appl. 71, 251–271 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Sheng, Z., Yuan, G., Cui, Z.: A new adaptive trust region algorithm for optimization problems. Acta Math. Scientia. 38B(2), 479–496 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Sheng, Z., Yuan, G., Cui, Z., et al.: An adaptive trust region algorithm for large-residual nonsmooth least squares problems. J. Ind. Manage. Optim. 14, 707–718 (2018)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Wei, Z., Yao, S., Liu, L.: The convergence properties of some new conjugate gradient methods. Appl. Math. Comput. 183, 1341–1350 (2006)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Yuan, G.: Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems. Optim. Let. 3, 11–21 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Yuan, G., Lu, X.: A modified PRP conjugate gradient method. Anna. Operat. Res. 166, 73–90 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Yuan, G., Lu, S., Wei, Z.: A new trust-region method with line search for solving symmetric nonlinear equations. Intern. J. Comput. Math. 88, 2109–2123 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Yuan, G., Lu, X., Wei, Z.: A conjugate gradient method with descent direction for unconstrained optimization. J. Comput. Appl. Math. 233, 519–530 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Yuan, G., Lu, X., Wei, Z.: BFGS trust-region method for symmetric nonlinear equations. J. Compu. Appl. Math. 230, 44–58 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Yuan, G., Meng, Z., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory. Appl. 168, 129–152 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Yuan, G., Sheng, Z., Wang, B., et al.: The global convergence of a modified BFGS method for nonconvex functions. J. Comput. Appl. Math. 327, 274–294 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Yuan, G., Wei, Z., Li, G.: A modified Polak-Ribière-Polyak conjugate gradient algorithm for nonsmooth convex programs. J. Comput. Appl. Math. 255, 86–96 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Yuan, G., Wei, Z., Lu, X.: Global convergence of the BFGS method and the PRP method for general functions under a modified weak Wolfe-Powell line search. Appl. Math. Model. 47, 811–825 (2017)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Yuan, G., Wei, Z., Yang, Y.: The global convergence of the Polak-Ribière-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions. J. Comput. Appl. Math. 362, 262–275 (2019)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Yuan, G., Zhang, M.: A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations. J. Comput. Appl. Math. 286, 186–195 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Yuan, Y.: Analysis on the conjugate gradient method. Optim. Meth. Soft. 2, 19–29 (1993)CrossRefGoogle Scholar
  44. 44.
    Zoutendijk, G.: Nonlinear programming computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, Northholland, Amsterdam, pp 37–86 (1970)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Mathematics and Information ScienceGuangxi UniversityNanningChina
  2. 2.School of Mathematical SciencesDalian University of TechnologyDalianChina
  3. 3.Department of MathematicsNanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations