A class of globally convergent conjugate gradient methods

  • Yuhong Dai
  • Yaxiang Yuan


Conjugate gradient methods are very important ones for solving nonlinear optimization problems, especially for large scale problems. However, unlike quasi-Newton methods, conjugate gradient methods were usually analyzed individually. In this paper, we propose a class of conjugate gradient methods, which can be regarded as some kind of convex combination of the Fletcher-Reeves method and the method proposed by Dai et al. To analyze this class of methods, we introduce some unified tools that concern a general method with the scalar βk having the form of φk/ φ k−1.Consequently, the class of conjugate gradient methods can uniformly be analyzed.


unconstrained optimization conjugate gradient line search global convergence 


  1. 1.
    Dai, Y. H., Yuan, Y., A nonlinear conjugate gradient method with a strong global convergence property, SIAM Journal on Optimization, 1999, 10(1): 177–182.MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Daniel, J. W., The conjugate gradient method for linear and nonlinear operator equations, SIAM J. Numer. Anal., 1967, 4: 10–26.MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Fletcher, R., Practical Methods of Optimization, Vol. 1: Unconstrained Optimization, New York: John Wiley and Sons, 1987.Google Scholar
  4. 4.
    Fletcher, R., Reeves, C., Function minimization by conjugate gradients, Comput. J., 1964, 7: 149–154.MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Hestenes, M. R., Stiefel, E. L., Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 1952, 49(5): 409–436.MATHMathSciNetGoogle Scholar
  6. 6.
    Liu, Y., Storey, C., Efficient generalized conjugate gradient algorithms (Part 1): Theory, Journal of Optimization Theory and Applications, 1991, 69: 129–137.MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Polak, E., Ribiére, G., Note sur la convergence de directions conjugées, Rev. française Informat Recherche Opérationelle, 1969, 16: 35–43.Google Scholar
  8. 8.
    Polyak, B. T., The conjugate gradient method in extremem problems, USSR Comp. Math. and Math. Phys., 1969, 9: 94–112.CrossRefGoogle Scholar
  9. 9.
    Shanno, D. F., Conjugate gradient methods with inexact searches, Math. Oper. Res., 1978, 3: 244–256.MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Powell, M. J. D., Restart procedures of the conjugate gradient method, Math. Program., 1977, 2: 241–254.CrossRefMathSciNetGoogle Scholar
  11. 11.
    Al-Baali, M., Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 1985, 5: 121–124.MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Dai, Y. H., Yuan, Y., Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal., 1996, 16(2): 155–164.MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Dai, Y. H., Yuan, Y., Convergence properties of the conjugate descent method, Advances in Mathematics (in Chinese), 1996, 25(6): 552–562.MATHMathSciNetGoogle Scholar
  14. 14.
    Dai, Y. H., Yuan, Y., Some properties of a new conjugate gradient method, in Advances in Nonlinear Programming (ed. Yuan, Y.), Boston: Kluwer, 1998, 251–262.Google Scholar
  15. 15.
    Gilbert, J. C., Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optimization, 1992, 2(1): 21–42.MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Grippo, L., Lucidi, S., A globally convergent version of the Polak-Ribière conjugate gradient method, Math. Prog., 1997, 78: 375–391.MathSciNetGoogle Scholar
  17. 17.
    Liu, G. H., Han, J. Y., Yin, H. X., Global convergence of the Fletcher-Reeves algorithm with an inexact line search, Appl. Math. J. Chinese Univ., Ser. B, 1995, 10: 75–82.MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Powell, M. J. D., Nonconvex minimization calculations and the conjugate gradient method, in Lecture Notes in Mathematics (Vol. 1066), Berlin: Springer-Verlag, 1984, 122–141.Google Scholar
  19. 19.
    Qi, H. D., Han, J. Y., Liu, G. H., A modified Hestenes-Stiefel conjugate gradient algorithm, Chinese Annals of Mathematics (in Chinese), 1996, 17A(3): 277–284.MathSciNetGoogle Scholar
  20. 20.
    Touati-Ahmed, D., Storey, C., Efficient hybrid conjugate gradient techniques, J. Optimization Theory Appl., 1990, 64: 379–397.MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Broyden, C. G., The convergence of a class of double-rank minimization algorithms (1): General considerations, J. Inst. Math. Appl., 1970, 6: 76–90.MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Byrd, R., Nocedal, J., Yuan, Y., Global convergence of a class of variable metric algorithms, SIAM J. Numer. Anal., 1987, 4: 1171–1190.CrossRefMathSciNetGoogle Scholar
  23. 23.
    Peng, J. M., Yuan, Y., Properties of Ψ-function and its application, Chinese Numerical Mathematics (in Chinese), 1994, 16: 102–107.MATHMathSciNetGoogle Scholar
  24. 24.
    Huang, H. Y., Unified approach to quadratically convergent algorithms for function minimization, J. Optimization Theory and Appl., 1970, 5: 405–423.MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Wolfe, P., Convergence conditions for ascent methods, SIAM Review, 1969, 11: 226–235.MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    olfe, P., Convergence conditions for ascent methods (II)—Some corrections, SIAM Review, 1969, 13: 185–188.Google Scholar
  27. 27.
    Zoutendijk, G., Nonlinear programming, computational methods, in Integer and Nonlinear Programming (ed. Abadie, J.), Amsterdam: North-Holland, 1970, 37–86.Google Scholar
  28. 28.
    Pu, D., Yu, W., On the convergence properties of the DFP algorithms, Annals of Operations Research, 1990, 24: 175–184.MATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Dai, Y. H., Han, J. Y., Liu, G. H. et al., Convergence properties of nonlinear conjugate gradient methods, SIAM Journal on Optimization, 1999, 10(2): 345–358.MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Hu, Y. F., Storey, C., Global convergence result for conjugate gradient methods, JOTA, 1991, 71(2): 399–405.MATHCrossRefMathSciNetGoogle Scholar
  31. 31.
    Dai, Y. H., New properties of a nonlinear conjugate gradient method, Numerische Mathematik, 2001, 89: 83–98.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Science in China Press 2003

Authors and Affiliations

  1. 1.State Key Laboratory of Scientific and Engineering ComputingInstitute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and System Sciences, Chinese Academy of SciencesBeijingChina

Personalised recommendations