Skip to main content
SpringerLink
Log in

A class of globally convergent conjugate gradient methods

  • Published:
Science in China Series A: Mathematics Aims and scope Submit manuscript

Abstract

Conjugate gradient methods are very important ones for solving nonlinear optimization problems, especially for large scale problems. However, unlike quasi-Newton methods, conjugate gradient methods were usually analyzed individually. In this paper, we propose a class of conjugate gradient methods, which can be regarded as some kind of convex combination of the Fletcher-Reeves method and the method proposed by Dai et al. To analyze this class of methods, we introduce some unified tools that concern a general method with the scalar βk having the form of φk/ φ k−1.Consequently, the class of conjugate gradient methods can uniformly be analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Dai, Y. H., Yuan, Y., A nonlinear conjugate gradient method with a strong global convergence property, SIAM Journal on Optimization, 1999, 10(1): 177–182.

    Article  MATH  MathSciNet  Google Scholar 

  2. Daniel, J. W., The conjugate gradient method for linear and nonlinear operator equations, SIAM J. Numer. Anal., 1967, 4: 10–26.

    Article  MATH  MathSciNet  Google Scholar 

  3. Fletcher, R., Practical Methods of Optimization, Vol. 1: Unconstrained Optimization, New York: John Wiley and Sons, 1987.

    Google Scholar 

  4. Fletcher, R., Reeves, C., Function minimization by conjugate gradients, Comput. J., 1964, 7: 149–154.

    Article  MATH  MathSciNet  Google Scholar 

  5. Hestenes, M. R., Stiefel, E. L., Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 1952, 49(5): 409–436.

    MATH  MathSciNet  Google Scholar 

  6. Liu, Y., Storey, C., Efficient generalized conjugate gradient algorithms (Part 1): Theory, Journal of Optimization Theory and Applications, 1991, 69: 129–137.

    Article  MATH  MathSciNet  Google Scholar 

  7. Polak, E., Ribiére, G., Note sur la convergence de directions conjugées, Rev. française Informat Recherche Opérationelle, 1969, 16: 35–43.

    Google Scholar 

  8. Polyak, B. T., The conjugate gradient method in extremem problems, USSR Comp. Math. and Math. Phys., 1969, 9: 94–112.

    Article  Google Scholar 

  9. Shanno, D. F., Conjugate gradient methods with inexact searches, Math. Oper. Res., 1978, 3: 244–256.

    Article  MATH  MathSciNet  Google Scholar 

  10. Powell, M. J. D., Restart procedures of the conjugate gradient method, Math. Program., 1977, 2: 241–254.

    Article  MathSciNet  Google Scholar 

  11. Al-Baali, M., Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 1985, 5: 121–124.

    Article  MATH  MathSciNet  Google Scholar 

  12. Dai, Y. H., Yuan, Y., Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal., 1996, 16(2): 155–164.

    Article  MATH  MathSciNet  Google Scholar 

  13. Dai, Y. H., Yuan, Y., Convergence properties of the conjugate descent method, Advances in Mathematics (in Chinese), 1996, 25(6): 552–562.

    MATH  MathSciNet  Google Scholar 

  14. Dai, Y. H., Yuan, Y., Some properties of a new conjugate gradient method, in Advances in Nonlinear Programming (ed. Yuan, Y.), Boston: Kluwer, 1998, 251–262.

    Google Scholar 

  15. Gilbert, J. C., Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optimization, 1992, 2(1): 21–42.

    Article  MATH  MathSciNet  Google Scholar 

  16. Grippo, L., Lucidi, S., A globally convergent version of the Polak-Ribière conjugate gradient method, Math. Prog., 1997, 78: 375–391.

    MathSciNet  Google Scholar 

  17. Liu, G. H., Han, J. Y., Yin, H. X., Global convergence of the Fletcher-Reeves algorithm with an inexact line search, Appl. Math. J. Chinese Univ., Ser. B, 1995, 10: 75–82.

    Article  MATH  MathSciNet  Google Scholar 

  18. Powell, M. J. D., Nonconvex minimization calculations and the conjugate gradient method, in Lecture Notes in Mathematics (Vol. 1066), Berlin: Springer-Verlag, 1984, 122–141.

    Google Scholar 

  19. Qi, H. D., Han, J. Y., Liu, G. H., A modified Hestenes-Stiefel conjugate gradient algorithm, Chinese Annals of Mathematics (in Chinese), 1996, 17A(3): 277–284.

    MathSciNet  Google Scholar 

  20. Touati-Ahmed, D., Storey, C., Efficient hybrid conjugate gradient techniques, J. Optimization Theory Appl., 1990, 64: 379–397.

    Article  MATH  MathSciNet  Google Scholar 

  21. Broyden, C. G., The convergence of a class of double-rank minimization algorithms (1): General considerations, J. Inst. Math. Appl., 1970, 6: 76–90.

    Article  MATH  MathSciNet  Google Scholar 

  22. Byrd, R., Nocedal, J., Yuan, Y., Global convergence of a class of variable metric algorithms, SIAM J. Numer. Anal., 1987, 4: 1171–1190.

    Article  MathSciNet  Google Scholar 

  23. Peng, J. M., Yuan, Y., Properties of Ψ-function and its application, Chinese Numerical Mathematics (in Chinese), 1994, 16: 102–107.

    MATH  MathSciNet  Google Scholar 

  24. Huang, H. Y., Unified approach to quadratically convergent algorithms for function minimization, J. Optimization Theory and Appl., 1970, 5: 405–423.

    Article  MATH  MathSciNet  Google Scholar 

  25. Wolfe, P., Convergence conditions for ascent methods, SIAM Review, 1969, 11: 226–235.

    Article  MATH  MathSciNet  Google Scholar 

  26. olfe, P., Convergence conditions for ascent methods (II)—Some corrections, SIAM Review, 1969, 13: 185–188.

    Google Scholar 

  27. Zoutendijk, G., Nonlinear programming, computational methods, in Integer and Nonlinear Programming (ed. Abadie, J.), Amsterdam: North-Holland, 1970, 37–86.

    Google Scholar 

  28. Pu, D., Yu, W., On the convergence properties of the DFP algorithms, Annals of Operations Research, 1990, 24: 175–184.

    Article  MATH  MathSciNet  Google Scholar 

  29. Dai, Y. H., Han, J. Y., Liu, G. H. et al., Convergence properties of nonlinear conjugate gradient methods, SIAM Journal on Optimization, 1999, 10(2): 345–358.

    Article  MATH  MathSciNet  Google Scholar 

  30. Hu, Y. F., Storey, C., Global convergence result for conjugate gradient methods, JOTA, 1991, 71(2): 399–405.

    Article  MATH  MathSciNet  Google Scholar 

  31. Dai, Y. H., New properties of a nonlinear conjugate gradient method, Numerische Mathematik, 2001, 89: 83–98.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and System Sciences, Chinese Academy of Sciences, 100080, Beijing, China

    Yuhong Dai & Yaxiang Yuan

Authors
  1. Yuhong Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaxiang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yuhong Dai or Yaxiang Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dai, Y., Yuan, Y. A class of globally convergent conjugate gradient methods. Sci. China Ser. A-Math. 46, 251–261 (2003). https://doi.org/10.1360/03ys9027

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1360/03ys9027

Keywords

Search

Navigation