Skip to main content
SpringerLink
Log in

An Improved Conjugate Gradient Neural Networks Based on a Generalized Armijo Search Method

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10637))

Included in the following conference series:

  • 4154 Accesses

Abstract

In this paper, by constructing a generalized Armijo search method, a novel conjugate gradient (CG) model has been proposed to training a common three-layer backpropagation (BP) neural network. Compared with the classical gradient descent method, this algorithm efficiently accelerates the convergence speed due to the existence of the additional conjugate direction. Essentially, the optimal learning rate of each epoch is determined by the given inexact line search strategy. The presented model does not significantly increase the computational cost in dealing with real applications. Two benchmark simulations have been performed to illustrate the promising advantages of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Xia, Y., Hu, R.: Fuzzy neural network based energy efficiencies control in the heating energy supply system responding to the changes of user demands. J. Netw. Intell. 2, 186–194 (2017)

    Google Scholar 

  2. Erkaymaz, O., Şenyer, İ., Uzun, R.: Detection of knee abnormality from surface EMG signals by artificial neural networks. In: 2017 25th Signal Processing and Communications Applications Conference (SIU) (2017)

    Google Scholar 

  3. Li, X.D., Rakkiyappan, R.: Impulsive controller design for exponential synchronization of chaotic neural networks with mixed delays. Commun. Nonlinear Sci. Numer. Simul. 18, 1515–1523 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  4. Li, X.D., Rakkiyappan, R., Velmurugan, G.: Dissipativity analysis of memristor-based complex-valued neural networks with time-varying delays. Inf. Sci. 294, 645–665 (2015)

    Article  MATH  MathSciNet  Google Scholar 

  5. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  MATH  Google Scholar 

  6. Park, D.C., Elsharkawi, M.A., Marks, R.J., Atlas, L.E., Damborg, M.J.: Electric-load forecasting using an artificial neural networks. IEEE Trans. Power Syst. 6, 442–449 (1991)

    Article  Google Scholar 

  7. Saini, L.M., Soni, M.K.: Artificial neural network-based peak load forecasting using conjugate gradient methods. IEEE Trans. Power Syst. 17, 907–912 (2002)

    Article  Google Scholar 

  8. Luo, B., Wu, H.N., Li, H.X.: Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 26, 684–696 (2015)

    Article  MathSciNet  Google Scholar 

  9. Luo, B., Liu, D.R., Wu, H.N., Wang, D., Lewis, F.L.: Policy gradient adaptive dynamic programming for data-based optimal control. IEEE Trans. Cybern. 99, 1–14 (2016)

    Article  Google Scholar 

  10. Zhang, H.S., Tang, Y.L.: Online gradient method with smoothing \(\ell _0\) regularization for feedforward neural networks. Neurocomputing 224, 1–8 (2017)

    Article  Google Scholar 

  11. Lu, C.N., Wu, H.T., Vemuri, S.: Neural network based short-term load forecasting. IEEE Trans. Power Syst. 8, 336–342 (1993)

    Article  Google Scholar 

  12. Papalexopoulos, A.D., Hao, S.Y., Peng, T.M.: An implementation of a neural-network-based load forecasting-model for the EMS. IEEE Trans. Power Syst. 9, 1956–1962 (1994)

    Article  Google Scholar 

  13. Zhang, H.S., Mandic, D.P.: Is a complex-valued stepsize advantageous in complex-valued gradient learning algorithms? IEEE Trans. Neural Netw. Learn. Syst. 27, 2730–2735 (2016)

    Article  MathSciNet  Google Scholar 

  14. Goodband, J.H., Haas, O.C.L., Mills, J.A.: A comparison of neural network approaches for on-line prediction in IGRT. Med. Phys. 35, 1113–1122 (2008)

    Article  Google Scholar 

  15. Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publisher, Boston (1996)

    Google Scholar 

  16. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006). doi:10.1007/978-0-387-40065-5

    MATH  Google Scholar 

  17. Hestenes, M.R., Stiefel, E.L.: Method of conjugate gradients for solving linear systems. National Bureau of Standards, Washington (1952)

    Google Scholar 

  18. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    Article  MATH  MathSciNet  Google Scholar 

  19. Dai, Y.H., Yuan, Y.X.: Nonlinear Conjugate Gradient Methods. Shanghai Scientific, Technical Publishers, Shanghai (2000)

    Google Scholar 

  20. Gonzalez, A., Dorronsoro, J.R.: Natural conjugate gradient training of multilayer perceptrons. Neurocomputing 71, 2499–2506 (2008)

    Article  Google Scholar 

  21. Sun, Q.Y., Liu, X.H.: Global convergence results of a new three terms conjugate gradient method with generalized Armijo step size rule. Mathematica Numerica Sinica 26, 25–36 (2004)

    MathSciNet  Google Scholar 

  22. Polak, E., Ribiere, G.: Note sur la convergence de directions conjugates. Revue Francaise d’Informatique et de Recherche Operationnelle 16, 94–112 (1969)

    MATH  Google Scholar 

  23. Polyak, B.T.: The conjugate gradient method in extremal problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  MATH  Google Scholar 

  24. Shen, X.Z., Shi, X.Z., Meng, G.: Online algorithm of blind source separation based on conjugate gradient method. Circuits Syst. Sig. Process. 25, 381–388 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  25. Wang, J., Wu, W., Zurada, J.M.: Deterministic convergence of conjugate gradient method for feedforward neural networks. Neurocomputing 74, 2368–2376 (2011)

    Article  Google Scholar 

  26. Magoulas, G.D., Vrahatis, M.N., Androulakis, G.S.: Effective backpropagation training with variable stepsize. Neural Netw. 10, 69–82 (1997)

    Article  MATH  Google Scholar 

  27. Orozco-Henao, C., Bretas, A.S., Chouhy-Leborgne, R., Herrera-Orozco, A.R., Marín-Quintero, J.: Active distribution network fault location methodology: a minimum fault reactance and Fibonacci search approach. Electr. Power Energy Syst. 84, 232–241 (2017)

    Article  Google Scholar 

  28. Vieira, D.A.G., Lisboa, A.C.: Line search methods with guaranteed asymptotical convergence to an improving local optimum of multimodal functions. Eur. J. Oper. Res. 235, 38–46 (2014)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (No. 61305075, 11401185), the China Postdoctoral Science Foundation (No. 2012M520624), the Natural Science Foundation of Shandong Province (Nos. ZR2013FQ004, ZR2013DM015, ZR2015AL014), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20130133120014) and the Fundamental Research Funds for the Central Universities (Nos. 13CX05016A, 14CX05042A, 15CX05053A, 15CX08011A, 15CX02064A).

Author information

Authors and Affiliations

  1. College of Science, China University of Petroleum, Qingdao, 266580, China

    Bingjie Zhang, Tao Gao & Jian Wang

  2. College of Mathematics and Statistics, Hengyang Normal University, Hengyang, 421008, China

    Long Li

  3. Shandong Provincial Key Laboratory of Computer Networks, Shandong Computer Science Center (National Supercomputer Center in Jinan), Jinan, 250014, Shandong, China

    Zhanquan Sun

Authors
  1. Bingjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Long Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanquan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Wang .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, B., Gao, T., Li, L., Sun, Z., Wang, J. (2017). An Improved Conjugate Gradient Neural Networks Based on a Generalized Armijo Search Method. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10637. Springer, Cham. https://doi.org/10.1007/978-3-319-70093-9_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70093-9_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70092-2

  • Online ISBN: 978-3-319-70093-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation