Neural Processing Letters

, Volume 22, Issue 2, pp 163–169 | Cite as

A Method of Accelerating Neural Network Learning

  • Sotir Sotirov


The article presents of accelerating neural network learning by the Back Propagation algorithm and one of its fastest modifications – the Levenberg–Marqurdt method. The learning is accelerated by introducing the ‘single-direction’ coefficient of the change of x for calculating its new values (the number of iterations is decreased by approximately 30%). Simulation results of learning neural networks by applying both the classic method and the method of accelerating the procedure are presented.


Back Propagation Levenberg–Marqurdt method of learning neural networks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bishop, C. M.: Neural Networks for Pattern Recognition, Oxford university press, ISBN 0 19 853864 2, 2000.Google Scholar
  2. 2.
    Haykin, S. Neural Networks: A Comprehensive Foundation, Prentice Hall, New Jersey, 1999.Google Scholar
  3. 3.
    Webros, J. P. 1974Beyond regression: New tool for prediction and analysis in behavioral scienceHarvard UniversityCambridge, MAPh.D. ThesisGoogle Scholar
  4. 4.
    Parker, D. B. 1985Learning-logic: Casting the cortex of the human brain in silicon, Technical report TR-47, Center for computational research in Economics and Management ScienceMITCambridge, MAGoogle Scholar
  5. 5.
    Hagan, M. T., Demuth, H. B., Beale, M. 1996Neural Network DesignPWS Publishing CompanyBostonGoogle Scholar
  6. 6.
    Rumelhart, D. E., Hinton, G. E., Williams, R. J. 1986Learning representation by back-propagation errorsNature323533536CrossRefGoogle Scholar
  7. 7.
    Kamarthi, S. V., Pittner, S. 1999Accelerating neural network training using weight extrapolationsNeural Networks1212851299CrossRefPubMedGoogle Scholar
  8. 8.
    Meybodi, M.R., Beigy, H. 2002New learning automata based algorithms for adaptation of backpropagation algorithm parametersInternational Journal of Neural Systems124567PubMedGoogle Scholar
  9. 9.
    Bates, D. M., Watts, D. G. 1988Nonlinear Regression and Its ApplicationsWileyNew YorkGoogle Scholar
  10. 10.
    Gill, P. R., Murray, W., Wright, M.H. 1981The Levenberg–Marquardt Method §4. 7.3 in Practical OptimizationAcademic PressLondon136137Google Scholar
  11. 11.
    Levenberg, K. 1944A method for the solution of certain problems in least squaresQuarterly Applied Mathematics2164168Google Scholar
  12. 12.
    Marquardt, D. 1963An algorithm for least-squares estimation of nonlinear parametersSIAM Journal of Applied Mathematics11431441CrossRefGoogle Scholar
  13. 13.
    Tai-cong, C., Da-jian H., Au, F. T. K. and Tham, L.G.: Acceleration of Levenberg– Marquardt training of neural networks with variable decay rate, Proceedings of the International Joint Conference on Neural Networks, 3 (2003) pp. 1873–1878.Google Scholar
  14. 14.
    Bogdan, M. W., Serdar, I., Okyay, K., Önder Efe, M.: An Algorithm for Fast Convergence in Training Neural Networks, International Joint Conference on Neural Networks (IJCNN01), Washington D.C, July 15–19, 2001.Google Scholar

Copyright information

© Springer 2005

Authors and Affiliations

  1. 1.Department of Computer TechnologiesUniversity “Prof. D-R Asen Zlatarov”BourgasBulgaria

Personalised recommendations