Fast Robust Learning Algorithm Dedicated to LMLS Criterion

  • Andrzej Rusiecki
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6114)


Robust neural network learning algorithms are often applied to deal with the problem of gross errors and outliers. Unfortunately, such methods suffer from high computational complexity, which makes them ineffective. In this paper, we propose a new robust learning algorithm based on the LMLS (Least Mean Log Squares) error criterion. It can be considered, as a good trade-off between robustness to outliers and learning efficiency. As it was experimentally demonstrated, the novel method is not only faster but also more robust than the LMLS algorithm. Results of implementation and simulation of nets trained with the new algorithm, the traditional backpropagation (BP) algorithm and robust LMLS method are presented and compared.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, D.S., Jain, R.C.: A robust back propagation learning algorithm for function approximation. IEEE Transactions on Neural Networks 5, 467–479 (1994)CrossRefGoogle Scholar
  2. 2.
    Chuang, C., Su, S., Hsiao, C.: The Annealing Robust Backpropagation (ARBP) Learning Algorithm. IEEE Transactions on Neural Networks 11, 1067–1076 (2000)CrossRefGoogle Scholar
  3. 3.
    Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publishing, Boston (1996)Google Scholar
  4. 4.
    Hagan, M.T., Menhaj, M.B.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans. on Neural Networks 5(6) (1994)Google Scholar
  5. 5.
    Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics the Approach Based on Influence Functions. John Wiley & Sons, New York (1986)zbMATHGoogle Scholar
  6. 6.
    Haykin, S.: Neural Networks - A Comprehensive Foundation, 2nd edn. Prentice Hall, NJ (1999)zbMATHGoogle Scholar
  7. 7.
    Hornik, K., Stinchconbe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)CrossRefGoogle Scholar
  8. 8.
    Huber, P.J.: Robust Statistics. Wiley, New York (1981)zbMATHCrossRefGoogle Scholar
  9. 9.
    Liano, K.: Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks 7, 246–250 (1996)CrossRefGoogle Scholar
  10. 10.
    Marquardt, D.: An algorithm for least squares estimation of non-linear parameters. J. Soc. Ind. Appl. Math., 431–441 (1963)Google Scholar
  11. 11.
    Olive, D.J., Hawkins, D.M.: Robustifying Robust Estimators, NY (2007)Google Scholar
  12. 12.
    Pernia-Espinoza, A.V., Ordieres-Mere, J.B., Martinez-de-Pison, F.J., Gonzalez-Marcos, A.: TAO-robust backpropagation learning algorithm. Neural Networks 18, 191–204 (2005)CrossRefGoogle Scholar
  13. 13.
    Rusiecki, A.L.: Robust MCD-based backpropagation learning algorithm. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 154–163. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Andrzej Rusiecki
    • 1
  1. 1.Wroclaw University of TechnologyWroclawPoland

Personalised recommendations