A Novel Recursive Solution to LS-SVR for Robust Identification of Dynamical Systems

  • José Daniel A. SantosEmail author
  • Guilherme A. Barreto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9375)


Least Squares Support Vector Regression (LS-SVR) is a powerful kernel-based learning tool for regression problems. However, since it is based on the ordinary least squares (OLS) approach for parameter estimation, the standard LS-SVR model is very sensitive to outliers. Robust variants of the LS-SVR model, such as the WLS-SVR and IRLS-SVR models, have been developed aiming at adding robustness to the parameter estimation process, but they still rely on OLS solutions. In this paper we propose a totally different approach to robustify the LS-SVR. Unlike previous models, we maintain the original LS-SVR loss function, while the solution of the resulting linear system for parameter estimation is obtained by means of the Recursive Least M-estimate (RLM) algorithm. We evaluate the proposed approach in nonlinear system identification tasks, using artificial and real-world datasets contaminated with outliers. The obtained results for infinite-steps-ahead prediction shows that proposed model consistently outperforms the WLS-SVR and IRLS-SVR models for all studied scenarios.


Nonlinear regression Outliers LS-SVR System identification M-estimation 



The authors thank the financial support of IFCE, NUTEC and CNPq (grant no. 309841/2012-7).


  1. 1.
    Suykens, J.A.K., De Brabanter, J., Lukas, L., Vandewalle, J.: Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1), 85–105 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    De Brabanter, K., Pelckmans, K., De Brabanter, J., Debruyne, M., Suykens, J.A.K., Hubert, M., De Moor, B.: Robustness of Kernel based regression: a comparison of iterative weighting schemes. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part I. LNCS, vol. 5768, pp. 100–110. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  3. 3.
    Zou, Y., Chan, S., Ng, T.: A recursive least \(m\)-estimate (RLM) adaptive filter for robust filtering in impulse noise. IEEE Signal Proccess. Lett. 7(11), 324–326 (2000)CrossRefGoogle Scholar
  4. 4.
    Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In:Proceedings of the 15th International Conference on Machine Learning (ICML 1998), pp. 515–521. Morgan Kaufmann (1998)Google Scholar
  5. 5.
    Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRefzbMATHGoogle Scholar
  6. 6.
    Van Gestel, T., Suykens, J.A., Baestaens, D.E., Lambrechts, A., Lanckriet, G., Vandaele, B., De Moor, B., Vandewalle, J.: Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans. Neural Netw. 12(4), 809–821 (2001)CrossRefGoogle Scholar
  7. 7.
    Khalil, H.M., El-Bardini, M.: Implementation of speed controller for rotary hydraulic motor based on LS-SVM. Expert Syst. Appl. 38(11), 14249–14256 (2011)Google Scholar
  8. 8.
    Falck, T., Suykens, J.A., De Moor, B.: Robustness analysis for least squares kernel based regression: an optimization approach. In: Proceedings of the 48th IEEE Conference on Decision and Control (CDC 2009), pp. 6774–6779 (2009)Google Scholar
  9. 9.
    Huber, P.J., et al.: Robust estimation of a location parameter. Ann. Math. Stat. 35(1), 73–101 (1964)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with gaussian processes. Math. Comput. Model. Dyn. Syst. 11(4), 411–424 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Narendra, K.S., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1(1), 4–27 (1990)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • José Daniel A. Santos
    • 1
    Email author
  • Guilherme A. Barreto
    • 2
  1. 1.Department of IndustryFederal Institute of Education, Science and Technology of CearáMaracanaúBrazil
  2. 2.Department of Teleinformatics Engineering, Center of TechnologyFederal University of CearáFortalezaBrazil

Personalised recommendations