A Novel Recursive Solution to LS-SVR for Robust Identification of Dynamical Systems
Least Squares Support Vector Regression (LS-SVR) is a powerful kernel-based learning tool for regression problems. However, since it is based on the ordinary least squares (OLS) approach for parameter estimation, the standard LS-SVR model is very sensitive to outliers. Robust variants of the LS-SVR model, such as the WLS-SVR and IRLS-SVR models, have been developed aiming at adding robustness to the parameter estimation process, but they still rely on OLS solutions. In this paper we propose a totally different approach to robustify the LS-SVR. Unlike previous models, we maintain the original LS-SVR loss function, while the solution of the resulting linear system for parameter estimation is obtained by means of the Recursive Least M-estimate (RLM) algorithm. We evaluate the proposed approach in nonlinear system identification tasks, using artificial and real-world datasets contaminated with outliers. The obtained results for infinite-steps-ahead prediction shows that proposed model consistently outperforms the WLS-SVR and IRLS-SVR models for all studied scenarios.
KeywordsNonlinear regression Outliers LS-SVR System identification M-estimation
The authors thank the financial support of IFCE, NUTEC and CNPq (grant no. 309841/2012-7).
- 2.De Brabanter, K., Pelckmans, K., De Brabanter, J., Debruyne, M., Suykens, J.A.K., Hubert, M., De Moor, B.: Robustness of Kernel based regression: a comparison of iterative weighting schemes. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part I. LNCS, vol. 5768, pp. 100–110. Springer, Heidelberg (2009) CrossRefGoogle Scholar
- 4.Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In:Proceedings of the 15th International Conference on Machine Learning (ICML 1998), pp. 515–521. Morgan Kaufmann (1998)Google Scholar
- 7.Khalil, H.M., El-Bardini, M.: Implementation of speed controller for rotary hydraulic motor based on LS-SVM. Expert Syst. Appl. 38(11), 14249–14256 (2011)Google Scholar
- 8.Falck, T., Suykens, J.A., De Moor, B.: Robustness analysis for least squares kernel based regression: an optimization approach. In: Proceedings of the 48th IEEE Conference on Decision and Control (CDC 2009), pp. 6774–6779 (2009)Google Scholar