An Empirical Evaluation of Robust Gaussian Process Models for System Identification

  • César Lincoln C. MattosEmail author
  • José Daniel A. Santos
  • Guilherme A. Barreto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9375)


System identification comprises a number of linear and nonlinear tools for black-box modeling of dynamical systems, with applications in several areas of engineering, control, biology and economy. However, the usual Gaussian noise assumption is not always satisfied, specially if data is corrupted by impulsive noise or outliers. Bearing this in mind, the present paper aims at evaluating how Gaussian Process (GP) models perform in system identification tasks in the presence of outliers. More specifically, we compare the performances of two existing robust GP-based regression models in experiments involving five benchmarking datasets with controlled outlier inclusion. The results indicate that, although still sensitive in some degree to the presence of outliers, the robust models are indeed able to achieve lower prediction errors in corrupted scenarios when compared to conventional GP-based approach.


Robust system identification Gaussian process Approximate Bayesian inference 



The authors thank the financial support of FUNCAP, IFCE, NUTEC and CNPq (grant no. 309841/2012-7).


  1. 1.
    Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning, 1st edn. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  2. 2.
    Williams, C.K.I., Barber, D.: Bayesian classification with Gaussian processes. IEEE Trans. Pattern Anal. 20(12), 1342–1351 (1998)CrossRefGoogle Scholar
  3. 3.
    Lawrence, N.D.: Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in Neural Information Processing Systems, pp. 329–336 (2004)Google Scholar
  4. 4.
    Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. Math. Comput. Model. Dyn. 11(4), 411–424 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Murray-Smith, R., Johansen, T.A., Shorten, R.: On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures. In: European Control Conference (ECC 1999), Karlsruhe, BA-14. Springer (1999)Google Scholar
  6. 6.
    Solak, E., Murray-Smith, R., Leithead, W.E., Leith, D.J., Rasmussen, C.E.: Derivative observations in Gaussian process models of dynamic systems. In: Advances in Neural Information Processing Systems 16 (2003)Google Scholar
  7. 7.
    Rottmann, A., Burgard, W.: Learning non-stationary system dynamics online using Gaussian processes. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 192–201. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  8. 8.
    Ažman, K., Kocijan, J.: Dynamical systems identification using Gaussian process models with incorporated local models. Eng. Appl. Artif. Intel. 24(2), 398–408 (2011)CrossRefGoogle Scholar
  9. 9.
    Petelin, D., Grancharova, A., Kocijan, J.: Evolving Gaussian process models for prediction of ozone concentration in the air. Simul. Model. Pract. Theory 33, 68–80 (2013)CrossRefGoogle Scholar
  10. 10.
    Frigola, R., Chen, Y., Rasmussen, C.: Variational Gaussian process state-space models. In: Advances in Neural Information Processing Systems (NIPS), vol. 27, pp. 3680–3688 (2014)Google Scholar
  11. 11.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)CrossRefzbMATHGoogle Scholar
  12. 12.
    Minka, T.P.: Expectation propagation for approximate Bayesian inference. In: Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI 2001), pp. 362–369. Morgan Kaufmann (2001)Google Scholar
  13. 13.
    Nakayama, H., Arakawa, M., Sasaki, R.: A computational intelligence approach to optimization with unknown objective functions. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 73–80. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  14. 14.
    Kuss, M., Pfingsten, T., Csató, L., Rasmussen, C.E.: Approximate inference for robust Gaussian process regression. Technical report 136, Max Planck Institute for Biological Cybernetics, Tubingen, Germany (2005)Google Scholar
  15. 15.
    Tipping, M.E., Lawrence, N.D.: Variational inference for student-\(t\) models: robust Bayesian interpolation and generalised component analysis. Neurocomputing 69(1), 123–141 (2005)CrossRefGoogle Scholar
  16. 16.
    Jylänki, P., Vanhatalo, J., Vehtari, A.: Robust gaussian process regression with a student-\(t\) likelihood. J. Mach. Learn. Res. 12, 3227–3257 (2011)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Berger, B., Rauscher, F.: Robust Gaussian process modelling for engine calibration. In: Proceedings of the 7th Vienna International Conference on Mathematical Modelling (MATHMOD 2012), pp. 159–164 (2012)Google Scholar
  18. 18.
    Kuss, M.: Gaussian process models for robust regression, classification, and reinforcement learning. Ph.D. thesis, TU Darmstadt (2006)Google Scholar
  19. 19.
    Rasmussen, C.E.: Evaluation of Gaussian processes and other methods for non-linear regression. Ph.D. thesis, University of Toronto, Toronto, Canada (1996)Google Scholar
  20. 20.
    Narendra, K.S., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Networks 1(1), 4–27 (1990)CrossRefGoogle Scholar
  21. 21.
    Majhi, B., Panda, G.: Robust identification of nonlinear complex systems using low complexity ANN and particle swarm optimization technique. Expert Syst. Appl. 38(1), 321–333 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • César Lincoln C. Mattos
    • 1
    Email author
  • José Daniel A. Santos
    • 2
  • Guilherme A. Barreto
    • 1
  1. 1.Department of Teleinformatics Engineering, Center of TechnologyFederal University of CearáFortalezaBrazil
  2. 2.Department of IndustryFederal Institute of Education, Science and Technology of CearáMaracanaúBrazil

Personalised recommendations