Approximate Inference in Related Multi-output Gaussian Process Regression

  • Ankit ChiplunkarEmail author
  • Emmanuel Rachelson
  • Michele Colombo
  • Joseph Morlier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10163)


In Gaussian Processes a multi-output kernel is a covariance function over correlated outputs. Using a prior known relation between outputs, joint auto- and cross-covariance functions can be constructed. Realizations from these joint-covariance functions give outputs that are consistent with the prior relation. One issue with gaussian process regression is efficient inference when scaling upto large datasets. In this paper we use approximate inference techniques upon multi-output kernels enforcing relationships between outputs. Results of the proposed methodology for theoretical data and real world applications are presented. The main contribution of this paper is the application and validation of our methodology on a dataset of real aircraft flight tests, while imposing knowledge of aircraft physics into the model.


Gaussian process Kernel methods Approximate inference Multi-output regression Flight-test data 


  1. 1.
    Alvarez, M.A., Luengo, D., Lawrence, N.D.: Latent force models. In: Dyk, D.A.V., Welling, M. (eds.) AISTATS, JMLR Proceedings, vol. 5, pp. 9–16. (2009)
  2. 2.
    Bonilla, E., Chai, K.M., Williams, C.: Multi-task Gaussian process prediction. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems 20, pp. 153–160. MIT Press, Cambridge (2008)Google Scholar
  3. 3.
    Boyle, P., Frean, M.: Dependent Gaussian processes. In: Advances in Neural Information Processing Systems, vol. 17, pp. 217–224. MIT Press (2005)Google Scholar
  4. 4.
    Cao, Y., Fleet, D.J.: Generalized product of experts for automatic and principled fusion of Gaussian process predictions. CoRR abs/1410.7827 (2014).
  5. 5.
    Chen, T., Ren, J.: Bagging for Gaussian process regression. Neurocomputing 72(7), 1605–1610 (2009)CrossRefGoogle Scholar
  6. 6.
    Chiplunkar, A., Rachelson, E., Colombo, M., Morlier, J.: Sparse physics-based Gaussian process for multi-output regression using variational inference. In: Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods, pp. 437–445 (2016)Google Scholar
  7. 7.
    Constantinescu, E.M., Anitescu, M.: Physics-based covariance models for Gaussian processes with multiple outputs. Int. J. Uncertain. Quantif. 3, 47–71 (2013)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Deisenroth, M.P., Ng, J.W.: Distributed Gaussian processes. arXiv preprint arXiv:1502.02843 (2015)
  9. 9.
    Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)CrossRefGoogle Scholar
  10. 10.
    March, W.B., Xiao, B., Biros, G.: Askit: approximate skeletonization kernel-independent treecode in high dimensions. SIAM J. Sci. Comput. 37(2), A1089–A1110 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Rasmussen, C.E., Ghahramani, Z.: Infinite mixtures of Gaussian process experts. In: Advances in Neural Information Processing Systems, vol. 2, pp. 881–888 (2002)Google Scholar
  12. 12.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). MIT Press, Cambridge (2005)Google Scholar
  13. 13.
    Solak, E., Murray-smith, R., Leithead, W.E., Leith, D.J., Rasmussen, C.E.: Derivative observations in gaussian process models of dynamic systems. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15, pp. 1057–1064. MIT Press, Cambridge (2003)Google Scholar
  14. 14.
    Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York (1999)CrossRefzbMATHGoogle Scholar
  15. 15.
    Titsias, M.K.: Variational learning of inducing variables in sparse gaussian processes. In: Artificial Intelligence and Statistics, vol. 12, pp. 567–574 (2009)Google Scholar
  16. 16.
    Vanhatalo, J., Riihimäki, J., Hartikainen, J., Jylänki, P., Tolvanen, V., Vehtari, A.: GPstuff: Bayesian modeling with Gaussian processes. J. Mach. Learn. Res. 14(1), 1175–1179 (2013)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ankit Chiplunkar
    • 1
    • 3
    Email author
  • Emmanuel Rachelson
    • 2
  • Michele Colombo
    • 1
  • Joseph Morlier
    • 2
    • 3
  1. 1.Airbus Operations S.A.S.ToulouseFrance
  2. 2.ISAE-Supaero, Département d’Ingénierie des Systèmes Complexes (DISC)ToulouseFrance
  3. 3.Université de Toulouse, CNRS, ISAE-SUPAERO, Institut Clément Ader (ICA)Toulouse Cedex 4France

Personalised recommendations