Automatic Model Selection of the Mixtures of Gaussian Processes for Regression

  • Zhe Qiang
  • Jinwen Ma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9377)


For the learning of mixtures of Gaussian processes, model selection is an important but difficult problem. In this paper, we develop an automatic model selection algorithm for mixtures of Gaussian processes in the light of the reversible jump Markov chain Monte Carlo framework for Gaussian mixtures. In this way, the component number and the parameters are updated according the five types of random moves and model selection can be made automatically. The key idea is that the moves of component splitting or merging preserve the zeroth, first and second moments of the components so that the covariance parameters of the new components can be related to the origin ones. It is demonstrated by the simulation experiments that this automatic model selection algorithm is feasible and effective.


Mixtures of Gaussian processes Reversible jump MCMC Model selection Regression Split and merge moves 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian process for machine learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  2. 2.
    Tresp, V.: Mixtures of Gaussian processes. In: Proc. of the Conf. on Neural Information Processing Systems (NIPS), pp. 654–660 (2000)Google Scholar
  3. 3.
    Shi, J.Q., Murray-Smith, R., Titterington, D.M.: Bayesian regression and classification using mixtures of Gaussian processes. International Journal of Adaptive Control and Signal Processing 17(2), 149–161 (2003)CrossRefGoogle Scholar
  4. 4.
    Shi, J.Q., Murray-Smith, R., Titterington, D.M.: Hierarchical Gaussian process mixtures for regression. Statistics and Computing 15(1), 31–41 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Green, P.J.: Reversible jump markov chain monte carlo computation and bayesian model determination. Biometrika 82(4), 711–732 (1995)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Richardson, S., Green, P.J.: On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 59(4), 731–792 (1997)CrossRefGoogle Scholar
  7. 7.
    Rasmussen, C.E.: Evaluation of Gaussian processes and other methods for non-linear regression. Diss. University of Toronto (1996)Google Scholar
  8. 8.
    Neal, R.M.: MCMC using Hamiltonian dynamics. In: Handbook of Markov Chain Monte Carlo (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

<SimplePara><Emphasis Type="Bold">Open Access</Emphasis> This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. </SimplePara> <SimplePara>The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.</SimplePara>

Authors and Affiliations

  • Zhe Qiang
    • 1
  • Jinwen Ma
    • 1
  1. 1.Department of Information Science, School of Mathematical Sciences and LMAMPeking UniversityBeijingChina

Personalised recommendations