Definition
Gaussian processes generalize multivariate Gaussian distributions over finite-dimensional vectors to infinite dimensionality. Specifically, a Gaussian process is a stochastic process that has Gaussian-distributed finite-dimensional marginal distributions, hence the name. In doing so, it defines a distribution over functions, i.e., each draw from a Gaussian process is a function. Gaussian processes provide a principled, practical, and probabilistic approach to inference and learning in kernel machines.
Motivation and Background
Bayesian probabilistic approaches have many virtues, including their ability to incorporate prior knowledge and their ability to link related sources of information. Typically, we are given a set of data points sampled from an underlying but unknown distribution, each of which includes input x and output y, such as the ones shown in Fig. 1a. The task is to learn a...
This is a preview of subscription content, log in via an institution to check access.
Recommended Reading
Abrahamsen P (1992) A review of Gaussian random fields and correlation functions. Rapport 917, Norwegian Computing Center, Oslo. www.nr.no/publications/917_Rapport.ps
Bratières S, Quadrianto N, Ghahramani Z (to appear) GPstruct: Bayesian structured prediction using Gaussian processes. IEEE Trans Pattern Anal Mach Intell
Chu W, Ghahramani Z (2005a) Gaussian processes for ordinal regression. J Mach Learn Res 6:1019–1041
Chu W, Ghahramani Z (2005b) Preference learning with Gaussian processes. In: Proceedings of international conference on machine learning (ICML), Bonn
Chu W, Sindhwani V, Ghahramani Z, Keerthi S (2006) Relational learning with Gaussian processes. In: Proceedings of advances in neural information processing systems (NIPS), Vancouver
Deisenroth MP, Rasmussen CE, Peters J (2009) Gaussian process dynamic programming. Neurocomputing 72(7–9):1508–1524
Engel Y, Mannor S, Meir R (2005) Reinforcement learning with Gaussian processes. In: Proceedings of international conference on machine learning (ICML), Bonn
Guiver J, Snelson E (2008) Learning to rank with softrank and Gaussian processes. In: Proceedings of special interest group on information retrieval (SIGIR), Singapore
Hensman J, Fusi N, Lawrence N (2013) Gaussian processes for big data. In: Proceedings of uncertainty in artificial intelligence (UAI), Bellvue
Kersting K, Xu Z (2009) Learning preferences with hidden common cause relations. In: Proceedings of European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD), Bled
Krige DG (1951) A statistical approach to some basic mine valuation problems on the witwatersrand. J Chem Metall Mining Soc S Afr 52(6):119–139
Lawrence N (2005) Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J Mach Learn Res 6:1783–1816
Lawrence N, Urtasun R (2009) Non-linear matrix factorization with Gaussian processes. In: Proceedings of international conference on machine learning (ICML), Montreal
Lloyd JR, Duvenaud D, Grosse R, Tenenbaum JB, Ghahramani Z (2014) Automatic construction and natural-language description of nonparametric regression models. In: Proceedings of association for the advancement of artificial intelligence (AAAI), Québec City
Hennig P (2013) Animating samples from Gaussian distributions. Technical report, 8. Max Planck Institute for Intelligent Systems
Hernández-Lobato D, Sharmanska V, Kersting K, Lampert C, Quadrianto N (2014) Mind the Nuisance: Gaussian process classification using privileged noise. In: Proceedings of advances in neural information processing systems (NIPS), Montreal
MacKay DJC (1992) The evidence framework applied to classification networks. Neural Comput 4(5):720–736
Matheron G (1963) Principles of geostatistics. Econ Geol 58:1246–1266
Neal R (1996) Bayesian learning in neural networks. Springer, New York
Nickisch H, Rasmussen CE (2008) Approximations for binary Gaussian process classification. J Mach Learn Res 9:2035–2078
Plagemann C, Kersting K, Pfaff P, Burgard W (2007) Gaussian beam processes: a nonparametric Bayesian measurement model for range finders. In: Proceedings of robotics: science and systems (RSS), Atlanta
Quiñonero-Candela J, Rasmussen CE (2005) A unifying view of sparse approximate Gaussian process regression. J Mach Learn Res 6:1939–1959
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge
Särkkä S, Solin A, Hartikainen J (2013) Spatiotemporal learning via infinite-dimensional Bayesian filtering and smoothing. IEEE Signal Process Mag 30:51–61
Schwaighofer A, Tresp V (2003) Transductive and inductive methods for approximate guassian process regression. In: Proceedings of advances in neural information processing systems (NIPS), Vancouver
Seeger M, Williams CKI, Lawrence N (2003) Fast forward selection to speed up sparse Gaussian process regression. In: Proceedings of artificial intelligence and statistics (AISTATS), Key West
Silva R, Chu W, Ghahramani Z (2007) Hidden common cause relations in relational learning. In: Proceedings of advances in neural information processing systems (NIPS), Vancouver
Silverman BW (1985) Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J R Stat Soc B 47(1):1–52
Snelson E, Ghahramani Z (2006) Sparse Gaussian processes using pseudo-inputs. In: Proceedings of advances in neural information processing systems (NIPS), Vancouver
Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. In: Proceedings of advances in neural information processing systems (NIPS), Lake Tahoe
Tresp V (2000a) A Bayesian committee machine. Neural Comput 12(11):2719–2741
Tresp V (2000b) Mixtures of Gaussian processes. In: Proceedings of advances in neural information processing systems (NIPS), Denver
Williams C, Barber D (1998) Bayesian classification with Gaussian processes. IEEE Trans Pattern Anal Mach Intell PAMI 20(12):1342–1351
Williams C, Rasmussen C (1996) Gaussian processes for regression. In: Proceedings of advances in neural information processing systems (NIPS), Denver
Xu Z, Kersting K, Tresp V (2009) Multi-relational learning with Gaussian processes. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), Pasadena
Yu K, Tresp V, Schwaighofer A (2005) Learning Gaussian processes from multiple tasks. In: Proceedings of international conference on machine learning (ICML), Bonn
Yu K, Chu W, Yu S, Tresp V, Xu Z (2006a) Stochastic relational models for discriminative link prediction. In: Proceedings of advances in neural information processing systems (NIPS), Vancouver
Yu S, Yu K, Tresp V, Kriegel HP (2006b) Collaborative ordinal regression. In: Proceedings of international conference on machine learning (ICML), Pittsburgh
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media New York
About this entry
Cite this entry
Quadrianto, N., Kersting, K., Xu, Z. (2016). Gaussian Process. In: Sammut, C., Webb, G. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7502-7_108-1
Download citation
DOI: https://doi.org/10.1007/978-1-4899-7502-7_108-1
Received:
Accepted:
Published:
Publisher Name: Springer, Boston, MA
Online ISBN: 978-1-4899-7502-7
eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering