Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves

  • Simo Särkkä
  • Arno Solin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7944)

Abstract

Gaussian process regression is a machine learning paradigm, where the regressor functions are modeled as realizations from an a priori Gaussian process model. We study abstract continuous-space Gaussian regression problems where the training set covers the whole input space instead of consisting of a finite number of distinct points. The model can be used for analyzing theoretical properties of Gaussian process regressors. In this paper, we present the general continuous-space Gaussian process regression equations and discuss their close connection with Wiener filtering. We apply the results to estimation of learning curves as functions of training set size and input dimensionality.

Keywords

Gaussian process regression continuous-space measurement Wiener filter learning curve 

References

  1. 1.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press (2006)Google Scholar
  2. 2.
    Sollich, P., Williams, C.: Using the equivalent kernel to understand Gaussian process regression. In: NIPS, vol. 17, pp. 1313–1320. MIT Press (2005)Google Scholar
  3. 3.
    Opper, M., Vivarelli, F.: General bounds on Bayes errors for regression with gaussian processes. In: NIPS, vol. 11, pp. 302–308. The MIT Press (1999)Google Scholar
  4. 4.
    Sollich, P., Halees, A.: Learning curves for Gaussian process regression: Approximations and bounds. Neural Computation 14(6), 1393–1428 (2002)MATHCrossRefGoogle Scholar
  5. 5.
    Särkkä, S.: Learning curves for Gaussian processes via numerical cubature integration. In: Honkela, T. (ed.) ICANN 2011, Part I. LNCS, vol. 6791, pp. 201–208. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Wiener, N.: Extrapolation, Interpolation and Smoothing of Stationary Time Series with Engineering Applications. John Wiley & Sons, Inc., New York (1950)Google Scholar
  7. 7.
    Van Trees, H.L.: Detection, Estimation, and Modulation Theory Part I. John Wiley & Sons, New York (1968)MATHGoogle Scholar
  8. 8.
    Papoulis, A.: Probability, Random Variables, and Stochastic Processes. McGraw-Hill (1984)Google Scholar
  9. 9.
    Särkkä, S.: Linear operators and stochastic partial differential equations in Gaussian process regression. In: Honkela, T. (ed.) ICANN 2011, Part II. LNCS, vol. 6792, pp. 151–158. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  10. 10.
    Gonzalez, R.C., Woods, R.E.: Digital image processing. Prentice Hall (2002)Google Scholar
  11. 11.
    Opper, M.: Regression with Gaussian processes: Average case performance. In: Theoretical Aspects of Neural Computation. Springer (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Simo Särkkä
    • 1
  • Arno Solin
    • 1
  1. 1.Department of Biomedical Engineering and Computational ScienceAalto UniversityFinland

Personalised recommendations