Abstract
This paper is concerned with estimation of learning curves for Gaussian process regression with multidimensional numerical integration. We propose an approach where the recursion equations for the generalization error are approximately solved using numerical cubature integration methods. The advantage of the approach is that the eigenfunction expansion of the covariance function does not need to be known. The accuracy of the proposed method is compared to eigenfunction expansion based approximations to the learning curve.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
O’Hagan, A.: Curve fitting and optimal design for prediction (with discussion). Journal of the Royal Statistical Society B 40(1), 1–42 (1978)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press (2006)
Bishop, C.M.: Pattern recognition and machine learning. Springer, Heidelberg (2006)
Opper, M.: Regression with Gaussian processes: Average case performance. In: Theoretical Aspects of Neural Computation. Springer, Heidelberg (1997)
Opper, M., Vivarelli, F.: General bounds on bayes errors for regression with gaussian processes. In: NIPS, vol. 11, pp. 302–308. The MIT Press, Cambridge (1999)
Sollich, P.: Approximate learning curves for Gaussian processes. In: Proceedings of ICANN 1999, pp. 437–442 (1999)
Sollich, P.: Learning curves for Gaussian processes. In: NIPS, vol. 11. The MIT Press, Cambridge (1999)
Sollich, P., Halees, A.: Learning curves for Gaussian process regression: Approximations and bounds. Neural Computation 14, 1393–1428 (2002)
Williams, C.K.I., Vivarelli, F.: Upper and lower bounds on the learning curve for Gaussian processes. Machine Learning 40, 77–102 (2000)
Sollich, P., Williams, C.: Using the equivalent kernel to understand Gaussian process regression. In: NIPS, vol. 17, pp. 1313–1320. MIT Press, Cambridge (2005)
Van Trees, H.L.: Detection, Estimation, and Modulation Theory Part I. John Wiley & Sons, New Yark (1968)
Papoulis, A.: Probability, Random Variables, and Stochastic Processes. McGraw-Hill, New York (1984)
Malzahn, D., Opper, M.: Learning curves for Gaussian process regression: A framework for good approximations. In: NIPS, vol. 13, MIT Press, Cambridge (2001)
Malzahn, D., Opper, M.: A variational approach to learning curves. In: NIPS, vol. 14, MIT Press, Cambridge (2002)
Ito, K., Xiong, K.: Gaussian filters for nonlinear filtering problems. IEEE Transactions on Automatic Control 45, 910–927 (2000)
Wu, Y., Hu, D., Wu, M., Hu, X.: A numerical-integration perspective on Gaussian filters. IEEE Transactions on Signal Processing 54, 2910–2921 (2006)
Arasaratnam, I., Haykin, S.: Cubature Kalman filters. IEEE Transactions on Automatic Control 54, 1254–1269 (2009)
Särkkä, S., Hartikainen, J.: On Gaussian optimal smoothing of non-linear state space models. IEEE Transactions on Automatic Control 55, 1938–1941 (2010)
Curtain, R.: A survey of infinite-dimensional filtering. SIAM Review 17(3), 395–411 (1975)
Ray, W.H., Lainiotis, D.G.: Distributed Parameter Systems. Dekker, New York (1978)
Cools, R.: Constructing cubature formulae: The science behind the art. Acta Numerica 6, 1–54 Cambridge University Press, Cambridge (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Särkkä, S. (2011). Learning Curves for Gaussian Processes via Numerical Cubature Integration. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6791. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21735-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-21735-7_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21734-0
Online ISBN: 978-3-642-21735-7
eBook Packages: Computer ScienceComputer Science (R0)