Abstract
In this contribution we describe an approach to evolve composite covariance functions for Gaussian processes using genetic programming. A critical aspect of Gaussian processes and similar kernel-based models such as SVM is, that the covariance function should be adapted to the modeled data. Frequently, the squared exponential covariance function is used as a default. However, this can lead to a misspecified model, which does not fit the data well.
In the proposed approach we use a grammar for the composition of covariance functions and genetic programming to search over the space of sentences that can be derived from the grammar.
We tested the proposed approach on synthetic data from two-dimensional test functions, and on the Mauna Loa CO 2 time series. The results show, that our approach is feasible, finding covariance functions that perform much better than a default covariance function. For the CO 2 data set a composite covariance function is found, that matches the performance of a hand-tuned covariance function.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Affenzeller, M., Winkler, S., Wagner, S., Beham, A.: Genetic Algorithms and Genetic Programming: Modern Concepts and Practical Applications. Numerical Insights. CRC Press, Singapore (2009)
Duvenaud, D., Lloyd, J.R., Grosse, R., Tenenbaum, J.B., Ghahramani, Z.: Structure Discovery in Nonparametric Regression through Compositional Kernel Search. ArXiv e-prints (February 2013)
Duvenaud, D., Nickisch, H., Rasmussen, C.E.: Additive Gaussian processes. arXiv preprint arXiv:1112.4394 (2011)
Gagné, C., Schoenauer, M., Sebag, M., Tomassini, M.: Genetic programming for kernel-based learning with co-evolving subsets selection. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN IX. LNCS, vol. 4193, pp. 1008–1017. Springer, Heidelberg (2006)
Hasegawa, Y., Iba, H.: Latent variable model for estimation of distribution algorithm based on a probabilistic context-free grammar. IEEE Transactions on Evolutionary Computation 13(4), 858–878 (2009)
Howley, T., Madden, M.G.: An evolutionary approach to automatic kernel construction. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4132, pp. 417–426. Springer, Heidelberg (2006)
Koch, P., Bischl, B., Flasch, O., Beielstein, T., Weihs, C., Konen, W.: Tuning and evolution of support vector kernels. Evolutionary Intelligence 5, 153–170 (2012)
Koza, J.R.: Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge (1992)
McKay, R.I., Hoai, N.X., Whigham, P.A., Shan, Y., O’Neill, M.: Grammar-based genetic programming: a survey. Genetic Programming and Evolvable Machines 11(3/4), 365–396 (2010)
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning. MIT Press (2006)
Wagner, S.: Heuristic optimization software systems – Modeling of heuristic optimization algorithms in the HeuristicLab software environment. Ph.D. thesis, Institute for Formal Models and Verification, Johannes Kepler University, Linz (2009)
Wilson, A.G., Prescott Adams, R.: Gaussian Process Covariance Kernels for Pattern Discovery and Extrapolation. ArXiv e-prints (February 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kronberger, G., Kommenda, M. (2013). Evolution of Covariance Functions for Gaussian Process Regression Using Genetic Programming. In: Moreno-Díaz, R., Pichler, F., Quesada-Arencibia, A. (eds) Computer Aided Systems Theory - EUROCAST 2013. EUROCAST 2013. Lecture Notes in Computer Science, vol 8111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53856-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-53856-8_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53855-1
Online ISBN: 978-3-642-53856-8
eBook Packages: Computer ScienceComputer Science (R0)