Abstract
With the Gaussian Process model, the predictive distribution of the output corresponding to a new given input is Gaussian. But if this input is uncertain or noisy, the predictive distribution becomes non-Gaussian. We present an analytical approach that consists of computing only the mean and variance of this new distribution (Gaussian approximation). We show how, depending on the form of the covariance function of the process, we can evaluate these moments exactly or approximately (within a Taylor approximation of the covariance function). We apply our results to the iterative multiple-step ahead prediction of non-linear dynamic systems with propagation of the uncertainty as we predict ahead in time. Finally, using numerical examples, we compare the Gaussian approximation to the numerical approximation of the true predictive distribution by simple Monte-Carlo.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Williams, C.K.I., Rasmussen, C.E.: Gaussian Processes for Regression. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 514–520. MIT Press, Cambridge (1996)
Williams, C.K.I.: Prediction with Gaussian Processes: From linear regression to linear prediction and beyond. Technical Report NCRG-97-012, Dept of Computer Science and Applied Mathematics. Aston University (1997)
Williams, C.K.I.: Gaussian Processes, 2nd edn. The handbook of Brain Theory and Neural Networks. MIT Press, Cambridge (2002)
Mackay, D.J.C.: Information theory, Inference and Learning Algorithms. Cambridge University Press, Cambridge (2003)
Girard, A., Rasmussen, C., Quinonero-Candela, J., Murray-Smith, R.: Gaussian Process PriorsWith Uncertain Inputs – Application to Multiple-Step Ahead Time Series Forecasting. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 545–552. MIT Press, Cambridge (2003)
Quinonero-Candela, J., Girard, A., Larsen, J., Rasmussen, C.E.: Propagation of Uncertainty in Bayesian Kernels Models – Application to Multiple-Step Ahead Forecasting. In: IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 2, pp. 701–704 (2003)
Girard, A., Rasmussen, C., Murray-Smith, R.: Gaussian Process Priors with Uncertain Inputs: Multiple-Step Ahead Prediction. Technical Report TR-2002- 119, Computing Science Department, University of Glasgow (2002)
Lindley, D.V.: Introduction to Probability and Statistics from a Bayesian viewpoint. Cambridge University Press, Cambridge (1969)
Papoulis, A.: Probability, random variables, and stochastic processes. McGraw-Hill, New York (1991)
Rasmussen, C.E.: Evaluation of Gaussian Processes and other methods for nonlinear regresion. PhD thesis, University of Toronto (1996)
Neal, R.M.: Bayesian learning for neural networks. PhD thesis, University of Toronto (1995)
MacKay, D.J.C.: Bayesian methods for backpropagation networks. In: Domany, E., van Hemmen, J.L., Schulten, K. (eds.) Models of Neural Networks III, pp. 211–254. Springer, New York (1994)
Takens, F.: Detecting strange attractors in turbulence. In: Steffens, P. (ed.) EAMT-WS 1993. LNCS, vol. 898, pp. 366–381. Springer, Heidelberg (1995)
Murray-Smith, R., Girard, A.: Gaussian Process priors with ARMA noise models. In: Irish Signals and Systems Conference, Maynooth, pp. 147–152 (2001)
Tresp, V., Hofmann, R.: Missing and Noisy Data in Nonlinear Time-Series Prediction. In: Girosi Jr., S.F., Mahoul, E.M., Wilson, E. (eds.) Neural Networks for Signal Processing, New York. IEEE Signal Processing Society, vol. 24, pp. 1–10 (1995)
Tresp, V., Hofmann, R.: Nonlinear Time-Series Prediction with Missing and Noisy Data. Neural Computation 10, 731–747 (1998)
Ahmad, S., Tresp, V.: Some Solutions to the Missing Feature Problem in Vision. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 393–400. Morgan Kaufmann, San Mateo (1993)
Judd, K., Small, M.: Towards long-term prediction. Physica D. 136, 31–44 (2000)
Mackey, M.C., Glass, L.: Oscillation and chaos in physiological control systems. Science 197, 287–289 (1977)
Quinonero-Candela, J., Girard, A.: Prediction at an Uncertain Input for Gaussian Processes and Relevance Vector Machines – Application to Multiple-Step Ahead Time-Series Forecasting. Technical report, Informatics and Mathematical Modelling, Technical Univesity of Denmark (2002)
Murray-Smith, R., Sbarbaro, D.: Nonlinear adaptive control using non-parametric Gaussian process prior models. In: 15th IFAC Triennial World Congress. International Federation of Automatic Control (2002)
Murray-Smith, R., Sbarbaro, D., Rasmussen, C.E., Girard, A.: Adaptive, Cautious, Predictive control with Gaussian Process priors. In: IFAC International Symposium on System Identification, Rotterdam (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Girard, A., Murray-Smith, R. (2005). Gaussian Processes: Prediction at a Noisy Input and Application to Iterative Multiple-Step Ahead Forecasting of Time-Series. In: Murray-Smith, R., Shorten, R. (eds) Switching and Learning in Feedback Systems. Lecture Notes in Computer Science, vol 3355. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30560-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-30560-6_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24457-8
Online ISBN: 978-3-540-30560-6
eBook Packages: Computer ScienceComputer Science (R0)