Abstract
One of the main objectives of science is to predict future events, i.e., more precisely, the results of future measurements and observations. If we take into account the probabilistic uncertainty related to the inaccuracy of the measurement results, to the inaccuracy of the model, and to the inaccuracy of the prior information, then the most adequate approach is to generate a posterior distribution by using Bayes’ theorem. For the simplest posterior distributions, we can deduce explicit analytical formulas for the resulting statistical characteristics (mean, standard deviation, etc.) of the predict future measurement result. However, in general, such formulas are not possible, so we have to use a Monte-Carlo simulation of the corresponding joint distribution of the future measurement results and model parameters.
The main computational challenge here is that there is no general algorithm for simulating an arbitrary multi-variate distribution; such algorithms are known only for single-variate distributions and – in some cases – for the case of several variables. Thus, we need to reduce the general simulation problem to such simplified cases. We show that this problem can be solved by using the general constraints approach, and that this idea clarifies Gibbs sampling – one of the most widely used techniques for such simulation. This interpretation of Gibbs sampling enables us to analyze Gibbs sampling – in particular, to obtain a (somewhat counterintuitive) result that while a straightforward parallelization is possible for deterministic constraint propagation, parallelization does not work even in the simplest two-variable probabilistic case.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Houser, D., Keane, M., McCabe, K.: Behavior in a dynamic decision problem: An analysis of experimental evidence using a bayesian type classification algorithm. Econometrica 72(3), 781–822 (2004)
Jaulin, L., Kieffer, M., Didrit, O., Walter, E.: Applied Interval Analysis, with Examples in Parameter and State Estimation, Robust Control and Robotics. Springer, London (2001)
Jaynes, E.T.: Probability Theory: The Logic of Science, vol. 1. Cambridge University Press, Cambridge (2003)
Luce, R.D., Raiffa, H.: Games and Decisions: Introduction and Critical Survey. Dover, New York (1989)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Koshelev, M. (2014). Gibbs Sampling as a Natural Statistical Analog of Constraints Techniques: Prediction in Science under General Probabilistic Uncertainty. In: Ceberio, M., Kreinovich, V. (eds) Constraint Programming and Decision Making. Studies in Computational Intelligence, vol 539. Springer, Cham. https://doi.org/10.1007/978-3-319-04280-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-04280-0_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04279-4
Online ISBN: 978-3-319-04280-0
eBook Packages: EngineeringEngineering (R0)