Abstract
We explore the possibilities of Markov Chain Monte Carlo simulation methods to solve sequential decision processes evolving stochastically in time. The application areas of such processes are fairly wide, embedded typically in the Decision Analysis framework, such as preventive maintenance of systems, where we shall find our illustrative examples.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bielza, C., Müller, P., Ríos-Insua, D.: Decision Analysis by augmented probability simulation. Management Science 45, 1552–1569 (1999)
Chib, S., Greenberg, E.: Understanding the Metropolis Hastings Algorithm. The American Statistician 49(4), 327–335 (1995)
Cho, D.I., Parlar, M.: A Survey of Maintenance Models for Multi-units Systems. European Journal of Operational Research 51, 1–23 (1991)
de Freitas, N., Hojen-Sorensen, P., Jordan, M.I., Russell, S.: Variational MCMC. In: Breese, J., Koller, D. (eds.) Uncertainty in Artificial Intelligence, Proceedings of the Seventeenth Conference (2001)
Dekker, R., Van der Duyn Schouten, F., Wildeman, R.: A Review of Multi- Component Maintenance Models with Economic Dependence. Mathematical Methods of Operations Research 45, 411–435 (1997)
French, S., Ríos Insua, D.: Statistical Decision Theory. Arnold (2000)
Gamerman, D.: Markov Chain Monte Carlo: stochastic simulation for Bayesian inference. Chapman & Hall, Boca Raton (1997)
Hastings, W.K.: Monte Carlo Sampling Methods Using Markov Chains and its Applications. Biometrika 57, 97–109 (1970)
Puterman, M.L.: Markov Decision Processes, In Handbooks in OR & MS, vol. 2. Elsevier, Amsterdam (1990)
Shaked, M., Shantikumar, J.G.: Reliability and Maintainability, In Handbooks in OR & MS, vol. 2. Elsevier, Amsterdam (1990)
Virto, M.A.: Métodos Montecarlo en Análisis de Decisiones. PhD. Thesis. Universidad Nacional de Educación a Distancia. Madrid (2002)
Virto, M.A., Martín, J., Ríos Insua, D., Moreno-Díaz, A.: Approximate Solutions of Complex Influence Diagrams through MCMC Methods. In: Gámez, Salmerón (eds.) Proceeedings of First European Workshop on Probabilistic Graphical Models, pp. 169–175 (2002)
Virto, M.A., Martín, J., Ríos Insua, D., Moreno-Díaz, A.: A Method for Sequential Optimization in Bayesian Analysis. In: Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (eds.) Bayesian Statistics, vol. 7, pp. 701–710. Oxford University Press, Oxford (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Moreno-Díaz, A., Virto, M.A., Martín, J., Insua, D.R. (2003). Approximate Solutions to Semi Markov Decision Processes through Markov Chain Montecarlo Methods. In: Moreno-Díaz, R., Pichler, F. (eds) Computer Aided Systems Theory - EUROCAST 2003. EUROCAST 2003. Lecture Notes in Computer Science, vol 2809. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45210-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-45210-2_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20221-9
Online ISBN: 978-3-540-45210-2
eBook Packages: Springer Book Archive