Abstract
In this paper the SPI system is presented. SPI is a software tool for planning under uncertainty based on learning Markov Decision Processes. A brief review of some similar tools as well as the scientific basis of factored representations and some of its variants are included. Among these variants are qualitative representations and hybrid qualitative-discrete representations that are the core of the software tool. The functional structure of SPI, which is composed of four main modules, is also described. These modules are: the compiler, the policy server, a format translator and a didactic simulator. The experimental results obtained when testing SPI in a robot navigation domain using different types of representations and different state partitions demonstrated its capability to reduce state spaces.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)
Chadès, I., Chapron, G., Cros, M.J., Garcia, F., Sabbadin, R.: MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37, 916–920 (2014)
Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)
Hoey, J., St-Aubin, R., Hu, A., Boutilier, C.: SPUDD: stochastic planning using decision diagrams. In: Proceedings of the 15th Conference on Uncertainty in AI, UAI 1999, pp. 279–288 (1999)
Munos, R., Moore, A.: Variable resolution discretization for high-accuracy solutions of optimal control problems. In: Dean, T. (ed.) Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1348–1355. Morgan Kaufmann Publishers, San Francisco (1999)
Porta, J.M., Vlassis, N., Spaan, M.T.J., Poupart, P.: Point-based value iteration for continuous POMDPs. J. Mach. Learn. Res. 7, 2329–2367 (2006)
Poupart, P.: Exploiting structure to efficiently solve large scale partially observable Markov decision processes. Ph.D. thesis, University of Toronto (2005)
Puterman, M.L.: Markov Decision Processes. Wiley, New York (1994)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Reyes, A., Sucar, L.E., Morales, E., Ibarguengoytia, P.H.: Abstraction and refinement for solving Markov decision processes. In: Workshop on Probabilistic Graphical Models, PGM 2006, Chezch Republic, pp. 263–270 (2006)
Reyes, A., Sucar, L.E., Morales, E.F.: AsistO: a qualitative MDP-based recommender system for power plant operation. Computacion y Sistemas 13(1), 5–220 (2009)
Reyes, A., Sucar, L.E., Morales, E.F., Ibargüengoytia, P.H.: Solving hybrid Markov decision processes. In: Gelbukh, A., Reyes-Garcia, C.A. (eds.) MICAI 2006. LNCS (LNAI), vol. 4293, pp. 227–236. Springer, Heidelberg (2006). https://doi.org/10.1007/11925231_22
Sandoval, C., Galindo, X., Salas, R.: Herramienta software para resolver procesos de decisiÃn de Markov utilizando recocido simulado. In: Memorias de la Décima Quinta Conferencia Iberoamericana en Sistemas, Cibernética e Informática (CISCI 2016) (2016)
Sarmiento, A., Riaño, G.: JMDP: an object oriented framework for modeling MDPs. In: Informatics Annual Meeting (2006)
Sigaud, O., Buffet, O.: Markov Decision Processes in Artificial Intelligence. ISTE Ltd./Wiley, London/Hoboken (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Reyes, A., Ibargüengoytia, P.H., Santamaría, G. (2019). SPI: A Software Tool for Planning Under Uncertainty Based on Learning Factored MDPs. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds) Advances in Soft Computing. MICAI 2019. Lecture Notes in Computer Science(), vol 11835. Springer, Cham. https://doi.org/10.1007/978-3-030-33749-0_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-33749-0_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33748-3
Online ISBN: 978-3-030-33749-0
eBook Packages: Computer ScienceComputer Science (R0)