Abstract
We describe an algorithm for solving MDPs with large state and action spaces, represented as factored MDPs with factored action spaces. Classical algorithms for solving MDPs are not effective since they require enumerating all the states and actions. As such, model minimization techniques have been proposed, and specifically, we extend the previous work on model minimization algorithm for MDPs with factored state and action spaces. Using algebraic decision diagrams, we compactly represent blocks of states and actions that can be regarded equivalent. We describe the model minimization algorithm that uses algebraic decision diagrams, and show that this new algorithm can handle MDPs with millions of states and actions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Iris Bahar, Erica Frohm, Charles Gaona, Gary Hachtel, Enrico Macii, Abelardo Pardo, and Fabio Somenzi. Algebraic decision diagrams and their applications. In Proceedings of the International Conference on Computer-Aided Design, 1993.
Craig Boutilier, Thomas Dean, and Steve Hanks. Decision-theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 11, 1999.
Randal E. Bryant. Graph-based algorithms for boolean function manipulation. IEEE Transactions on Computers, C-35(8):677–691, 1986.
Thomas Dean and Robert Givan. Model minimization in Markov decision processes. In Proceedings AAAI-97, 1997.
Thomas Dean, Robert Givan, and Kee-Eung Kim. Solving planning problems with large state and action spaces. In Proceedings AIPS-98, 1998.
Robert Givan, Sonia Leach, and Thomas Dean. Bounded-parameter Markov decision processes. Artificial Intelligence, 122:71–109, 2000.
Juris Hartmanis and R.E. Stearns. Algebraic Structure Theory of Sequential Machines. Prentice-Hall, 1966.
Jesse Hoey, Robert St-Aubin, Alan Hu, and Craig Boutilier. SPUDD: Stochastic planning using decision diagrams. In Proceedings UAI-99, 1999.
Fabio Somenzi. CUDD: CU Decision Diagram Package Release 2.3.0. Department of Electrical and Computer Engineering, University of Colorado at Boulder, 1998.
Robert St-Aubin, Jesse Hoey, and Craig Boutilier. APRICODD: Approximate policy construction using decision diagrams. In Proceedings NIPS-2000, 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, KE., Dean, T. (2002). Solving Factored MDPs with Large Action Space Using Algebraic Decision Diagrams. In: Ishizuka, M., Sattar, A. (eds) PRICAI 2002: Trends in Artificial Intelligence. PRICAI 2002. Lecture Notes in Computer Science(), vol 2417. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45683-X_11
Download citation
DOI: https://doi.org/10.1007/3-540-45683-X_11
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44038-3
Online ISBN: 978-3-540-45683-4
eBook Packages: Springer Book Archive