Abstract
In recent years, there has been significant interest in developing techniques for finding policies for Partially Observable Markov Decision Problems (POMDPs). This paper introduces a new POMDP filtering technique that is based on Incremental Pruning [1], but relies on geometries of hyperplane arrangements to compute for optimal policy. This new approach applies notions of linear algebra to transform hyperplanes and treat their intersections as witness points [5]. The main idea behind this technique is that a vector that has the highest value at any of the intersection points must be part of the policy. IPBS is an alternative of using linear programming (LP), which requires powerful and expensive libraries, and which is subjected to numerical instability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cassandra, A., Littman, M.L., Zhang, N.L.: Incremental pruning: A simple, fast, exact algorithm for partially observable Markov decision processes. In: Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (1997)
Cassandra, A.R., Kaelbling, L.P., Littman, M.L.: Acting optimally in partially observable stochastic domains. In: Proceedings of the Twelfth National Conference on Artificial Intelligence, Seattle, WA (1994)
Galántai, A.: Projectors and Projection Methods. Kluwer Academic Pub., 3300 AH Dordrecht (2004)
Goldsmith, J., Mundhenk, M.: Complexity issues in Markov decision processes. In: Proceedings of the IEEE Conference on Computational Complexity. IEEE, Los Alamitos (1998)
Littman, M.L.: The witness algorithm: Solving partially observable Markov decision processes. Technical Report CS-94-40, Brown University, Department of Computer Science, Providence, RI (December 1994)
Pineau, J.: Tractable Planning Under Uncertainty: Exploiting Structure. Ph.D. thesis, Carnegie Mellon University (August 2004)
Poupart, P., Boutilier, C.: VDCBPI: an approximate scalable algorithm for large scale POMDPs. In: Proceedings of NIPS, Vancouver (2004)
Smith, T., Simmons, R.: Heuristic search value iteration for pomdps. In: Uncertainty in Artificial Intelligence (2004)
Sondik, E.: The optimal control of partially observable Markov processes. Ph.D. thesis, Standford University (1971)
Spaan, M.T.J., Vlassis, N.: Perseus: Randomized point-based value iteration for pomdps. JAIR 24, 195–220 (2005)
Spaan, M.T.J.: Cooperative active perception using POMDPs. In: AAAI 2008 Workshop on Advancements in POMDP Solvers (July 2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Borera, E.C., Pyeatt, L.D., Randrianasolo, A.S., Naser-Moghadasi, M. (2010). POMDP Filter: Pruning POMDP Value Functions with the Kaczmarz Iterative Method. In: Sidorov, G., Hernández Aguirre, A., Reyes GarcÃa, C.A. (eds) Advances in Artificial Intelligence. MICAI 2010. Lecture Notes in Computer Science(), vol 6437. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16761-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-16761-4_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16760-7
Online ISBN: 978-3-642-16761-4
eBook Packages: Computer ScienceComputer Science (R0)