Sequential Decision Making Under Uncertainty

Abstract

My main research centers around sequential decision making under uncertainty. In a complex dynamical system useful abstractions of knowledge can be essential to an autonomous agent for e.cient decision making. Predictive State Representation, PSR, has been developed to provide a maintainable, self-verifiable and learnable representation of the knowledge of the world. I was very much intrigued by the PSR work, and started working on incorporating PSRs into POMDP control algorithms. Since the representational power of PSRs is equivalent to the belief state representation in POMDPs, one can imagine PSR planning algorithms, working in the context of controlling dynamical systems. In prior work [1] I developed an exact planning algorithm based on known PSR parameters. However, like all other exact algorithms, this approach has exponential complexity in the worst case. In preliminary experiments on a variety of standard domains, the empirical performance seems similar to belief-based planning.