Abstract
In this paper, we examine a pursuit-evasion problem where more than one pursuer may search for one evader in indoor environments. Partially Observable Markov Decision Processes (POMDPs) provide a framework to model the uncertainty arisen from the unknown location of the evader. However, the approach is intractable even with a single pursuer and an evader. Therefore, we propose a Hybrid Hierarchical POMDP structure for improved scalability and efficiency. The structure consists of (i) the base MDPs for the cases where the evader is visible to the pursuers, (ii) the abstract POMDPs for the evader states that are not directly observable, and (iii) the transition states bridging between the base MDPs and abstract POMDPs. This hybrid approach significantly reduces the number of states expanded in the policy tree to solve the problem by abstracting environment structures. Experimental results show that our method expands only 5% of nodes generated from a standard POMDP solution.
This research was funded by AFOSR awards, FA9550-18-1-0097 and FA9550-15-1-0442.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arai, S., Sycara, K., Payne, T.R.: Experience-Based Reinforcement Learning to Acquire Effective Behavior in a Multi-agent Domain, pp. 125–135. Springer (2000)
Bellman, R.: Dynamic Programming. Courier Corporation (2013)
Eddy, W.F.: A new convex hull algorithm for planar sets. ACM Trans. Math. Softw. (TOMS) 3(4), 398–403 (1977)
Gopalan, N., des Jardins, M., Littman, M.L., MacGlashan, J., Squire, S., Tellex, S., Winder, J., Wong, L.L.: Planning with Abstract Markov Decision Processes (2017)
Guibas, L.J., Latombe, J.C., LaValle, S.M., Lin, D., Motwani, R.: A visibility-based pursuit-evasion problem. Int. J. Comput. Geom. Appl. 9(04), 471–493 (1999)
Hauskrecht, M.: Value-function approximations for partially observable markov decision processes. J. Artif. Intell. Res. 13, 33–94 (2000)
Hollinger, G., Kehagias, A., Singh, S.: Probabilistic strategies for pursuit in cluttered environments with multiple robots. In: IEEE International Conference on Robotics and Automation, pp. 3870–3876. IEEE (2007)
Hollinger, G., Singh, S., Djugash, J., Kehagias, A.: Efficient multi-robot search for a moving target. Int. J. Robot. Res. 28(2), 201–219 (2009)
Isler, V., Sun, D., Sastry, S.: Roadmap based pursuit-evasion and collision avoidance. Robot. Sci. Syst. 1, 257–264 (2005)
Ong, S.C., Png, S.W., Hsu, D., Lee, W.S.: Planning under uncertainty for robotic tasks with mixed observability. Int. J. Robot. Res. 29(8), 1053–1068 (2010)
Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of markov decision processes. Math. Oper. Res. 12(3), 441–450 (1987)
Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: analyzing teamwork theories and models. J. Artif. Intell. Res. 16, 389–423 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Yi, S., Nam, C., Sycara, K. (2019). Indoor Pursuit-Evasion with Hybrid Hierarchical Partially Observable Markov Decision Processes for Multi-robot Systems. In: Correll, N., Schwager, M., Otte, M. (eds) Distributed Autonomous Robotic Systems. Springer Proceedings in Advanced Robotics, vol 9. Springer, Cham. https://doi.org/10.1007/978-3-030-05816-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-05816-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05815-9
Online ISBN: 978-3-030-05816-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)