Abstract
This paper introduces a novel path planning technique called MCRT which is aimed at non-deterministic, partially known, real-time domains populated with dynamically moving obstacles, such as might be found in a real-time strategy (RTS) game. The technique combines an efficient form of Monte-Carlo tree search with the randomized exploration capabilities of rapidly exploring random tree (RRT) planning. The main innovation of MCRT is in incrementally building an RRT structure with a collision-sensitive reward function, and then re-using it to efficiently solve multiple, sequential goals. We have implemented the technique in MCRT-planner, a program which solves non-deterministic path planning problems in imperfect information RTS games, and evaluated it in comparison to four other state of the art techniques. Planners embedding each technique were applied to a typical RTS game and evaluated using the game score and the planning cost. The empirical evidence demonstrates the success of MCRT-planner.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balla, R.K. and Fern, A.: UCT for Tactical Assault Planning in Real-Time Strategy Games. In: Poceedings of the 21st International Joint Conference on Artificial Intelligence, pp. 40-45 (2009)
Barto, A.G. and Bradtke, S.J. and Singh, S.P.: Learning to act using Real-time Dynamic Programming. Artificial Intelligence. 72, 81–138 (1995)
Bellman, R.: The Theory of Dynamic Programming. Bulletin of The American Mathematical Society-BULL AMER MATH SOC. 60(6), 503-516 (1954)
Bond, D.M. and Widger, N.A. and Ruml, W. and Sun, X.: Real-Time Search in Dynamic Worlds. In: Proceedings of the Third Annual Symposium on Combinatorial Search, (2010)
Bonet, B. and Geffner, H.: Labelled RTDP: Improving the Convergence of Real-Time Dynamic Programming. In: Proceedings of ICAPS, pp. 12–21 (2003)
Bonet, B. and Geffner, H.: mGPT: A Probabilistic Planner Based on Heuristic Search. Journal of Artificial Intelligence Research. 24, 933–944 (2005)
Bulitko, V. and Lee, G.: Learning in Real-Time Search: A unifying framework. Journal of Aritificial Intelligence Research. 25(1), 119–157 (2006)
Buro, M.: ORTS: A Hack-free RTS Game Environment. In: Proceedings of the International Computers and Games Conference. pp. 280–291 (2002)
Gelly, S. and Silver, D.: Combining Online and Offline Knowledge in UCT. In: ICML 2007. pp. 273-280 (2007)
Hart, P.E. and Nilsson, N.J. and Raphael, B.: A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions of Systems Science and Cybernetics. 4(2), 100–107 (1968)
Hernández, C. and Baier J.: Real-Time Heuristic Search with Depression Avoidance. In: Proceedings of the twenty-second international joint conference on Artificial Intelligence. (2011)
Ishida, T.: Moving target search with intelligence. In: Proceedings of the tenth national conference on Artificial intelligence (AAAI92). (1992)
Kocsis, L. and Szepesvári, Cs.: Bandit Based Monte-Carlo Planning. In: Proceedings of the 17th European Conference on Machine Learning. pp. 282-293 (2006)
Koenig, S. and Sun, X.: Comparing Real-Time and Incremental Heuristic Search for Real- Time Situated Agents. Journal of Autonomous Agents and Multi-Agent Systems. 18(3), 313–341 (2009)
Korf, R. E.: Real-Time Heuristic Search. Artificial Intelligence. 42, 189–211 (1990)
Kuffner, J.J. and LaValle, S.M.: RRT-Connect: An Efficient Approach to Single-Query Path Planning. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 995–1001 (2000)
LaValle, S.M.: Planning Algorithms. Cambridge University Press. (2006)
Naveed, M. and Kitchin, D. and Crampton, A.: Monte-Carlo Planning for Pathfinding in Real- Time Strategy Games. In: Proceedings of PlanSIG 2010. pp. 125-132 (2010)
Sutton, R.S. and Barto, A.G.: Reinforcement Learning An Introduction. MIT Press. (1998)
Wood, J.D.: The Geomorphological Characterisation of Digital Elevation Models. Phd Thesis. University of Leicester, UK. (1996)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag London Limited
About this paper
Cite this paper
Naveed, M., Crampton, A., Kitchin, D., McCluskey, L. (2011). Real-Time Path Planning using a Simulation-Based Markov Decision Process. In: Bramer, M., Petridis, M., Nolle, L. (eds) Research and Development in Intelligent Systems XXVIII. SGAI 2011. Springer, London. https://doi.org/10.1007/978-1-4471-2318-7_3
Download citation
DOI: https://doi.org/10.1007/978-1-4471-2318-7_3
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-2317-0
Online ISBN: 978-1-4471-2318-7
eBook Packages: Computer ScienceComputer Science (R0)