Skip to main content
SpringerLink
Log in

Monte-Carlo Tree Search

  • Living reference work entry
  • First Online:
Encyclopedia of Computer Graphics and Games

  • 829 Accesses

Synonyms

MCTS; Monte-Carlo Tree Search; UCT

Definition

Monte-Carlo Tree Search (MCTS) (Coulom 2007; Kocsis et al. 2006) is a best-first search method that does not require a positional evaluation function. It is based on a randomized exploration of the search space. Using the results of previous explorations, the algorithm gradually builds up a game tree in memory and successively becomes better at accurately estimating the values of the most promising moves. MCTS consists of four strategic steps, repeated as long as there is time left (Chaslot et al. 2008b). The steps, outlined in Fig. 1, are as follows:

Fig. 1
figure 1

Outline of Monte-Carlo Tree Search (adapted from Chaslot et al. 2008b; Winands et al. 2010)

Full size image

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References and Further Reading

  • Abramson, B.: Expected-outcome: A general model of static evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 12(2), 182–193 (1990)

    Article  Google Scholar 

  • Arneson, B., Hayward, R.B., Henderson, P.: Monte Carlo Tree Search in Hex. IEEE Trans. Comput. Intell. AI Games 2(4), 251–258 (2010)

    Article  Google Scholar 

  • Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)

    Article  MATH  Google Scholar 

  • Balla, R.K., Fern A.: UCT for tactical assault planning in real-time strategy games. In: Boutilier, C. (ed.) Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09), pp. 40–45. AAAI Press, Menlo Park, CA, USA (2009)

    Google Scholar 

  • Billings, D., Peña, L., Schaeffer, J., Szafron, D.: Using probabilistic knowledge and simulation to play poker. In: Hendler, J., Subramanian, D. (eds) Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, pp. 697–703. AAAI Press/The MIT Press, Menlo Park, CA, USA (1999)

    Google Scholar 

  • Björnsson, Y., Finnsson, H.: CadiaPlayer: A simulation-based General Game Player. IEEE Trans. Comput. Intell. AI Games 1(l), 4–15 (2009)

    Article  Google Scholar 

  • Bouzy, B., Helmstetter, B.: Monte-Carlo Go developments. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) Advances in Computer Games 10: Many Games, Many Challenges. IFIP Advances in Information and Communication Technology, vol. 135, pp. 159–174. Kluwer, Boston (2004)

    Chapter  Google Scholar 

  • Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo Tree Search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)

    Article  Google Scholar 

  • Cazenave, T., Saffidine, A.: Score bounded Monte-Carlo Tree Search. In: van den Herik, H.J., Iida, H., Plaat, A. (eds.) Computers and Games (CG 2010). Lecture Notes in Computer Science, vol. 6515, pp. 93–104. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • Chaslot, G.M.J.-B., Winands, M.H.M., van den Herik, H.J.: Parallel Monte-Carlo Tree Search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) Computers and Games (CG 2008). Lecture Notes in Computer Science, vol. 5131, pp. 60–71. Springer, Berlin (2008a)

    Chapter  Google Scholar 

  • Chaslot, G.M.J.-B., Winands, M.H.M., van den Herik, H.J., Uiterwijk, J.W.H.M., Bouzy, B.: Progressive strategies for Monte-Carlo Tree Search. New Math. Nat. Comput. 4(3), 343–357 (2008b)

    Article  MATH  MathSciNet  Google Scholar 

  • Childs, B.E., Brodeur, J.H., Kocsis, L.: Transpositions and move groups in Monte Carlo Tree Search. In: Hingston, P., Barone, L. (eds.) Proceedings of the 2008 IEEE Symposium on Computational Intelligence and Games, pp. 389–395. IEEE, Piscataway, NJ, USA (2008)

    Google Scholar 

  • Ciancarini, P., Favini, G.P.: Monte Carlo Tree Search in Kriegspiel. AI J. 174(11), 670–684 (2010)

    MathSciNet  Google Scholar 

  • Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo Tree Search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M. (eds.) Computers and Games (CG 2006). Lecture Notes in Computer Science, vol. 4630, pp. 72–83. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Cowling, P.I., Powley, E.J., Whitehouse, D.: Information set Monte Carlo Tree Search. IEEE Trans. Comput. Intell. AI Games 4(2), 120–143 (2012)

    Article  Google Scholar 

  • Enzenberger, M., Müller, M.: A lock-free multithreaded Monte-Carlo Tree Search algorithm. In: van den Herik, H.J., Spronck, P. (eds.) Advances in Computer Games (ACG 2009). Lecture Notes in Computer Science (LNCS), vol. 6048, pp. 14–20. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Enzenberger, M., Müller, M., Arneson, B., Segal, R.: Fuego – an open-source framework for board games and Go engine based on Monte Carlo Tree Search. IEEE Trans. Comput. Intell AI Games 2(4), 259–270 (2010)

    Article  Google Scholar 

  • Gelly, S., Kocsis, L., Schoenauer, M., Sebag, M., Silver, D., Szepesvári, C., Teytaud, O.: The grand challenge of computer Go: Monte Carlo Tree Search and extensions. Commun. ACM 55(3), 106–113 (2012)

    Article  Google Scholar 

  • Ginsberg, M.L.: GIB: Steps toward an expert-level bridge-playing program. In: Dean, T. (ed.) Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99), vol. 1, pp. 584–589. Morgan Kaufmann, San Francisco, CA, USA (1999)

    Google Scholar 

  • Hennes, D., Izzo, D.: Interplanetary trajectory planning with Monte Carlo Tree Search. In: Yang, Q., Wooldridge, M. (eds.) Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), pp. 769–775. AAAI Press, Menlo Park, CA, USA (2015)

    Google Scholar 

  • Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo Planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) Machine Learning: ECML 2006. Lecture Notes in Artificial Intelligence, vol. 4212, pp. 282–293. Springer, Berlin (2006)

    Google Scholar 

  • Lorentz, R.J.: Amazons discover Monte-Carlo. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) Computers and Games (CG 2008). Lecture Notes in Computer Science, vol. 5131, pp. 13–24. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Nguyen, K.Q., Thawonmas, R.: Monte Carlo Tree Search for collaboration control of Ghosts in Ms. Pac-Man. IEEE Trans. Comput. Intell. AI Games 5(1), 57–68 (2013)

    Article  Google Scholar 

  • Nijssen, J.A.M., Winands, M.H.M.: Enhancements for multi-player Monte-Carlo Tree Search. In: van den Herik, H.J., Iida, H., Plaat, A. (eds.) Computers and Games (CG 2010). Lecture Notes in Computer Science, vol. 6151, pp. 238–249. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • Nijssen, J.A.M., Winands, M.H.M.: Monte Carlo Tree Search for the hide-and-seek game Scotland Yard. Trans. Comput. Intell. AI Games 4(4), 282–294 (2012)

    Article  Google Scholar 

  • Pepels, T., Winands, M.H.M., Lanctot, M.: Real-time Monte Carlo Tree Search in Ms Pac-Man. IEEE Trans. Comput. Intell. AI Games 6(3), 245–257 (2014)

    Article  Google Scholar 

  • Perez, D., Samothrakis, S., Lucas, S.M.: Knowledge-based fast evolutionary MCTS for general video game playing. In: Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2014), pp. 68–75 (2014)

    Google Scholar 

  • Ruijl, B., Vermaseren, J., Plaat, A. van den Herik, H.J.: Combining simulated annealing and Monte Carlo Tree Search for expression simplification. In: ICAART 2014, pp. 724–731 (2014)

    Google Scholar 

  • Schadd, M.P.D., Winands, M.H.M., Tak, M.J.W., Uiterwijk, J.W.H.M.: Single-player Monte-Carlo Tree Search for SameGame. Knowl.-Based Syst. 34, 3–11 (2012)

    Article  Google Scholar 

  • Sheppard, B.: World-championship-caliber Scrabble. Artif. Intell. 134(1–2), 241–275 (2002)

    Article  MATH  Google Scholar 

  • Sturtevant, N.R.: An analysis of UCT in multi-player games. ICGA J. 31(4), 195–208 (2008)

    Google Scholar 

  • Tak, M.J.W., Winands, M.H.M., Björnsson, Y.: N-Grams and the last-good-reply policy applied in general game playing. IEEE Trans. Comput. Intell. AI Games 4(2), 73–83 (2012)

    Article  Google Scholar 

  • Tesauro, G., Galperin, G.R.: On-line policy improvement using Monte-Carlo search. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 1068–1074. MIT Press, Cambridge, MA, USA (1997)

    Google Scholar 

  • Winands, M.H.M., Björnsson, Y., Saito, J.-T.: Monte Carlo Tree Search in Lines of Action. IEEE Trans. Comput. Intell. AI Games 2(4), 239–250 (2010)

    Article  Google Scholar 

  • Zhu, G., Lizotte, D., Hoey, J.: Scalable approximate policies for Markov decision process models of hospital elective admissions. Artif. Intell. Med. 61(1), 21–34 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Data Science and Knowledge Engineering, Maastricht University, Maastricht, The Netherlands

    Mark H. M. Winands

Authors
  1. Mark H. M. Winands
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark H. M. Winands .

Editor information

Editors and Affiliations

  1. Newton Lee Laboratories, LLC, Tujunga, California, USA

    Newton Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this entry

Cite this entry

Winands, M.H.M. (2015). Monte-Carlo Tree Search. In: Lee, N. (eds) Encyclopedia of Computer Graphics and Games. Springer, Cham. https://doi.org/10.1007/978-3-319-08234-9_12-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-08234-9_12-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Online ISBN: 978-3-319-08234-9

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation