Abstract
Monte-Carlo tree search (MCTS) has been successfully applied to Chinese dark chess (CDC). In this paper, we study how to improve and analyze the playing strength of an MCTS-based CDC program, named DarkKnight, which won the CDC tournament in the 17th Computer Olympiad. We incorporate the three recent techniques, early playout terminations, implicit minimax backups, and quality-based rewards, into the program. For early playout terminations, playouts end when reaching states with likely outcomes. Implicit minimax backups use heuristic evaluations to help guide selections of MCTS. Quality-based rewards adjust rewards based on online collected information. Our experiments showed that the win rates against the original DarkKnight were 60.75 %, 70.90 % and 59.00 %, respectively for incorporating the three techniques. By incorporating all together, we obtained a win rate of 76.70 %.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)
Baier, H., Winands, M.H.: Monte-Carlo tree search and minimax hybrids with heuristic evaluation functions. In: Cazenave, T., Winands, M.H., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 45–63. Springer, Heidelberg (2014)
Björnsson, Y., Finnsson, H.: CadiaPlayer: a simulation-based general game player. IEEE Trans. Comput. Intell. AI Games 1(1), 4–15 (2009)
Borsboom, J., Saito, J.-T., Chaslot, G., Uiterwijk, J.: A comparison of Monte-Carlo methods for phantom go. In: Proceedings of BeNeLux Conference on Artificial Intelligence, Utrecht, The Netherlands, pp. 57–64 (2007)
Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)
Chang, H.-J., Hsu, T.-S.: A quantitative study of 2 × 4 Chinese dark chess. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 151–162. Springer, Heidelberg (2014)
Chen, B.-N., Hsu, T.-S.: Automatic generation of opening books for dark chess. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 221–232. Springer, Heidelberg (2014)
Chen, B.-N., Shen, B.-J., Hsu, T.-S.: Chinese dark chess. ICGA J. 33(2), 93–106 (2010)
Chen, J.-C., Lin, T.-Y., Chen, B.-N., Hsu, T.-S.: Equivalence classes in chinese dark chess endgames. IEEE Trans. Comput. Intell. AI Games 7(2), 109–122 (2015)
Chen, J.-C., Lin, T.-Y., Hsu, S.-C., Hsu, T.-S.: Design and implementation of computer Chinese dark chess endgame database. In: Proceeding of TCGA Workshop 2012, pp. 5–9, Hualien, Taiwan (2012) (in Chinese)
Enzenberger, M., Müller, M., Arneson, B., Segal, R.: Fuego: an open-source framework for board games and go engine based on Monte Carlo tree search. IEEE Trans. Comput. Intell. AI Games 2(4), 259–270 (2010)
Finnsson, H.: Generalized Monte-Carlo tree search extensions for general game playing. In: The Twenty-Sixth AAAI Conference on Artificial Intelligence, pp. 1550–1556, Toronto, Canada (2012)
Gelly, S., Silver, D.: Monte-Carlo tree search and rapid action value estimation in computer go. Artif. Intell. 175(11), 1856–1875 (2011)
Gelly, S., Wang, Y., Munos, R., Teytaud, O.: Modification of UCT with patterns in Monte-Carlo go. Technical report, HAL - CCSd - CNRS, France (2006)
Jouandeau, N.: Varying complexity in CHINESE DARK CHESS stochastic game. In: Proceeding of TCGA Workshop 2014, pp. 86, Taipei, Taiwan (2014)
Jouandeau, N., Cazenave, T.: Monte-Carlo tree reductions for stochastic games. In: Cheng, S.-M., Day, M.-Y. (eds.) TAAI 2014. LNCS, vol. 8916, pp. 228–238. Springer, Heidelberg (2014)
Jouandeau, N., Cazenave, T.: Small and large MCTS playouts applied to Chinese dark chess stochastic game. In: Cazenave, T., Winands, M.H., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 78–89. Springer, Heidelberg (2014)
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)
Lanctot, M., Winands, M.H.M., Pepels, T., Sturtevant, N.R.: Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In: 2014 IEEE Conference on Computational Intelligence and Games, CIG 2014, pp. 1–8 (2014)
Lin, Y.-S., Wu, I.-C., Yen, S.-J.: TAAI 2011 computer-game tournaments. ICGA J. 34(4), 248–250 (2011)
Lorentz, R.J.: Amazons discover Monte-Carlo. In: van den Herik, H., Xu, X., Ma, Z., Winands, M.H. (eds.) CG 2008. LNCS, vol. 5131, pp. 13–24. Springer, Heidelberg (2008)
Lorentz, R.: Early playout termination in MCTS. In: The 14th Conference on Advances in Computer Games (ACG2015), Leiden, The Netherlands (2015)
Lorentz, R., Horey, T.: Programming breakthrough. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 49–59. Springer, Heidelberg (2014)
Pepels, T., Tak, M.J., Lanctot, M., Winands, M.H.M.: Quality-based rewards for Monte-Carlo tree search simulations. In: 21st European Conference on Artificial Intelligence, Prague, Czech Republic (2014)
Saffidine, A., Jouandeau, N., Buron, C., Cazenave, T.: Material symmetry to partition endgame tables. In: van den Herik, H., Iida, H., Plaat, A. (eds.) CG 2013. LNCS, vol. 8427, pp. 187–198. Springer, Heidelberg (2014)
Su, T.-C., Yen, S.-J., Chen, J.-C., Wu, I.-C.: TAAI 2012 computer game tournaments. ICGA J. 37(1), 33–35 (2014)
Theory of computer games, a course in National Taiwan University taught by Tsu, T.-S. http://www.iis.sinica.edu.tw/~tshsu/tcg/index.html
Tseng, W.-J., Chen, J.-C., Chen, L.-P., Yen, S.-J., Wu, I.-C.: TCGA 2013 computer game tournament report. ICGA J. 36(3), 166–168 (2013)
Van Lishout, F., Chaslot, G., Uiterwijk, J.W.: Monte-Carlo tree search in Backgammon. In: Computer Games Workshop, pp. 175–184, Amsterdam, The Netherlands (2007)
Winands, M.H.M., Björnsson, Y., Saito, J.-T.: Monte Carlo tree search in lines of action. IEEE Trans. Comput. Intell. AI Games 2(4), 239–250 (2010)
Winands, M.H., Björnsson, Y., Saito, J.-T.: Monte-Carlo tree search solver. In: van den Herik, H., Xu, X., Ma, Z., Winands, M.H. (eds.) CG 2008. LNCS, vol. 5131, pp. 25–36. Springer, Heidelberg (2008)
Yang, J.-K., Su, T.-C., Wu, I.-C.: TCGA 2012 computer game tournament report. ICGA J. 35(3), 178–180 (2012)
Yen, S.-J., Chou, C.-W., Chen, J.-C., Wu, I.-C., Kao, K.-Y.: Design and implementation of Chinese dark chess programs. IEEE Trans. Comput. Intell. AI Games 7(1), 66–74 (2015)
Yen, S.-J., Chen, J.-C., Chen, B.-N., Tseng, W.-J.: DarkKnight wins Chinese dark chess tournament. ICGA J. 36(3), 175–176 (2013)
Yen, S.-J., Su, T.-C., Wu, I.-C.: The TCGA 2011 computer-games tournament. ICGA J. 34(2), 108–110 (2011)
Acknowledgements
The authors would like to thank the Ministry of Science and Technology of the Republic of China (Taiwan) for financial support of this research under contract numbers MOST 102-2221-E-009-069-MY2, 102-2221-E-009-080-MY2, 104-2221-E-009-127-MY2, and 104-2221-E-009-074-MY2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Hsueh, CH., Wu, IC., Tseng, WJ., Yen, SJ., Chen, JC. (2015). Strength Improvement and Analysis for an MCTS-Based Chinese Dark Chess Program. In: Plaat, A., van den Herik, J., Kosters, W. (eds) Advances in Computer Games. ACG 2015. Lecture Notes in Computer Science(), vol 9525. Springer, Cham. https://doi.org/10.1007/978-3-319-27992-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-27992-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27991-6
Online ISBN: 978-3-319-27992-3
eBook Packages: Computer ScienceComputer Science (R0)