Skip to main content
SpringerLink
Log in

Investigations with Monte Carlo Tree Search for Finding Better Multivariate Horner Schemes

  • Conference paper
  • First Online:
Agents and Artificial Intelligence (ICAART 2013)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 449))

Included in the following conference series:

Abstract

After a computer chess program had defeated the human World Champion in 1997, many researchers turned their attention to the oriental game of Go. It turned out that the minimax approach, so successful in chess, did not work in Go. Instead, after some ten years of intensive research, a new method was developed: MCTS (Monte Carlo Tree Search), with promising results. MCTS works by averaging the results of random play-outs. At first glance it is quite surprising that MCTS works so well. However, deeper analysis revealed the reasons.

The success of MCTS in Go caused researchers to apply the method to other domains. In this article we report on experiments with MCTS for finding improved orderings for multivariate Horner schemes, a basic method for evaluating polynomials. We report on initial results, and continue with an investigation into two parameters that guide the MCTS search. Horner’s rule turns out to be a fruitful testbed for MCTS, allowing easy experimentation with its parameters. The results reported here provide insight into how and why MCTS works. It will be interesting to see if these insights can be transferred to other domains, for example, back to Go.

Parts of this work have appeared in a keynote speech by the first author at the International Conference on Agents and Artifical Intelligence ICAART 2013 in Barcelona under the title “Connecting Sciences.” These parts are reprinted with permission by the publisher.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The 7-5 resultant has 11380 terms and 14 variables, the 7-6 resultant has 43166 terms and 15 variables.

References

  1. Allis, V.: Searching for Solutions in Games and Artificial Intelligence. (Ph.D. thesis), University of Limburg, Maastricht, The Netherlands (1994)

    Google Scholar 

  2. Althöfer, I.: The origin of dynamic komi. ICGA J. 35(1), 31–34 (2012)

    Google Scholar 

  3. Aoyama, T., Hayakawa, M., Kinoshita, T., Nio, M.: Tenth-Order QED Lepton Anomalous Magnetic Moment – Eighth-Order Vertices Containing a Second-Order Vacuum Polarization. e-Print: arXiv:1110.2826 [hep-ph] (2011)

  4. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2), 235–256 (2002)

    Article  MATH  Google Scholar 

  5. Bouzy, B., Helmstetter, B.: Monte-Carlo Go developments. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) ACG-10. IFIP, vol. 135, pp. 159–174. Springer, Boston (2003)

    Google Scholar 

  6. Bouzy, B., Métivier, M., Pellier, D.: MCTS experiments on the voronoi game. In: van den Herik, H.J., Plaat, A. (eds.) ACG 2011. LNCS, vol. 7168, pp. 96–107. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  7. Brügmann, B.: Monte-Carlo Go. In: AAAI Fall symposium on Games: Playing, Planning, and Learning (1993). http://www.cgl.ucsf.edu/go/Programs/Gobble.html

  8. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo Tree Search Methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)

    Article  Google Scholar 

  9. Ceberio, M., Kreinovich, V.: Greedy algorithms for optimizing multivariate Horner schemes. ACM SIGSAM Bull. 38, 8–15 (2004)

    Article  Google Scholar 

  10. Chaslot, G., Saito, J.-T., Bouzy, B., Uiterwijk, J.W.H.M., van den Herik, H.J.: Monte-Carlo strategies for computer Go. In: Proceedings of the 18th BeNeLux Conference on Articial Intelligence, pp. 83–90 (2006)

    Google Scholar 

  11. Chaslot, G.M.J.-B., de Jong, S., Saito, J.-T., Uiterwijk, J.W.H.M.: Monte-Carlo tree search in production management problems. In: Proceedings of the BeNeLux Conference on Artificial Intelligence, Namur, Belgium, pp. 91–98 (2006)

    Google Scholar 

  12. Chaslot, G.M.J-B., Winands, M.H.M., Uiterwijk, J.W.H.M., van den Herik, H.J., Bouzy, B.: Progressive strategies for Monte-Carlo tree search. In: Wang, P., et al. (eds.) Proceedings of the 10th Joint Conference on Information Sciences (JCIS 2007), pp. 655–661. World Scientific Publishing Co., Pte. Ltd. (2007); New Mathematics and Natural Computation, vol. 4(3), pp. 343–357 (2008)

    Google Scholar 

  13. Chaslot, G.M.J-B., Bakkes, S., Szita, I., Spronck, P.: Monte-Carlo tree search: a new framework for game AI. In: Mateas, M., Darken, C. (eds.) Proceedings of the 4th Artificial Intelligence and Interactive Digital Entertainment Conference. AAAI Press, Menlo Park (2008)

    Google Scholar 

  14. Chaslot, G.M.J.-B., Hoock, J.-B., Rimmel, A., Teytaud, O., Lee, C.-S., Wang, M.-H., Tsai, S.-R., Hsu, S.-C.: Human-computer go revolution 2008. ICGA J. 31(3), 179–185 (2008)

    Google Scholar 

  15. Chinchalkar, S.: An upper bound for the number of reachable positions. ICCA J. 19(3), 181–182 (1996)

    Google Scholar 

  16. Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  17. Donkers, J.H.L.M., van den Herik, H.J., Uiterwijk, J.W.H.M.: Selecting evaluation functions in opponent model search. Theoret. Comput. Sci. (TCS) 349(2), 245–267 (2005)

    Article  MATH  Google Scholar 

  18. de Groot, A.D.: Het denken van den schaker, Ph. D. thesis in dutch (1946); translated in 1965 as “Thought and Choice in chess”, Mouton Publishers, The Hague (2nd edn. 1978). Freely available as e-book from Google (1946)

    Google Scholar 

  19. Enzenberger, M.: Evaluation in go by a neural network using soft segmentation. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) Advances in Computer Games. IFIP, vol. 135, pp. 97–108. Springer, Boston (2003)

    Google Scholar 

  20. Horner, W.G.: A new method of solving numerical equations of all orders, by continuous approximation. Philos. Trans. (R. Soc. Lond.) 109, 308–335 (1819); Reprinted with appraisal in Smith, D.E.: A Source Book in Mathematics, McGraw-Hill (1929); Dover reprint, vol. 2 (1959)

    Google Scholar 

  21. Gelly, S., Wang, Y., Munos, R., Teytaud, O.: Modification of UCT with patterns in monte-carlo go. Inst. Nat. Rech. Inform. Auto. (INRIA), Paris, Technical report (2006)

    Google Scholar 

  22. Hartmann, D.: How to extract relevant knowledge from grandmaster games. Part 1: Grandmaster have insights–the problem is what to incorporate into practical problems. ICCA J. 10(1), 14–36 (1987)

    Google Scholar 

  23. van den Herik, H.J.: Informatica en het Menselijk Blikveld. Inaugural address Rijksuniversiteit Limburg, Maastricht, The Netherlands (1988)

    Google Scholar 

  24. Junghanns, A.: Are there practical alternatives to alpha-beta? ICCA J. 21(1), 14–32 (1998)

    Google Scholar 

  25. Kocsis, L., Szepesvàri, C.: Bandit based monte-carlo planning. In: European Conference on Machine Learning, pp. 282–293. Springer, Berlin, Germany (2006)

    Google Scholar 

  26. Kuipers, J., Vermaseren, J.A.M., Plaat, A., van den Herik, H.J.: Improving multivariate Horner schemes with Monte Carlo tree search, July 2012. arXiv 1207.7079

  27. Kuipers, J., Ueda, T., Vermaseren, J.A.M., Vollinga, J.: FORM version 4.0 (2012) (preprint). arXiv:1203.6543

  28. Kloetzer, J.: Monte-Carlo opening books for amazons. In: van den Herik, H.J., Iida, H., Plaat, A. (eds.) CG 2010. LNCS, vol. 6515, pp. 124–135. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  29. Landis, E.M., Yaglom, I.M.: About aleksandr semenovich kronrod. Russ. Math. Surv. 56, 993–1007 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  30. Leiserson, C.E., Li, L., Maza, M.M., Xie, Y.: Efficient evaluation of large polynomials. In: Fukuda, K., Hoeven, J., Joswig, M., Takayama, N. (eds.) ICMS 2010. LNCS, vol. 6327, pp. 342–353. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  31. Lorentz, R.: Experiments with monte carlo tree search in the game of havannah. ICGA J. 34(3), 140–149 (2011)

    MathSciNet  Google Scholar 

  32. Lorentz, R.J.: An MCTS program to play einstein würfelt nicht!. In: van den Herik, H.J., Plaat, A. (eds.) ACG 2011. LNCS, vol. 7168, pp. 52–59. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  33. Moch, S.-O., Vermaseren, J.A.M., Vogt, A.: Nucl. Phys. B688, B691, 101–134, 129–181 (2004); B724, 3–182 (2005)

    Google Scholar 

  34. Müller, M.: Computer Go. Artif. Intell. 134(1–2), 145–179 (2002)

    Article  MATH  Google Scholar 

  35. Pearl, J.: Asymptotical properties of minimax trees and game searching procedures. Artif. Intell. 14(2), 113–138 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  36. Pearl, J.: Heuristics Intelligent Search Strategies for Computer Problem Solving. Addison-WesleyPublishing Co, Reading (1984)

    Google Scholar 

  37. Plaat, A., Schaeffer, J., Pijls, W., de Bruin, A.: Best-First Fixed-Depth Minimax Algorithms. Artificial Intelligence 87(1–2), 255–293 (1996)

    Article  MathSciNet  Google Scholar 

  38. Rivest, R.: Game-tree searching by min-max approximation. Artif. Intell. 34(1), 77–96 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  39. Rosin, C.D.: Nested rollout policy adaptation for monte carlo tree search. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-2011, pp. 649–654 (2011)

    Google Scholar 

  40. Saito, J.-T., Chaslot, G.M.J.-B., Uiterwijk, J.W.H.M., van den Herik, H.J.: Monte-carlo proof-number search for computer go. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 50–61. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  41. Schadd, M.P.D., Winands, M.H.M., van den Herik, H.J., Chaslot, G.M.J.-B., Uiterwijk, J.W.H.M.: Single-player monte-carlo tree search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 1–12. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  42. Stockman, G.C.: A minimax algorithm better than alpha-beta? Artif. Intell. 12(2), 179–196 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  43. Szita, I., Chaslot, G., Spronck, P.: Monte-Carlo tree search in settlers of catan. In: van den Herik, H.J., Spronck, P. (eds.) ACG 2009. LNCS, vol. 6048, pp. 21–32. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  44. van der Werf, E.C.D., van den Herik, H.J., Uiterwijk, J.W.H.M.: Learning to score final positions in the game of Go. Theoret. Comput. Sci. 349(2), 168–183 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  45. van der Werf, E.C.D., Winands, M.H.M., van den Herik, H.J., Uiterwijk, J.W.H.M.: Learning to predict Life and Death from Go game records. Inf. Sci. 175(4), 258–272 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tilburg Center for Cognition and Communication, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands

    H. Jaap van den Herik & Aske Plaat

  2. Nikhef Theory Group, Science Park 105, 1098 XG, Amsterdam, The Netherlands

    Jan Kuipers & Jos A. M. Vermaseren

Authors
  1. H. Jaap van den Herik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Kuipers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jos A. M. Vermaseren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aske Plaat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to H. Jaap van den Herik .

Editor information

Editors and Affiliations

  1. INSTICC, Setúbal, Portugal

    Joaquim Filipe

  2. Instituto de Telecomunicações, Lisbon, Portugal

    Ana Fred

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

van den Herik, H.J., Kuipers, J., Vermaseren, J.A.M., Plaat, A. (2014). Investigations with Monte Carlo Tree Search for Finding Better Multivariate Horner Schemes. In: Filipe, J., Fred, A. (eds) Agents and Artificial Intelligence. ICAART 2013. Communications in Computer and Information Science, vol 449. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44440-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44440-5_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44439-9

  • Online ISBN: 978-3-662-44440-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation