Skip to main content
SpringerLink
Log in

Online Evolution for Multi-action Adversarial Games

  • Conference paper
  • First Online:
Applications of Evolutionary Computation (EvoApplications 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9597))

Included in the following conference series:

Abstract

We present Online Evolution, a novel method for playing turn-based multi-action adversarial games. Such games, which include most strategy games, have extremely high branching factors due to each turn having multiple actions. In Online Evolution, an evolutionary algorithm is used to evolve the combination of atomic actions that make up a single move, with a state evaluation function used for fitness. We implement Online Evolution for the turn-based multi-action game Hero Academy and compare it with a standard Monte Carlo Tree Search implementation as well as two types of greedy algorithms. Online Evolution is shown to outperform these methods by a large margin. This shows that evolutionary planning on the level of a single move can be very effective for this sort of problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/njustesen/hero-aicademy.

  2. 2.

    http://www.robotentertainment.com/games/heroacademy/.

References

  1. Branavan, S., Silver, D., Barzilay, R.: Non-linear monte-carlo search in civilization ii. In: AAAI Press/International Joint Conferences on Artificial Intelligence (2011)

    Google Scholar 

  2. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S., et al.: A survey of monte carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)

    Article  Google Scholar 

  3. Cardamone, L., Loiacono, D., Lanzi, P.L.: Evolving competitive car controllers for racing games with neuroevolution. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pp. 1179–1186. ACM (2009)

    Google Scholar 

  4. Chaslot, G., Bakkes, S., Szita, I., Spronck, P.: Monte-carlo tree search: a new framework for game ai. In: AIIDE (2008)

    Google Scholar 

  5. Churchill, D., Buro, M.: Portfolio greedy search and simulation for large-scale combat in starcraft. In: 2013 IEEE Conference on Computational Intelligence in Games (CIG), pp. 1–8. IEEE (2013)

    Google Scholar 

  6. Elias, G.S., Garfield, R., Gutschera, K.R.: Characteristics of Games. MIT Press, Cambridge (2012)

    Google Scholar 

  7. Gelly, S., Wang, Y.: Exploration exploitation in go: uct for monte-carlo go. In: NIPS: Neural Information Processing Systems Conference On-line trading of Exploration and Exploitation Workshop (2006)

    Google Scholar 

  8. Glover, F., Laguna, M.: Tabu Search*. Springer, New York (2013)

    MATH  Google Scholar 

  9. Helmbold, D.P., Parker-Wood, A.: All-moves-as-first heuristics in monte-carlo go. In: IC-AI, pp. 605–610 (2009)

    Google Scholar 

  10. Justesen, N.: Artificial intelligence for hero academy. Master’s thesis, IT University of Copenhagen (2015)

    Google Scholar 

  11. Justesen, N., Tillman, B., Togelius, J., Risi, S.: Script-and cluster-based uct for starcraft. In: 2014 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1–8. IEEE (2014)

    Google Scholar 

  12. Kozelek, T.: Methods of mcts and the game arimaa. Charles University, Prague, Faculty of Mathematics and Physics (2009)

    Google Scholar 

  13. Levine, J., Congdon, C.B., Ebner, M., Kendall, G., Lucas, S.M., Miikkulainen, R., Schaul, T., Thompson, T., Lucas, S.M., Mateas, M., et al.: General video game playing. Artif. Comput. Intell. Games 6, 77–83 (2013)

    Google Scholar 

  14. Mahfoud, S.W.: Niching methods for genetic algorithms. Urbana 51(95001), 62–94 (1995)

    Google Scholar 

  15. Neumann, J.V.: Zur Theorie der Gesellschaftsspiele. Math. Ann. 100(1), 295–320 (1928)

    Article  MathSciNet  MATH  Google Scholar 

  16. Perez, D., Rohlfshagen, P., Lucas, S.M.: Monte-Carlo tree search for the physical travelling salesman problem. In: Di Chio, C., et al. (eds.) EvoApplications 2012. LNCS, vol. 7248, pp. 255–264. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  17. Perez, D., Samothrakis, S., Lucas, S., Rohlfshagen, P.: Rolling horizon evolution versus tree search for navigation in single-player real-time games. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 351–358. ACM (2013)

    Google Scholar 

  18. Rosin, C.D., Belew, R.K.: New methods for competitive coevolution. Evol. Comput. 5(1), 1–29 (1997)

    Article  Google Scholar 

  19. Shannon, C.E.: XXII. programming a computer for playing chess. Lond. Edinb. Dublin Philos. Mag. J. Sci. 41(314), 256–275 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  20. Togelius, J., Karakovskiy, S., Baumgarten, R.: The 2009 mario ai competition. In: 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2010)

    Google Scholar 

  21. Togelius, J., Karakovskiy, S., Koutník, J., Schmidhuber, J.: Super mario evolution. In: IEEE Symposium on Computational Intelligence and Games, 2009, CIG 2009, pp. 156–161. IEEE (2009)

    Google Scholar 

  22. Zhou, A., Qu, B.Y., Li, H., Zhao, S.Z., Suganthan, P.N., Zhang, Q.: Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol. Comput. 1(1), 32–49 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IT University of Copenhagen, Copenhagen, Denmark

    Niels Justesen

  2. Lund University, Lund, Sweden

    Tobias Mahlmann

  3. New York University, New York, USA

    Julian Togelius

Authors
  1. Niels Justesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tobias Mahlmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julian Togelius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Mahlmann .

Editor information

Editors and Affiliations

  1. Politecnico di Torino, Turin, Italy

    Giovanni Squillero

  2. Aalborg University, Copenhagen, Denmark

    Paolo Burelli

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Justesen, N., Mahlmann, T., Togelius, J. (2016). Online Evolution for Multi-action Adversarial Games. In: Squillero, G., Burelli, P. (eds) Applications of Evolutionary Computation. EvoApplications 2016. Lecture Notes in Computer Science(), vol 9597. Springer, Cham. https://doi.org/10.1007/978-3-319-31204-0_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31204-0_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31203-3

  • Online ISBN: 978-3-319-31204-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation