Advertisement

Co-Evolutionary Optimization of Autonomous Agents in a Real-Time Strategy Game

  • Antonio Fernández-AresEmail author
  • Antonio M. Mora
  • Maribel García-Arenas
  • Juan Julián Merelo Guervós
  • Pablo García-Sánchez
  • Pedro A. Castillo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8602)

Abstract

This paper presents an approach based in an evolutionary algorithm, aimed to improve the behavioral parameters which guide the actions of an autonomous agent (bot) inside the real-time strategy game, Planet Wars. The work describes a co-evolutionary implementation of a previously presented method GeneBot, which yielded successful results, but focused in 4vs matches this time. Thus, there have been analyzed the effects of considering several individuals to be evolved (improved) at the same time in the algorithm, along with the use of three different fitness functions measuring the goodness of each bot in the evaluation. They are based in turns and position, and also in mathematical computations of linear regression and area regarding the number of ships belonging to the bot/individual to be evaluated. In addition, the variance of using an evolutionary algorithm with and without previous knowledge in the co-evolution phase is also studied, i.e., respectively using specific rivals to perform the evaluation, or just considering to this end individuals in the population being evolved. The aim of these co-evolutionary approaches are mainly two: first, reduce the computational time; and second find a robust fitness function to be used in the generation of evolutionary bots optimized for 4vs battles.

Keywords

Previous Knowledge Autonomous Agent Coevolutionary Approach Competitive Coevolution Team Tactic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fernández-Ares, A., Mora, A.M., Merelo, J.J., García-Sánchez, P., Fernandes, C.: Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: IEEE Congress on Evolutionary Computation (CEC 2011), pp. 2017–2024 (2011)Google Scholar
  2. 2.
    Mora, A.M., Fernández-Ares, A., Merelo, J.J., García-Sánchez, P., Fernandes, C.M.: Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J. Comput. Sci. Technol. 27(5), 1007–1023 (2012)CrossRefGoogle Scholar
  3. 3.
    Paredis, J.: Coevolutionary computation. Artif. Life 2(4), 355–375 (1995)CrossRefGoogle Scholar
  4. 4.
    Rosin, C.D., Belew, R.K.: New methods for competitive coevolution. Evol. Comput. 5(1), 1–29 (1997)CrossRefGoogle Scholar
  5. 5.
    Potter, M.A., De Jong, K.A.: A cooperative coevolutionary approach to function optimization. In: Davidor, Y., Männer, Reinhard, Schwefel, Hans-Paul (eds.) PPSN 1994. LNCS, vol. 866, pp. 249–257. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  6. 6.
    Ponsen, M., Munoz-Avila, H., Spronck, P., Aha, D.W.: Automatically generating game tactics through evolutionary learning. AI Magazine 27(3), 75–84 (2006)Google Scholar
  7. 7.
    Jang, S.H., Yoon, J.W., Cho, S.B.: Optimal strategy selection of non-player character on real time strategy game using a speciated evolutionary algorithm. In: Proceedings of the 5th IEEE Symposium on Computational Intelligence and Games (CIG 2009), pp. 75–79. IEEE Press, Piscataway (2009)Google Scholar
  8. 8.
    Fernández-Ares, A., García-Sánchez, P., Mora, A.M., Merelo, J.J.: Adaptive bots for real-time strategy games via map characterization. In: 2012 IEEE Conference on Computational Intelligence and Games, CIG 2012, pp. 417–721. IEEE (2012)Google Scholar
  9. 9.
    Pollack, J.B., Blair, A.D.: Co-evolution in the successful learning of backgammon strategy. Mach. Learn. 32, 225–240 (1998)CrossRefzbMATHGoogle Scholar
  10. 10.
    Runarsson, T.P., Lucas, S.M.: Co-evolution versus self-play temporal difference learning for acquiring position evaluation in smallboard go. IEEE Trans. Evol. Comput. 9(6), 628–640 (2005)CrossRefGoogle Scholar
  11. 11.
    Thompson, T., Levine, J., Wotherspoon, R.: Evolution of counter-strategies: Application of co-evolution to texas hold’em poker. In: IEEE Symposium on Computational Intelligence and Games (CIG 2008), pp. 16–22. IEEE (2008)Google Scholar
  12. 12.
    Togelius, J., Burrow, P., Lucas, S.M.: Multi-population competitive co-evolution of car racing controllers. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 4043–4050 (2007)Google Scholar
  13. 13.
    Avery, P.M., Michalewicz, Z.: Adapting to human game play. In: IEEE Symposium on Computational Intelligence and Games (CIG 2008), pp. 8–15 (2008)Google Scholar
  14. 14.
    Cook, M., Colton, S., Gow, J.: Initial Results from Co-operative Co-evolution for Automated Platformer Design. In: Di Chio, C., et al. (eds.) EvoApplications 2012. LNCS, vol. 7248, pp. 194–203. Springer, Heidelberg (2012)Google Scholar
  15. 15.
    Cardona, A., Togelius, J., Nelson, M.: Competitive coevolution in Ms. Pac-Man. In: IEEE Congress on Evolutionary Computation (CEC 2013), pp. 1403–1410 (2013)Google Scholar
  16. 16.
    Livingstone, D.: Coevolution in hierarchical ai for strategy games. In: IEEE Symposium on Computational Intelligence and Games (CIG 2005). IEEE (2005)Google Scholar
  17. 17.
    Smith, G., Avery, P., Houmanfar, R., Louis, S.: Using co-evolved rts opponents to teach spatial tactics. In: IEEE Symposium on Computational Intelligence and Games (CIG 2010), pp. 146–153 (2010)Google Scholar
  18. 18.
    Avery, P., Louis, S.: Coevolving team tactics for a real-time strategy game. In: IEEE Congress on Evolutionary Computation (CEC 2010), pp. 1–8 (2010)Google Scholar
  19. 19.
    Nogueira, M., Cotta, C., Fernández-Leiva, A.J.: An Analysis of Hall-of-Fame Strategies in Competitive Coevolutionary Algorithms for Self-Learning in RTS Games. In: Nicosia, G., Pardalos, P. (eds.) LION 7. LNCS, vol. 7997, pp. 174–188. Springer, Heidelberg (2013)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Antonio Fernández-Ares
    • 1
    Email author
  • Antonio M. Mora
    • 1
  • Maribel García-Arenas
    • 1
  • Juan Julián Merelo Guervós
    • 1
  • Pablo García-Sánchez
    • 1
  • Pedro A. Castillo
    • 1
  1. 1.Departamento de Arquitectura y Tecnología de ComputadoresUniversidad de GranadaGranadaSpain

Personalised recommendations