Abstract
One of the main motivations for the use of competitive coevolution systems is their ability to capitalise on arms races between competing species to evolve increasingly sophisticated solutions. Such arms races can, however, be hard to sustain, and it has been shown that the competing species often converge prematurely to certain classes of behaviours. In this paper, we investigate if and how novelty search, an evolutionary technique driven by behavioural novelty, can overcome convergence in coevolution. We propose three methods for applying novelty search to coevolutionary systems with two species: (i) score both populations according to behavioural novelty; (ii) score one population according to novelty, and the other according to fitness; and (iii) score both populations with a combination of novelty and fitness. We evaluate the methods in a predator-prey pursuit task. Our results show that novelty-based approaches can evolve a significantly more diverse set of solutions, when compared to traditional fitness-based coevolution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ashlock, D., Willson, S., Leahy, N.: Coevolution and tartarus. In: Congress on Evolutionary Computation, CEC, vol. 2, pp. 1618–1624. IEEE Press (2004)
Avery, P., Louis, S.: Coevolving team tactics for a real-time strategy game. In: Congress on Evolutionary Computation, CEC, pp. 1–8. IEEE Press (2010)
Chong, S.Y., Tino, P., Yao, X.: Relationship between generalization and diversity in coevolutionary learning. IEEE Transactions on Computational Intelligence and AI in Games 1(3), 214–232 (2009)
Cliff, D., Miller, G.F.: Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations. In: Morán, F., Merelo, J.J., Moreno, A., Chacon, P. (eds.) ECAL 1995. LNCS, vol. 929, pp. 200–218. Springer, Heidelberg (1995)
Dziuk, A., Miikkulainen, R.: Creating intelligent agents through shaping of coevolution. In: Congress on Evolutionary Computation, CEC, pp. 1077–1083. IEEE Press (2011)
Ebner, M., Watson, R.A., Alexander, J.: Coevolutionary dynamics of interacting species. In: Di Chio, C., et al. (eds.) EvoApplicatons 2010, Part I. LNCS, vol. 6024, pp. 1–10. Springer, Heidelberg (2010)
Ficici, S.G., Pollack, J.B.: Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In: Artificial Life, pp. 238–247. MIT Press (1998)
Gomes, J., Mariano, P., Christensen, A.L.: Avoiding convergence in cooperative coevolution with novelty search. In: International Conference on Autonomous Agents and Multi-agent Systems, AAMAS, pp. 1149–1156. IFAAMAS (2014)
Gomes, J., Urbano, P., Christensen, A.L.: Progressive minimal criteria novelty search. In: Pavón, J., Duque-Méndez, N.D., Fuentes-Fernández, R. (eds.) IBERAMIA 2012. LNCS, vol. 7637, pp. 281–290. Springer, Heidelberg (2012)
Gomes, J., Urbano, P., Christensen, A.: Evolution of swarm robotics systems with novelty search. Swarm Intelligence 7(2-3), 115–144 (2013)
Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation 19(2), 189–223 (2011)
Mouret, J.B., Doncieux, S.: Encouraging behavioral diversity in evolutionary robotics: An empirical study. Evolutionary Computation 20(1), 91–133 (2012)
Nolfi, S.: Co-evolving predator and prey robots. Adaptive Behavior 20(1), 10–15 (2012)
Nolfi, S., Floreano, D.: Coevolving predator and prey robots: Do arms races arise in artificial evolution? Artificial Life 4(4), 311–335 (1998)
Popovici, E., Bucci, A., Wiegand, R.P., De Jong, E.D.: Coevolutionary principles. In: Handbook of Natural Computing, pp. 987–1033. Springer (2012)
Reisinger, J., Bahçeci, E., Karpov, I., Miikkulainen, R.: Coevolving strategies for general game playing. In: Computational Intelligence and Games, pp. 320–327. IEEE Press (2007)
Rosin, C.D., Belew, R.K.: New methods for competitive coevolution. Evolutionary Computation 5(1), 1–29 (1997)
Watson, R.A., Pollack, J.B.: Coevolutionary dynamics in a minimal substrate. In: Genetic and Evolutionary Computation Conference, GECCO, pp. 702–709. Morgan Kaufmann (2001)
Yannakakis, G.N., Hallam, J.: Modeling and augmenting game entertainment through challenge and curiosity. International Journal on Artificial Intelligence Tools 16(6), 981–999 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Gomes, J., Mariano, P., Christensen, A.L. (2014). Novelty Search in Competitive Coevolution. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds) Parallel Problem Solving from Nature – PPSN XIII. PPSN 2014. Lecture Notes in Computer Science, vol 8672. Springer, Cham. https://doi.org/10.1007/978-3-319-10762-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-10762-2_23
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10761-5
Online ISBN: 978-3-319-10762-2
eBook Packages: Computer ScienceComputer Science (R0)