Abstract
In most computer games as in life, the outcome of a match is uncertain due to several reasons: the characters or assets appear in different initial positions or the response of the player, even if programmed, is not deterministic; different matches will yield different scores. That is a problem when optimizing a game-playing engine: its fitness will be noisy, and if we use an evolutionary algorithm it will have to deal with it. This is not straightforward since there is an inherent uncertainty in the true value of the fitness of an individual, or rather whether one chromosome is better than another, thus making it preferable for selection. Several methods based on implicit or explicit average or changes in the selection of individuals for the next generation have been proposed in the past, but they involve a substantial redesign of the algorithm and the software used to solve the problem. In this paper we propose new methods based on incremental computation (memory-based) or fitness average or, additionally, using statistical tests to impose a partial order on the population; this partial order is considered to assign a fitness value to every individual which can be used straightforwardly in any selection function. Tests using several hard combinatorial optimization problems show that, despite an increased computation time with respect to the other methods, both memory-based methods have a higher success rate than implicit averaging methods that do not use memory; however, there is not a clear advantage in success rate or algorithmic terms of one method over the other.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mora, A.M., Fernández-Ares, A., Merelo-Guervós, J.J., García-Sánchez, P., Fernandes, C.M.: Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J. Comput. Sci. Technol. 27, 1007–1023 (2012)
García-Ortega, R.H., García-Sánchez, P., Merelo, J.J.: Emerging archetypes in massive artificial societies for literary purposes using genetic algorithms. ArXiv e-prints (2014). http://adsabs.harvard.edu/abs/2014arXiv1403.3084G
Castillo, P.A., González, J., Merelo-Guervós, J.J., Prieto, A., Rivas, V., Romero, G.: G-Prop-III: global optimization of multilayer perceptrons using an evolutionary algorithm. In: GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, p. 942 (1999)
Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments—a survey. IEEE Trans. Evol. Comput. 9, 303–317 (2005). (cited By (since 1996) 576)
Merelo, J.J., Castillo, P.A., Mora, A., Fernández-Ares, A., Esparcia-Alcázar, A.I., Cotta, C., Rico, N.: Studying and tackling noisy fitness in evolutionary design of game characters. In: Rosa, A., Merelo, J. J., Filipe, J. (eds.) ECTA 2014—Proceedings of the International Conference on Evolutionary Computation Theory and Applications, pp. 76–85 (2014)
Merelo-Guervós, J.J., Romero, G., García-Arenas, M., Castillo, P.A., Mora, A.M., Jiménez-Laredo, J.L.: Implementation matters: programming best practices for evolutionary algorithms. [30] 333–340
Qian, C., Yu, Y., Zhou, Z.H.: Analyzing evolutionary optimization in noisy environments. CoRR (2013). abs/1311.4987
Bhattacharya, M., Islam, R., Mahmood, A.: Uncertainty and evolutionary optimization: a novel approach. In: 2014 IEEE 9th Conference on Industrial Electronics and Applications (ICIEA), pp. 988–993 (2014)
Qian, C., Yu, Y., Jin, Y., Zhou, Z.H.: On the effectiveness of sampling for evolutionary optimization in noisy environments. In: Bartz-Beielstein, T., Branke, J., Filipic, B., Smith, J. (eds.) Parallel Problem Solving from Nature PPSN XIII. Volume 8672 of Lecture Notes in Computer Science, pp. 302–311. Springer (2014)
Mora, A.M., Montoya, R., Merelo, J.J., Sánchez, P.G., Castillo, P.A., Laredo, J.L.J., Martínez, A.I., Espacia, A.: Evolving bots ai in unreal. In: di Chio C., et al. (ed.) Applications of Evolutionary Computing, Part I. Volume 6024 of Lecture Notes in Computer Science, pp. 170–179. Springer, Istanbul (2010)
Liberatore, F., Mora, A.M., Castillo, P.A., Merelo-Guervós, J.J.: Evolving evil: optimizing flocking strategies through genetic algorithms for the ghost team in the game of Ms. Pac-Man. In: Camacho, D., Gómez-Martín, M.A., González-Calero, P.A. (eds.) Proceedings 1st Congreso de la Sociedad Española para las Ciencias del Videojuego, CoSECivi 2014, Barcelona, Spain, June 24, 2014. Volume 1196 of CEUR Workshop Proceedings, pp. 111–116. CEUR-WS.org (2014)
Merelo-Guervós, J.J., Prieto, A., Morán, F.: 4. In: Optimization of Classifiers Using Genetic Algorithms, pp. 91–108. MIT press (2001). ISBN: 0262162016. http://geneura.ugr.es/pub/papers/g-lvq-book.ps.gz
Miller, B.L., Goldberg, D.E.: Genetic algorithms, selection schemes, and the varying effects of noise. Evol. Comput. 4, 113–131 (1996)
Jun-hua, L., Ming, L.: An analysis on convergence and convergence rate estimate of elitist genetic algorithms in noisy environments. Opt. Int. J. Light Electron Opt. 124, 6780–6785 (2013)
Friedrich, T., Kötzing, T., Krejca, M., Sutton, A.M.: The Benefit of sex in noisy evolutionary search. ArXiv e-prints (2015)
Aizawa, A.N., Wah, B.W.: Scheduling of genetic algorithms in a noisy environment. Evol. Comput. 2, 97–122 (1994)
Costa, A., Vargas, P., Tinós, R.: Using explicit averaging fitness for studying the behaviour of rats in a maze. Adv. Artif. Life ECAL 12, 940–946 (2013)
Liu, J., St-Pierre, D.L., Teytaud, O.: A mathematically derived number of resamplings for noisy optimization. In: Proceedings of the 2014 Conference Companion on Genetic and Evolutionary Computation Companion. GECCO Comp ’14, New York, pp. 61–62. ACM, NY (2014)
Rada-Vilela, J., Johnston, M., Zhang, M.: Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems. In: Swarm and Evolutionary Computation (2014, in press)
Rakshit, P., Konar, A., Nagar, A.: Artificial bee colony induced multi-objective optimization in presence of noise. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 3176–3183 (2014)
Rudolph, G.: A partial order approach to noisy fitness functions. Proc. IEEE Conf. Evol. Comput. ICEC 1, 318–325 (2001)
Cauwet, M.L., Liu, J., Teytaud, O., et al.: Algorithm portfolios for noisy optimization: compare solvers early. In: Learning and Intelligent Optimization Conference (2014)
Lara-Cabrera, R., Cotta, C., Fernández-Leiva, A.J.: On balance and dynamism in procedural content generation with self-adaptive evolutionary algorithms. Nat. Comput. 13, 157–168 (2014)
Fernández-Ares, A., Mora, A.M., Guervós, J.J.M., García-Sánchez, P., Fernandes, C.M.: Optimizing strategy parameters in a game bot [30] 325–332
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1, 80–83 (1945)
Merelo-Guervós, J.J.: Using a Wilcoxon-test based partial order for selection in evolutionary algorithms with noisy fitness. Technical Report, GeNeura Group, University of Granada (2014). http://dx.doi.org/10.6084/m9.figshare.974598
Merelo-Guervós, J.J., Castillo, P.A., Alba, E.: Algorithm::Evolutionary, a flexible Perl module for evolutionary computation. Soft Comput. 14, 1091–1109 (2010). http://sl.ugr.es/000K
Deb, K., Goldberg, D.E.: Analyzing deception in trap functions. FOGA 2, 98–108 (1992)
Goldberg, D.E., Deb, K., Horn, J.: Massive multimodality, deception, and genetic algorithms. In: Männer, R., Manderick, B. (eds.) Parallel Problem Solving from Nature, vol. 2, pp. 37–48. Elsevier, Amsterdam (1992)
Cabestany, J., Rojas, I., Caparrós, G.J., (eds.): Advances in Computational Intelligence—11th International Work-Conference on Artificial Neural Networks, IWANN 2011, Torremolinos-Málaga, Spain, June 8-10, 2011, Proceedings, Part II. IWANN (2). Volume 6692 of Lecture Notes in Computer Science. Springer (2011)
Acknowledgments
This work has been supported in part by project EphemeCH (TIN2014-56494-C4-3-P and TIN2014-56494-C4-1-P) and DNEMESIS (P10-TIC-6083). The authors would like to thank the FEDER of European Union for financial support via project “Sistema de Información y Predicción de bajo coste y autónomo para conocer el Estado de las Carreteras en tiempo real mediante dispositivos distribuidos”(SIPEsCa) of the“Programa Operativo FEDER de Andalucía 2007-2013”. We also thank all Agency of Public Works of Andalusia Regional Government staff and researchers for their dedication and professionalism.
Our research group is committed to Open Science and the writing and development of this paper has been carried out in GitHub at this address https://github.com/JJ/noisy-fitness-eas. We encourage you to visit, comment and to do all kind of suggestions or feature requests.
We would like also to thank Marc Schoenauer for the suggestion that has been put to test in this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Merelo, J.J. et al. (2016). A Statistical Approach to Dealing with Noisy Fitness in Evolutionary Algorithms. In: Merelo, J.J., Rosa, A., Cadenas, J.M., Dourado, A., Madani, K., Filipe, J. (eds) Computational Intelligence. IJCCI 2014. Studies in Computational Intelligence, vol 620. Springer, Cham. https://doi.org/10.1007/978-3-319-26393-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-26393-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26391-5
Online ISBN: 978-3-319-26393-9
eBook Packages: EngineeringEngineering (R0)