Abstract
Novelty search is a recent and promising approach to evolve neurocontrollers, especially to drive robots. The main idea is to maximize the novelty of behaviors instead of the efficiency. However, abandoning the efficiency objective(s) may be too radical in many contexts. In this paper, a Pareto-based multi-objective evolutionary algorithmis employed to reconcile novelty search with objective-based optimization by following a multiobjectivization process. Several multiobjectivizations based on behavioral novelty and on behavioral diversity are compared on a maze navigation task. Results show that the bi-objective variant “Novelty + Fitness” is better at fine-tuning behaviors than basic novelty search, while keeping a comparable number of iterations to converge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abbass, H.A., Deb, K.: Searching under multi-evolutionary pressures. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) EMO 2003. LNCS, vol. 2632, pp. 391–404. Springer, Heidelberg (2003)
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Communication of the ACM 18(9), 509–517 (1975)
Bui, L., Abbass, H.A., Branke, J.: Multiobjective optimization for dynamic environments. Proceedings of the IEEE-CEC 3, 2349–2356 (2005)
Deb, K.: Multi-objectives optimization using evolutionnary algorithms. Wiley, Chichester (2001)
Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: Proceedings of PPSN VI, pp. 849–858 (2000)
Doncieux, S., Mouret, J.B.: Single step evolution of robot controllers for sequential tasks. In: Proceedings of GECCO 2009, ACM, New York (2009)
Fonseca, C.M., Fleming, P.J.: Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In: Proceedings of the Fourth International Conference on Evolutionary Programming, pp. 416–423 (1993)
Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of the Second International Conference on Genetic Algorithms, pp. 148–154. Morgan Kaufmann, San Francisco (1987)
Gomez, F.: Sustaining diversity using behavioral information distance. In: Proceedings of GECCO 2009 (2009)
Greiner, D., Emperador, J., Winter, G., Galván, B.: Improving Computational Mechanics Optimum Design Using Helper Objectives: An Application in Frame Bar Structures. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 575–589. Springer, Heidelberg (2007)
Handl, J., Lovell, S.C., Knowles, J.: Multiobjectivization by decomposition of scalar cost functions. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 31–40. Springer, Heidelberg (2008)
de Jong, E.D., Watson, R.A., Pollack, J.B.: Reducing bloat and promoting diversity using multi-objective methods. In: Proceedings of GECCO 2001, pp. 11–18 (2001)
Knowles, J.D., Watson, R.A., Corne, D.W.: Reducing Local Optima in Single-Objective Problems by Multi-objectivization. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 268–282. Springer, Heidelberg (2001)
Lehman, J., Stanley, K.: Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of Artificial Life XI, pp. 329–336 (2008)
Mouret, J.B., Doncieux, S.: Incremental Evolution of Animats Behaviors as a Multi-objective Optimization. In: Asada, M., Hallam, J.C.T., Meyer, J.-A., Tani, J. (eds.) SAB 2008. LNCS (LNAI), vol. 5040, pp. 210–219. Springer, Heidelberg (2008)
Mouret, J.B., Doncieux, S.: Evolving modular neural-networks through exaptation. In: Proceedings of IEEE Congress on Evolutionary Computation, IEEE-CEC (2009)
Mouret, J.B., Doncieux, S.: Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In: Proceedings of IEEE-CEC (2009)
Mouret, J.B., Doncieux, S.: Using behavioral exploration objectives to solve deceptive problems in neuro-evolution. In: Proceedings of GECCO (2009)
Praditwong, K., Yao, X.: How well do multi-objective evolutionary algorithms scale to large problems. In: Proceedings of IEEE-CEC, pp. 3959–3966 (2007)
Purshouse, R.C., Fleming, P.J.: On the Evolutionary Optimization of Many Conflicting Objectives. IEEE Transactions on Evolutionary Computation 11(6), 770–784 (2007)
Risi, S., Vanderbleek, S.D., Hughes, C.E., Stanley, K.O.: How novelty search escapes the deceptive trap of learning to learn. In: Proceedings of GECCO (2009)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 102(2), 99–127 (2002)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Toffolo, A., Benini, E.: Genetic diversity as an objective in multi-objective evolutionary algorithms. Evolutionary Computation 11(2), 151–167 (2003)
Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the Strength Pareto Evolutionary Algorithm. In: Evolutionary Methods for Design, Optimisation and Control with Application to Industrial Problems (EUROGEN 2001), pp. 95–100 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mouret, JB. (2011). Novelty-Based Multiobjectivization. In: Doncieux, S., Bredèche, N., Mouret, JB. (eds) New Horizons in Evolutionary Robotics. Studies in Computational Intelligence, vol 341. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-18272-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-18272-3_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-18271-6
Online ISBN: 978-3-642-18272-3
eBook Packages: EngineeringEngineering (R0)