Skip to main content
SpringerLink
Log in

On Applying Neuroevolutionary Methods to Complex Robotic Tasks

  • Conference paper
Book cover New Horizons in Evolutionary Robotics

Part of the book series: Studies in Computational Intelligence ((SCI,volume 341))

Abstract

In this paper, we describe possible methods of solving two problems encountered in evolutionary robotics, while applying neuroevolutionary methods to evolve controllers for complex robotic tasks. The first problem is the large number of evaluations required to obtain a solution. We propose that this problem can be addressed by accelerating neuroevolutionary methods using a Kalman filter. The second problem is the difficulty of obtaining a desirable solution that results from the difficulty of defining an appropriate fitness function for a complex robotic task. The solution towards this problem is to apply the principles of behavior based systems to decompose the solution space into smaller subsolutions with lower number of intrinsic dimensions, and incrementally modify the fitness function. We present two case studies towards the solutions to the stated problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Altug, E., Ostrowski, J.P., Taylor, C.J.: Quadrotor control using dual camera visual feedback. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2003, pp. 4294–4299 (2003)

    Google Scholar 

  2. Arkin, R.C.: Behaviour-Based Robotics. MIT Press, Massachusetts (1998)

    Google Scholar 

  3. Bongard, J.: Behavior chaining: Incremental behavior integration for evolutionary robotics. Artificial Life (2008)

    Google Scholar 

  4. Botvinick, M., Niv, Y., Barto, A.C.: Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition 113, 262–280 (2008)

    Article  Google Scholar 

  5. Brooks, R.A.: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation 2(1), 14–23 (1986)

    Article  Google Scholar 

  6. Calabretta, R., Nolfi, S., Parisi, D., Wagner, G.P.: Duplication of modules facilitates the evolution of functional specialization. Artificial Life 6, 69–84 (2000)

    Article  Google Scholar 

  7. Dürr, P., Mattiussi, C., Floreano, D.: Neuroevolution with analog genetic encoding. In: Proceedings of the 9th Conference on Parallel Problem Solving from Nature (PPSN IX), pp. 671–680 (2006)

    Google Scholar 

  8. Feldkamp, L.A., Puskorius, G.V.: Training controllers for robustness: multi-stream DEKF. In: IEEE International Conference on Neural Networks, pp. 2377–2382 (1994)

    Google Scholar 

  9. Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)

    MathSciNet  Google Scholar 

  10. Gomez, F.J., Miikkulainen, R.: Incremental evolution of complex general behavior. Adaptive Behavior 5, 317–342 (1997)

    Article  Google Scholar 

  11. Gould, S., Vrba, E.: Exaptation; a missing term in the science of form. Paleobiology 8(1), 4–15 (1982)

    Google Scholar 

  12. Gray, J.E., Murray, W.: A derivation of an analytic expression for the tracking index for alpha-beta-gamma filter. IEEE Transactions on Aerospace and Electronic Systems 29(3), 1064–1065 (1993)

    Article  Google Scholar 

  13. Gruau, F.: Neural Network Synthesis Using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon, Laboratoire de l’Informatique du Parallelisme, France (January 1994)

    Google Scholar 

  14. Gruau, F., Whitley, D., Pyeatt, L.: A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza, J.R., Goldberg, D.E., Fogel, D.B., Riolo, R.L. (eds.) Genetic Programming: Proceedings of the First Annual Conference, Standford University, CA, USA, pp. 81–89. MIT Press, Cambridge (1996)

    Google Scholar 

  15. Gruau, F., Quatramaran, K.: Cellular encoding for interactive evolutionary robotics. Technical report, Amsterdam, The Netherlands (1996)

    Google Scholar 

  16. Hamel, T., Mahony, R., Lozano, R., Ostrowski, J.: Dynamic modelling and configuration stabilization for an X4-flyer. In: International Federation of Automatic Control Symposium, IFAC (2002)

    Google Scholar 

  17. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)

    Article  Google Scholar 

  18. Haykin, S.: Kalman Filering and Neural Networks. Wiley, Chichester (2001)

    Book  Google Scholar 

  19. Heidrich-Meisner, V., Igel, C.: Neuroevolution strategies for episodic reinforcement learning. Journal of Algorithms (2009), doi:10.1016/j.jalgor.2009.04.002

    Google Scholar 

  20. Kalata, P.R.: Alpha-beta target tracking systems: A survey. In: American Control Conference, pp. 832–836 (1992)

    Google Scholar 

  21. Kassahun, Y., de Gea, J., Edgington, M., Metzen, J.H., Kirchner, F.: Accelerating neuroevolutionary methods using a Kalman filter. In: Proceedings of the 10th Genetic and Evolutionary Computation Conference, GECCO 2008, pp. 1397–1404 (2008)

    Google Scholar 

  22. Kassahun, Y., Edgington, M., Metzen, J.H., Sommer, G., Kirchner, F.: A common genetic encoding for both direct and indirect encodings of networks. In: Proceedings of the 9th Genetic and Evolutionary Computation Conference, GECCO 2007, pp. 1029–1036 (2007)

    Google Scholar 

  23. Kirchner, F.: Q-learning of complex behaviors on a six-legged walking machine. Journal of Robotics and Autonomous Systems 25, 256–263 (1998)

    Article  MathSciNet  Google Scholar 

  24. Lee, W., Hallam, J., Lund, H.: Learning complex robot behaviours by evolutionary computing with task decomposition. In: Learning Robots: 6th European Workshop, pp. 155–172 (1998)

    Google Scholar 

  25. Moriarty, D., Miikkulainen, R.: Efficient reinforcement learning through symbiotic evolution. Machine Learning 22, 11–33 (1996)

    Google Scholar 

  26. Mouret, J.-B., Doncieux, S.: Evolving modular neural-networks through exaptation. In: CEC 2009: Proceedings of the Eleventh conference on Congress on Evolutionary Computation, Piscataway, NJ, USA, pp. 1570–1577. IEEE Press, Los Alamitos (2009)

    Google Scholar 

  27. Ruck, D.W., Rogers, S.K., Kabrisky, M., Maybeck, P.S., Oxley, M.E.: Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(6), 686–691 (1992)

    Article  Google Scholar 

  28. Singhal, S., Wu, L.: Training multilayer perceptrons with the extended Kalman algorithm, pp. 133–140 (1989)

    Google Scholar 

  29. Stanley, K.O.: Efficient Evolution of Neural Networks through Complexification. PhD thesis, Artificial Intelligence Laboratory. The University of Texas at Austin., Austin, USA (August 2004)

    Google Scholar 

  30. Wieland, A.: Evolving controls for unstable systems. In: Proceedings of the International Joint Conference on Neural Networks, pp. 667–673 (1991)

    Google Scholar 

  31. Williams, R.J.: Training recurrent networks using the extended Kalman filter. In: Proceedings International Joint Conference on Neural Networks, pp. 241–246 (1992)

    Google Scholar 

  32. Ziemke, T., Bergfeldt, N., Buason, G., Susi, T., Svensson, H.: Evolving cognitive scaffolding and environment adaptation: a new research direction for evolutionary robotics. Connection Science 16(4), 339–350 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotics Group, University of Bremen, Robert-Hooke-Str. 5, D-28359, Bremen, Germany

    Yohannes Kassahun & Frank Kirchner

  2. German Research for Artificial Intelligence (DFKI), Robotics Innovation Center, Robert-Hooke-Str. 5, D-28359, Bremen, Germany

    Jose de Gea, Jakob Schwendner & Frank Kirchner

Authors
  1. Yohannes Kassahun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose de Gea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jakob Schwendner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Frank Kirchner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CNRS UMR 7222, ISIR - Pierre and Marie Curie University, 4, place Jussieu cedex 05, Boite courrier 173, 75252, Paris, France

    Stéphane Doncieux

  2. Université Paris-Sud, TAO - Univ. Paris-Sud, INRIASaclay, CNRS, LRI, Bat. 490, F-91405, Orsay, France

    Nicolas Bredèche

  3. CNRS UMR 7222, ISIR - Pierre andMarie Curie University, 4, place Jussieu cedex 05, Boite courrier 173, 75252, Paris, France

    Jean-Baptiste Mouret

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kassahun, Y., de Gea, J., Schwendner, J., Kirchner, F. (2011). On Applying Neuroevolutionary Methods to Complex Robotic Tasks. In: Doncieux, S., Bredèche, N., Mouret, JB. (eds) New Horizons in Evolutionary Robotics. Studies in Computational Intelligence, vol 341. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-18272-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-18272-3_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-18271-6

  • Online ISBN: 978-3-642-18272-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation