Abstract
This paper investigates a hybrid two-phase approach toward exploratory behavior in robotics. In a first phase, controllers are evolved to maximize the quantity of information in the sensori-motor datastream generated by the robot. In a second phase, the data acquired by the evolved controllers is used to support an information theory-based controller, selecting the most informative action in each time step. The approach, referred to as EvITE, is shown to outperform both the evolutionary and the information theory-based approaches standalone, in terms of actual exploration of the arena. Further, the EvITE controller features some generality property, being able to efficiently explore other arenas than the one considered during the first evolutionary phase.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akrour, R., Schoenauer, M., Sebag, M.: Preference-based policy learning. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part I. LNCS, vol. 6911, pp. 12–27. Springer, Heidelberg (2011)
Baranès, A., Oudeyer, P.Y.: R-iac: Robust intrinsically motivated exploration and active learning. IEEE Transactions on Autonomous Mental Development 1(3), 155–169 (2009)
Delarboulas, P., Schoenauer, M., Sebag, M.: Open-ended evolutionary robotics: an information theoretic approach. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 334–343. Springer, Heidelberg (2010)
Duda, P.O., Hart, P.E.: Pattern Classification and Scene analysis. John Wiley and sons (1973)
Heidrich-Meisner, V., Igel, C.: Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search. In: Int. Conf. on Machine Learning, pp. 401–408 (2009)
Kober, J., Bagnell, J.A., Peters, J.: Reinforcement learning in robotics: A survey. The Int. Jal of Robotics Research 32(11), 1238–1274 (2013)
Koos, S., Mouret, J.B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Trans. on Evolutionary Computation 17(1), 122–145 (2013)
Lehman, J., Risi, S., D’Ambrosio, D.B., Stanley, K.O.: Rewarding reactivity to evolve robust controllers without multiple trials or noise. Artificial Life 13, 379–386 (2012)
Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. Artificial Life 11, 329 (2008)
Lipson, H., Pollack, J.B.: Automatic design and manufacture of robotic lifeforms. Nature 406(6799), 974–978 (2000)
Liu, W., Winfield, A.F.: Modeling and optimization of adaptive foraging in swarm robotic systems. The Int. Jal of Robotics Research 29(14), 1743–1760 (2010)
Lopes, M., Lang, T., Toussaint, M., Oudeyer, P.-Y.: Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress. In: NIPS, pp. 206–214 (2012)
Mouret, J.B., Doncieux, S.: Encouraging behavioral diversity in evolutionary robotics: An empirical study. Evolutionary Computation 20(1), 91–133 (2012)
Nolfi, S., Floreano, D., Floreano, D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT Press, Cambridge (2000)
Oudeyer, P.Y., Kaplan, F., Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. on Evolutionary Computation 11(2), 265–286 (2007)
Pfeifer, R., Gomez, G.: Interacting with the real world: design principles for intelligent systems. Artificial life and Robotics 9(1), 1–6 (2005)
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. The Int. Jal of Robotics Research 27(2), 157–173 (2008)
Sutton, R., Barto, A.: Reinforcement learning: An introduction. Cambridge Univ. Press (1998)
Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT Press (2005)
Williams, H., Browne, W.N.: Integration of Learning Classifier Systems with simultaneous localisation and mapping for autonomous robotics. In: CEC 2012, pp. 1–8 (2012)
Hurst, J., Bull, L., Melhuish, C.: TCS learning classifier system controller on a real robot. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 588–597. Springer, Heidelberg (2002)
Koutnk, J., Cuccu, G., Schmidhuber, J.: Evolving large-scale neural networks for vision-based reinforcement learning. In: GECCO 2013, pp. 1061–1068 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhang, G., Sebag, M. (2014). Coupling Evolution and Information Theory for Autonomous Robotic Exploration. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds) Parallel Problem Solving from Nature – PPSN XIII. PPSN 2014. Lecture Notes in Computer Science, vol 8672. Springer, Cham. https://doi.org/10.1007/978-3-319-10762-2_84
Download citation
DOI: https://doi.org/10.1007/978-3-319-10762-2_84
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10761-5
Online ISBN: 978-3-319-10762-2
eBook Packages: Computer ScienceComputer Science (R0)