Abstract
In order to perform a reaching movement towards a moving target, an autonomously learning robot must first learn several transformations, such as motion detection, coordinate transformation between the camera and the arm and the inverse model of the arm. A curious reaching robot learns them better by performing the appropriate actions so as to expedite and improve their learning speed and accuracy. We implement a model of hierarchical curiosity loops in an autonomous active learning robot, whereby each loop converges to the optimal action that maximizes the robot’s learning of the appropriate transformation. It results in the emergence of unique behaviors that ultimately lead to the capability of reaching.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gordon, G., Kaplan, D.M., Lankow, B., Little, D.Y., Sherwin, J., Suter, B.A., Thaler, L.: Toward an integrated approach to perception and action: conference report and future directions. Front Syst. Neurosci. 5, 20 (2011)
Shadmehr, R.: Generalization as a behavioral window to the neural mechanisms of learning internal models. Human Movement Science 23, 543–568 (2004)
Natale, L., Orbona, F., Metta, G., Sandini, G.: Exploring the world through grasping: a developmental approach. In: IEEE International Symposium on Computational Intelligence in Robots and Automation (2005)
Edsinger, A., Kemp, C.C.: What can i control? a framework for robot self-discovery. In: The Sixth International Conference on Epigenetic Robotics, EpiRob (2006)
Nori, F., Natale, L., Sandini, G., Metta, G.: Autonomous learning of 3d reaching in a humanoid robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2007)
Gordon, G.: Hierarchical exhaustive construction of autonomously learning networks. In: Evolutionary and Reinforcement Learning for Autonomous Robot Systems, ERLARS (2011)
Schmidhuber, J.: A possibility for implementing curiosity and boredom in model-building neural controllers. In: Proceedings of the First International Conference on Simulation of Adaptive Behavior on From Animals to Animats, pp. 222–227. MIT Press (1990)
Barto, A.G., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: International Conference on Developmental Learning, ICDL (2004)
Weng, J.: Developmental robotics: Theory and experiments. International Journal of Humanoid Robotics 1(2), 199–236 (2004)
Oudeyer, P.Y., Kaplan, F., Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 11(2), 265–286 (2007)
Gordon, G., Ahissar, E.: Hierarchical curiosity loops and active sensing. Neural Networks 32, 119–129 (2012)
Schmidhuber, J.: Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Transactions on Autonomous Mental Development 2(3), 230–247 (2010)
Gordon, G., Ahissar, E.: Reinforcement active learning hierarchical loops. In: International Joint Conference on Neural Networks, IJCNN (2011)
Ahissar, E., Knutsen, P.M.: Object localization with whiskers. Biol. Cybern. 98, 449–458 (2008)
Knutsen, P.M., Ahissar, E.: Orthogonal coding of object location. Trends Neurosci. 32, 101–109 (2009)
Bhatnagar, S., Sutton, R., Ghavamzadeh, M., Lee, M.: Incremental natural actor-critic algorithms. In: Twenty-First Annual Conference on Advances in Neural Information Processing Systems, pp. 105–112 (2007)
Peters, J., Schaal, S.: Natural actor-critic. Neurocomput. 71(7-9), 1180–1190 (2005)
Kemp, C.C., Edsinger, A.: What can i control?: The development of visual categories for a robot’s body and the world that it in uences. In: 5th IEEE International Conference on Development and Learning (ICDL5): Special Session on Perceptual Systems and their Development (2006)
Garg, D.P., Kumar, M.: Camera calibration and sensor fusion in an automated flexible manufacturing multi-robot work cell. In: American Control Conference, pp. 4934–4939 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gordon, G., Ahissar, E. (2012). A Curious Emergence of Reaching. In: Herrmann, G., et al. Advances in Autonomous Robotics. TAROS 2012. Lecture Notes in Computer Science(), vol 7429. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32527-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-32527-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32526-7
Online ISBN: 978-3-642-32527-4
eBook Packages: Computer ScienceComputer Science (R0)