, Volume 3, Issue 3, pp 113–127 | Cite as

Incremental development of multiple tool models for robotic reaching through autonomous exploration

  • Lorenzo JamoneEmail author
  • Bruno Damas
  • Nobotsuna Endo
  • José Santos-Victor
  • Atsuo Takanishi
Research Article


Autonomy and flexibility are two major requirements for modern robots. In particular, humanoid robots should learn new skills incrementally through autonomous exploration, and adapt to different contexts. In this paper we consider the problem of learning forward models for task space control under dynamically varying kinematic contexts: the robot learns incrementally and autonomously its forward kinematics under different contexts, represented by the inclusion of different tools, and exploits the learned model to realize reaching with those tools. We model the forward kinematics as a multi-valued function, in which different outputs for the same input query are related to different tools (i.e. contexts). The model is estimated using IMLE, a recent online learning algorithm for multi-valued regression, and used for control. No information is given about the tool changes, nor any assumption is made about the tool kinematics. Results are provided both in simulation and with a full-body humanoid. In the latter case we show how the robot successfully performs reaching using a flexible tool, a clear example of complex kinematics.


motor learning and adaptation humanoid robots reaching with tools developmental robotics continuous online learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    N. Endo, A. Takanishi, Development of Whole-body Emotional Expression Humanoid Robot for ADL-assistive RT services. Journal of Robotics and Mechatronics 23,6, pp. 969–977 (2011)Google Scholar
  2. [2]
    G. Metta, G. Sandini, D. Vernon, L. Natale, F. Nori, The iCub humanoid robot: an open platform for research in embodied cognition. Workshop on Performance Metrics for Intelligent Systems (2008)Google Scholar
  3. [3]
    J. Peters, S. Schaal, Learning Operational Space Control. Robotics: Science and Systems (2006)Google Scholar
  4. [4]
    O. Sigaud, C. Salan, V. Padois, On-line regression algorithms for learning mechanical models of robots: A survey. Robotics and Autonomous Systems 59,12, pp. 1115–1129 (2011).CrossRefGoogle Scholar
  5. [5]
    D. Nguyen-Tuong, J. Peters, Local gaussian process regression for real-time model-based robot control. International Conference on Intelligent Robots and Systems (2008)Google Scholar
  6. [6]
    S. Vijayakumar, A. D’Souza, S. Schaal, Incremental Online Learning in High Dimensions. Neural Computation 17,12, pp. 2602–2634 (2005)MathSciNetCrossRefGoogle Scholar
  7. [7]
    B. Damas, J. Santos-Victor, An Online Algorithm for Simultaneously Learning Forward and Inverse Kinematics. International Conference on Intelligent Robots and Systems (2012)Google Scholar
  8. [8]
    F. Guerin, N. Kruger, D. Kraft, A Survey of the Ontogeny of Tool Use: from Sensorimotor Experience to Planning. IEEE Transactions on Autonomous Mental Development, Onlineearlyacces (2012)Google Scholar
  9. [9]
    J. Krakauer, Z. Pine, M. Ghilardi, C. Ghez, Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories. Journal of Neuroscience 20, pp. 8916–8924 (2000)Google Scholar
  10. [10]
    R. Shadmehr, F. Mussa-Ivaldi, Adaptive representation of dynamics during learning of a motor task. Journal of Neuroscience 14, pp. 3208–3224 (1994)Google Scholar
  11. [11]
    R. Welch, B. Bridgeman, S. Anand, K. Browman, Alternating prism exposure causes dual adaptation and generalization to a novel displacement. Perception and Psychophysics 54,2, pp. 195–204 (1993)CrossRefGoogle Scholar
  12. [12]
    T. Brashers-Krug, R. Shadmehr, E. Bizzi, Consolidation in human motor memory. Nature 382, pp. 252–255 (1996)CrossRefGoogle Scholar
  13. [13]
    G. Berlucchi, S. Aglioti, The body in the brain: neural bases of corporeal awareness. Trends in Neurosciences 20,12, pp. 560–564 (1997)CrossRefGoogle Scholar
  14. [14]
    A. Berti, F. Frassinetti, When Far Becomes Near: Remapping of Space by Tool Use. Journal of Cognitive Neuroscience 12,3, pp. 415–420 (2000)CrossRefGoogle Scholar
  15. [15]
    A. Iriki, M. Tanaka, Y. Iwamura, Coding of modified body schema during tool use by macaque postcentral neurones. NeuroReport 7, pp. 2325–2330 (1996)CrossRefGoogle Scholar
  16. [16]
    K. Narendra, J. Balakrishnan, Adaptive control using multiple models. IEEE Transactions on Automatic Control 42,2, pp. 171–187 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  17. [17]
    D. Wolpert, M. Kawato, Multiple paired forward and inverse models for motor control. Neural Networks 11,7–8, pp. 1317–1329 (1998)CrossRefGoogle Scholar
  18. [18]
    M. Haruno, D. Wolpert, M. Kawato, Mosaic model for sensorimotor learning and control. Neural Computation 13,10, pp. 2201–2220 (2001)zbMATHCrossRefGoogle Scholar
  19. [19]
    N. Sugimoto, J. Morimoto, S. Hyon, M. Kawato, The eMOSAIC model for humanoid robot control. Neural Networks 29,30, pp. 8–19 (2012)CrossRefGoogle Scholar
  20. [20]
    G. Petkos, S. Vijayakumar, Context estimation and learning control through latent variable extraction: From discrete to continuous contexts. International Conference on Robotics and Automation (2007)Google Scholar
  21. [21]
    M. Hoffmann, H. Marques, A. Arieta, H. Sumioka, M. Lungarella, R. Pfeifer, Body schema in robotics: A review. IEEE Transactions on Autonomous Mental Development 2,4, pp. 304–324 (2010)CrossRefGoogle Scholar
  22. [22]
    M. Hikita, S. Fuke, M. Ogino, M. Asada, Cross-modal body representation based on visual attention by saliency. International Conference on Intelligent Robots and Systems (2008)Google Scholar
  23. [23]
    C. Nabeshima, Y. Kuniyoshi, M. Lungarella, Adaptive body schema for robotic tool-use. Advanced Robotics 20,10, pp. 1105–1126 (2006)CrossRefGoogle Scholar
  24. [24]
    S. Nishide, J. Tani, T. Takahashi, H. Okuno, T. Ogata, Tool-Body Assimilation of Humanoid Robot Using a Neurodynamical System. IEEE Transactions on Autonomous Mental Development 4,2, pp. 139–149 (2012)CrossRefGoogle Scholar
  25. [25]
    M. Rolf, J. Steil, M. Gienger, Learning flexible full body kinematics for humanoid tool use. International Symposium on Learning and Adaptive Behavior in Robotic Systems (2010)Google Scholar
  26. [26]
    L. Xu, M. Jordan, G. Hinton, An Alternative Model for Mixtures of Experts. Advances in Neural Information Processing Systems, pp. 633–640 (1995)Google Scholar
  27. [27]
    D. Grollman, O. Jenkins, Incremental learning of subtasks from unsegmented demonstration. International Conference on Intelligent Robots and Systems (2010)Google Scholar
  28. [28]
    A. Ligeois, Automatic supervisory control of the configuration and behavior of multibody mechanisms. Transactions on System, Man and Cybernetics 7, pp. 868–871 (1977)CrossRefGoogle Scholar
  29. [29]
    Y. Nakamura, H. Hanafusa, Inverse kinematic solutions with singularity robustness for robot manipulator control. Transactions of the ASME Journal of Dynamic Systems, Measurement and Control 108, pp. 163–171 (1986)zbMATHCrossRefGoogle Scholar
  30. [30]
    C. Salaun, V. Padois, O. Sigaud, Control of reundant robots using learned models: an operational space control approach. International Conference on Intelligent Robots and Systems (2009)Google Scholar
  31. [31]
    V. Tikhanoff, P. Fitzpatrick, G. Metta, L. Natale, F. Nori, A. Cangelosi, An open source simulator for cognitive robotics research: The prototype of the icub humanoid robot simulator. Workshop on Performance Metrics for Intelligent Systems (2008)Google Scholar
  32. [32]
    G. Metta, P. Fitzpatrick, L. Natale, Yarp: yet another robot platform. International Journal on Advanced Robotics Systems 3,1, pp. 43–48 (2006)Google Scholar
  33. [33]
    Y. Ogura, H. Aikawa, K. Shimomura, H. Kondo, A. Morishima, H. Lim, A. Takanishi, Development of a new humanoid robot WABIAN-2. International Conference on Robotics and Automation (2006)Google Scholar
  34. [34]
    H. Miwa, T. Okuchi, H. Takanobu, A. Takanishi, Development of a new human-like head robot WE-4. International Conference on Intelligent Robots and Systems (2002)Google Scholar
  35. [35]
    M. Zecca, N. Endo, S. Momoki, K. Itoh, A. Takanishi, Design of the humanoid robot KOBIAN-preliminary analysis of facial and whole body emotion expression capabilities. International Conference on Humanoid Robots (2008)Google Scholar
  36. [36]
    N. Endo, K. Endo, K. Hashimoto, T. Kojima, F. Iida, A. Takanishi, Integration of Emotion Expression and Visual Tracking Locomotion Based on Vestibulo-Ocular Reflex. International Symposium on Robot and Human Interactive Communication (2010)Google Scholar
  37. [37]
    G. Metta, G. Sandini, J. Konczak, A developmental approach to visually-guided reaching in artificial systems. Neural Networks 12,10, pp. 1413–1427 (1999)CrossRefGoogle Scholar
  38. [38]
    L. Jamone, L. Natale, G. Metta, F. Nori, G. Sandini, Autonomous online learning of reaching behavior in a humanoid robot. International Journal of Humanoid Robotics 9,3, pp. 1250017.1–1250017.26 (2012)CrossRefGoogle Scholar
  39. [39]
    L. Jamone, L. Natale, K. Hashimoto, G. Sandini, A. Takanishi, Learning task space control through goal directed exploration. International Conference on Robotics and Biomimetics (2011)Google Scholar
  40. [40]
    L. Jamone, L. Natale, G. Sandini, A. Takanishi, Interactive online learning of the kinematic workspace of a humanoid robot. International Conference on Intelligent Robots and Systems (2012)Google Scholar

Copyright information

© Versita Warsaw and Springer-Verlag Wien 2012

Authors and Affiliations

  • Lorenzo Jamone
    • 1
    Email author
  • Bruno Damas
    • 2
    • 3
  • Nobotsuna Endo
    • 1
  • José Santos-Victor
    • 2
  • Atsuo Takanishi
    • 1
    • 4
  1. 1.Faculty of Science and EngineeringWaseda UniversityTokyoJapan
  2. 2.Instituto de Sistemas e RobóticaInstituto Superior TécnicoLisboaPortugal
  3. 3.Escola Superior de Tecnologia de SetúbalSetúbalPortugal
  4. 4.Humanoid Robotics InstituteWaseda UniversityTokyoJapan

Personalised recommendations