Motion-Language Association Model for Human-Robot Communication

  • Wataru TakanoEmail author
  • Minoru Kanazawa
  • Yoshihiko Nakamura
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 79)


Language is symbolic system, from which human intelligence originates. A robot is expected to use language. Recently a large amount of text corpus can be obtained, and human knowledge lies there. Thus, it is important for the robot to acquire the knowledge and to develop the ability to use language. This paper describes a novel approach to a humanoid robot that makes linguistic inference by using language knowledge in the dictionary. Sentences are interpreted as broader concepts according to the dictionary. The association between the broader concepts is stochastically represented. The abstract referential relationship between sentences and broader concepts, and the associative relationship among the broader concepts allow the robot to make linguistic inference by generating sentences from input one. As one of applications of the proposed linguistic inference, this framework is integrated with the symbols of motion patterns. The developed application demonstrates the validity of the proposed framework since the robot can make linguistic inference and associate motion patterns subsequently.


Motion Pattern Humanoid Robot Broad Concept Hide State Motion Primitive 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Deacon, T.: Symbolic species: The co-evolution of language and the brain. W.W.Norton and Company Inc. (1997)Google Scholar
  2. 2.
    Rizzolatti, G., Fogassi, L., Gallese, V.: Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews, 661–670 (2001)Google Scholar
  3. 3.
    Donald, M.: Origin of the modern mind. Harvard University Press, Cambridge (1991)Google Scholar
  4. 4.
    Tani, J., Ito, M.: Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man and Cybernetics Part A: Systems and Humans 33(4), 481–488 (2003)CrossRefGoogle Scholar
  5. 5.
    Sugita, Y., Tani, J.: Learning semantic combinatoriality from the interaction between linguistic and behavioral processes. Adaptive Behavior, 33–52 (2005)Google Scholar
  6. 6.
    Ogata, T., Murase, M., Tani, J., Komatani, K., Okuno, H.G.: Two-way translation of compound sentences and arm motions by recurrent neural networks. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1858–1863 (2007)Google Scholar
  7. 7.
    Kadone, H., Nakamura, Y.: Symbolic memory for humanoid robots using hierarchical bifurcations of attractors in nonmonotonic neural networks. In: Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2900–2905 (2005)Google Scholar
  8. 8.
    Haruno, M., Wolpert, D., Kawato, M.: Mosaic model for sensorimotor learning and control. Neural Computation 13, 2201–2220 (2001)zbMATHCrossRefGoogle Scholar
  9. 9.
    Inamura, T., Toshima, I., Tanie, H., Nakamura, Y.: Embodied symbol emergence based on mimesis theory. International Journal of Robotics Research 23(4), 363–377 (2004)CrossRefGoogle Scholar
  10. 10.
    Billard, A., Calinon, S., Guenter, F.: Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Systems 54, 370–384 (2006)CrossRefGoogle Scholar
  11. 11.
    Takano, W., Yamane, K., Sugihara, T., Yamamoto, K., Nakamura, Y.: Primitive communication based on motion recognition and generation with hierarchical mimesis model. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3602–3609 (2006)Google Scholar
  12. 12.
    Sugiura, K., Iwahashi, N., Kashioka, H., Nakamura, S.: Active learning of confidence measure function in robot language acquisition framework. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1774–1779 (2010)Google Scholar
  13. 13.
    Takano, W., Yamane, K., Nakamura, Y.: Capture database through symbolization, recognition and generation of motion patterns. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3092–3097 (2007)Google Scholar
  14. 14.
    Takano, W., Nakamura, Y.: Statistically integrated semiotics that enables mutual inference between linguistic and behavioral symbols for humanoid robots. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 646–652 (2009)Google Scholar
  15. 15.
    Takano, W., Nakamura, Y.: Incremental learning of integrated semiotics based on linguistic and behavioral symbols. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2545–2550 (2009)Google Scholar
  16. 16.
    Takeuchi, K., Matsumoto, Y.: Hmm parameter learning for japanese morphological analyzer. In: Proceedings of the 10th Pacific Asia Conference on Language, Information and Computation, pp. 163–172 (1995)Google Scholar
  17. 17.
    Takano, W., Nakamura, Y.: Humanoid robot’s autonomous acquisition of proto-symbols through motion segmentation. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots, pp. 425–431 (2006)Google Scholar
  18. 18.
    Kudo, T., Yamamoto, K., Matsumoto, Y.: Applying conditional random fields to japanese morphological analysis. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 230–237 (2004)Google Scholar

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2014

Authors and Affiliations

  • Wataru Takano
    • 1
    Email author
  • Minoru Kanazawa
    • 1
  • Yoshihiko Nakamura
    • 1
  1. 1.The University of TokyoBunkyokuJapan

Personalised recommendations