Perceiving Objects and Movements to Generate Actions on a Humanoid Robot

  • Tamim Asfour
  • Kai Welke
  • Aleš Ude
  • Pedram Azad
  • Rüdiger Dillmann
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 8)

The concept of Object-Action Complexes (OACs) has been introduced by the European PACO-PLUS consortium ( [8]) to emphasize the notion that for a cognitive agent objects and actions are inseparably intertwined and that categories are therefore determined (and also limited) by the action an agent can perform and by the attributes of the world it can perceive. The resulting OACs are the entities on which cognition develops (action-centered cognition). Entities (things) in the world of a robot (or human) will only become semantically useful objects through the action that the agent can/will perform on them.

In this work we present a new humanoid active head which features human-like characteristics in motion and response and mimics the human visual system. We present algorithms that can be applied to perceive objects and movements, which form the basis for learning actions on the humanoid. For action representation we use an HMM-based approach to reproduce the observed movements and build an action library. Hidden Markov Models (HMM) are used to represent movements demonstrated to a robot multiple times. They are trained with the characteristic features (key points) of each demonstration. We propose strategies for adaptation of movements to the given situation and for the interpolation between movements stored in a movement library.


Hide Markov Model Humanoid Robot Query Point Imitation Learning Movement Library 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akachi, K., Kaneko, K., Kanehira, N., Ota, S., Miyamori, G., Hirata, M., Kajita, S., Kanehiro, F.: Development of humanoid robot HRP-3. In: IEEE/RAS International Conference on Humanoid Robots (2005)Google Scholar
  2. 2.
    Asfour, T., Gyarfas, F., Azad, P., Dillmann, R.: Imitation learning of dual-arm manipulation tasks in humanoid robots. In: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2006). Genoa, Italy (2006)Google Scholar
  3. 3.
    Asfour, T., Regenstein, K., Azad, P., Schr öder, J., Bierbaum, A., Vahrenkamp, N., Dillmann, R.: ARMAR-III: An integrated humanoid platform for sensory-motor control. In: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2006). Genoa, Italy (2006)Google Scholar
  4. 4.
    Atkeson, C.G., Moore, A., Schaal, S.: Locally weighted learning. AI Review 11, 11-73 (1997)Google Scholar
  5. 5.
    Azad, P., Ude, A., Asfour, T., Cheng, G., Dillmann, R.: Image-based Markerless 3D Human Motion Capture using Multiple Cues. In: International Workshop on Vision Based HumanRobot Interaction. Palermo, Italy (2006)Google Scholar
  6. 6.
    Azad, P., Ude, A., Dillmann, R., Cheng, G.: A Full Body Human Motion Capture System using Particle Filtering and On-The-Fly Edge Detection. In: International Conference on Humanoid Robots (Humanoids). Santa Monica, USA (2004)Google Scholar
  7. 7.
    Cheng, G., Hyon, S., Morimoto, J., Ude, A., Jacobsen, S.: CB: a humanoid research platform for exploring neuroscience. In: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2006). Genoa, Italy (2006)Google Scholar
  8. 8.
    The PACO-PLUS project: Perception, Action and Cognition through Learning of ObjectAction Complexes. European Cognitive Systems project,
  9. 9.
    Gibson, J.: The ecological approach to visual perception. Houghton Mifflin, Boston (1979)Google Scholar
  10. 10.
    Hersch, M., Billard, A.: A biologically-inspired model of reaching movements. In: IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (2006)Google Scholar
  11. 11.
    I.W. Park J.Y. Kim, J.L., Oh, J.: Mechanical design of humanoid robot platform KHR-3 (kaist humanoid robot-3: Hubo). In: IEEE/RAS International Conference on Humanoid Robots (2005)Google Scholar
  12. 12.
    Nishiwaki, K., Sugihara, T., Kagami, S., Kanehiro, F., Inaba, M., Inoue, H.: Design and development of research platform for perception-action integration in humanoid robots: H6. In: IEEE/RSJ International. Conference on Intelligent Robots and Systems, pp. 1559-1564 (2000)Google Scholar
  13. 13.
    Rose, C., Bodenheimer, B., Cohen, M.F.: Verbs and adverbs: Multidimensional motion interpolation using radial basis functions. IEEE Computer Graphics and Applications 18(5) (1998)Google Scholar
  14. 14.
    Sakagami, S., Watanabe, T., Aoyama, C., Matsunage, S., Higaki, N., Fujimura, K.: The intelligent ASIMO: System overview and integration. In: IEEE/RSJ International. Conference on Intelligent Robots and Systems, pp. 2478-2483 (2002)Google Scholar
  15. 15.
    Ude, A., Atkeson, C.G., Riley, M.: Programming full-body movements for humanoid robots by observation. Robotics and Autonomous Systems 47(2-3), 93-108 (2004)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Tamim Asfour
    • 1
  • Kai Welke
    • 1
  • Aleš Ude
    • 2
  • Pedram Azad
    • 1
  • Rüdiger Dillmann
    • 1
  1. 1.Institute for Computer Science and EngineeringUniversity of KarlsruheGermany
  2. 2.Dept. of Automatics, Biocybernetics, and RoboticsJožef Stefan InstituteSlovenia

Personalised recommendations