Advertisement

Motion-Based Learning

  • Il Hong Suh
  • Sang Hyoung Lee
Chapter
Part of the Human–Computer Interaction Series book series (HCIS)

Abstract

In this Chapter, we introduce several learning approaches to generate non-preprogrammed motions for a virtual human. Motion primitives and their causalities should first be learned from a task, which consists of a cascade of sub-tasks. Using programming by demonstration (PbD), it is now common for a virtual human to learn motion primitives and their causalities from a human demonstration. Typically, a virtual human can swiftly and effortlessly acquire a human demonstration from a PbD. To generate non-preprogrammed motions, a virtual human should possess the abilities to: (i) segment a whole movement into meaning segments; (ii) learn motion primitives for their adaptation in a changing environment; (iii) represent a combination of a motion primitive and its causalities (a motion tuple) by considering reusability; and finally, (iv) swiftly and reasonably select a dependable motion primitive in accordance with current and goal situations. In this chapter, we review the state of the art and several solution approaches including their limitations. We then discuss future avenues to target motion tuples in terms of the generation of non-preprogrammed motions for a virtual human.

Keywords

Bayesian Network Gaussian Mixture Model Motion Trajectory Fine Movement Virtual Human 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Tsutomu H, Takashi S, Kunikatsu T (1992) A model-based manipulation system with skill-based execution. IEEE Trans Robot Autom 8(5):535–544CrossRefGoogle Scholar
  2. 2.
    Bainbridge C (2008) Characteristics of gifted childrenGoogle Scholar
  3. 3.
    Albert B, Dorothea R, Ross Sheila A (1963) Imitation of film-mediated aggressive models. J Abnorm Soc Psychol 66(1):3–11CrossRefGoogle Scholar
  4. 4.
    Sylvain C, Florent G, Aude B (2007) On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans SMC Part B: Cybern 37(2):286–298Google Scholar
  5. 5.
    Amor HB, Vogt D, Ewerton M, Berger E, Jung B, Peters J (2013) Learning responsive robot behavior by imitation. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, Japan, pp 3257–3264Google Scholar
  6. 6.
    Schaal S (2006) Dynamic movement primitives—a framework for motor control in humans and humanoid robotics. In: Adaptive motion of animals and machines. Springer, Tokyo, pp 261–280Google Scholar
  7. 7.
    Billing EA, Hellstrom T, Janlert LE (2010) Behavior recognition for learning from demonstration. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Anchorage, Alaska, USA, pp 866–872Google Scholar
  8. 8.
    Cohen BJ, Chitta S, Likhachev M (2010) Search-based planning for manipulation with motion primitives. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Anchorage, Alaska, USA, pp 2902–2908Google Scholar
  9. 9.
    Bentivegna DC, Atkeson CG (2001) Learning from observation using primitives. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Seoul, Korea, pp 1988–1993Google Scholar
  10. 10.
    Nicolescu MN, Mataric MJ (2003) Natural methods for robot task learning: instructive demonstrations, generalization and practice. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems, Melbourne, Australia, pp 241–248Google Scholar
  11. 11.
    Nejati N, Langley P, Konik T (2006) Learning hierarchical task networks by observation. In: Proceedings of the 23rd international conference on machine learning, Pittsburgh, PA, USA, pp 665–672Google Scholar
  12. 12.
    Drumwright E, Jenkins OC, Mataric M (2004) Exemplar-based primitives for humanoid movement classification and control. In: Proceedings of IEEE international conference on robotics and automation (ICRA), New Orleans, LA, USA, pp 140–145Google Scholar
  13. 13.
    Dana K, Wataru T, Yoshihiko N (2009) Online segmentation and clustering from continuous observation of whole body motions. IEEE Trans Robot 25(5):1158–1166CrossRefGoogle Scholar
  14. 14.
    Gribovskaya E, Billard A (2008) Combining dynamical systems control and programming by demonstration for teaching discrete bimanual coordination tasks to a humanoid robot. In: Proceedings of ACM/IEEE international conference on human-robot interaction (HRI), Amsterdam, NetherlandsGoogle Scholar
  15. 15.
    Muhlig M, Gienger M, Steil JJ (2010) Human-robot interaction for learning and adaptation of object movements. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Taipei, Taiwan, pp 4901–4907Google Scholar
  16. 16.
    Asfour T, Azad P, Gyarfas F, Dillmann R (2006) Imitation learning of dual-arm manipulation tasks in humanoid robots. In: Proceedings of IEEE/RAS international conference on humanoid robots, Genoa, Italy, pp 40–47Google Scholar
  17. 17.
    Sanmohan, Kruger V, Kragic D, Kjellstrom H (2011) Primitive-based action representation and recognition. Adv Robot 25(6–7):871–891Google Scholar
  18. 18.
    Kruger V, Tikhanoff V, Natale L, Sandini G (2012) Imitation learning of non-linear point-to-point robot motions using Dirichlet processes. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Saint Paul, Minnesota, USA, pp 2029–2034Google Scholar
  19. 19.
    Smith T, Simmons R (2004) Heuristic search value iteration for POMDPs. In: Proceedings of the 20th conference on uncertainty in artificial intelligence, Virginia, USA, pp 520–527Google Scholar
  20. 20.
    Kurniawati H, Hsu D, Lee WS (2008) SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces. In: Proceedings of robotics: science and systemsGoogle Scholar
  21. 21.
    Olivier L, Pierre B, Julien D, Emmanuel M (2004) Bayesian robot programming. Auton Robot 16(1):49–79CrossRefGoogle Scholar
  22. 22.
    Dearden A, Demiris Y (2005) Learning forward models for robots. In: Proceedings of international joint conference on artificial intelligence, Edinburgh, pp 1440–1445Google Scholar
  23. 23.
    Lee D, Nakamura Y (2006) Stochastic model of imitating a new observed motion based on the acquired motion primitives. In: Proceedings of IEEE/RSJ International conference on intelligent robots and systems (IROS), Beijing, China, pp 4994–5000Google Scholar
  24. 24.
    Michael P, Steffen K, Ruediger D, Raoul Z (2007) Incremental learning of tasks from user demonstrations, past experiences, and vocal comments. IEEE Trans SMC Part B: Cybern 37(2):322–332Google Scholar
  25. 25.
    Ekvall S, Kragic D (2006) Learning task models from multiple human demonstration. In: Proceedings of IEEE international symposium on robot and human interactive communication, Hatfield, UK, pp 358–363Google Scholar
  26. 26.
    Brooks Rodney A (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–23CrossRefGoogle Scholar
  27. 27.
    Hoshino Y, Takagi T, Profio UD, Fujita M (2004) Behavior description and control using behavior module for personal robot. In: Proceedings of international conference on robotics and automation (ICRA), New Orleans, LA, USA, pp 4165–4171Google Scholar
  28. 28.
    Jafreezal J, Eric M (2008) A fuzzy action selection method for virtual agent navigation in unknown virtual environments. J Uncertainty Syst 2(2):144–154Google Scholar
  29. 29.
    Matthias S, Virgil A (2004) Architectural mechanisms for dynamic changes of behavior selection strategies in behavior-based systems. IEEE Trans SMC Part B: Cybern 34(6):2377–2395Google Scholar
  30. 30.
    Lee S, Suh LH (2006) A programming framework supporting and ethology-based behavior control architecture. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Beijing, China, pp 4138–4144Google Scholar
  31. 31.
    Stoffregen TA (2003) Affordances as properties of the animal-environment system. Ecol Psychol 15(2):115–134CrossRefGoogle Scholar
  32. 32.
    Anthony C (2003) An outline of a theory of affordances. Ecol Psychol 15(2):181–195CrossRefGoogle Scholar
  33. 33.
    Steedman M (2002) Formalizing affordance. In: Proceedings of the 24th annual meeting of the cognitive science society, pp 834–839Google Scholar
  34. 34.
    Erol S, Maya C, Dogar Mehmet R, Emre U, Gokturk Ucluk (2007) To afford or not to afford: a new formalization of affordances toward affordance-based robot control. Adapt Behav 15(4):447–472CrossRefGoogle Scholar
  35. 35.
    Lee SH, Suh LH, Calinon S, Johansson R (2014) Autonomous framework for segmenting robot trajectories of manipulation task. Autonomous Robots. Springer, Berlin, pp 1–35Google Scholar
  36. 36.
    Zoubin G, Jordan Michael I (1994) Supervised learning from incomplete data via an EM approach. Adv Neural Inf Process Syst 6:120–127Google Scholar
  37. 37.
    Akgun B, Cakmak M, Yoo JW, Thomaz AL (2012) Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective. In: Proceedings of ACM/IEEE international conference on human-robot interaction (HRI), Boston, Massachusetts, USA, pp 391–398Google Scholar
  38. 38.
    Argall BD, Sauser EL, Billard AG (2010) Tactile guidance for policy refinement and reuse. In: Proceedings of international conference on development and learning (ICDL), pp 7–12Google Scholar
  39. 39.
    Lee SH, Cho NJ, Suh LH (2013) Learning of motor skills based on grossness and fineness of movements in daily-life tasks. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Hong Kong, China, pp 5260–5267Google Scholar
  40. 40.
    Hardoon DR, Sandor S, John S-T (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664zbMATHCrossRefGoogle Scholar
  41. 41.
    Muhlig M, Gienger M, Hellbach S, Steil JJ, Goerick C (2009) Task-level imitation learning using variance-based movement optimization. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Kobe, Japan, pp 1177–1184Google Scholar
  42. 42.
    Liping J, Ng MK, Huang ZJ (2007) An entropy weighting k-means algorithm for subspace clustering on high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041CrossRefGoogle Scholar
  43. 43.
    Kerdprasop K, Kerdprasop N, Sattayatham P (2005) Weighted k-means for density-biased clustering. In: Data warehousing and knowledge discovery. Springer, Berlin, pp 488–497Google Scholar
  44. 44.
    Pastor P, Hoffmann H, Asfour T, Schaal S (2009) Learning and generalization of motor skills by learning from demonstration. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Kobe, Japan, pp 763–768Google Scholar
  45. 45.
    Hoffmann H, Pastor P, Park DH, Schaal S (2009) Biologically-inspired dynamical systems for movement generation: automatic real-time goal adaptation and obstacle avoidance. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Kobe, Japan, pp 2587–2592Google Scholar
  46. 46.
    Lee SH, Suh LH (2013) Skill learning and inference framework for skilligent robot. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, Japan, pp 108–115Google Scholar
  47. 47.
    Gaver WW (1991) Technology affordances. In: Proceedings of the SIGCHI conference on human factors in computing systems: reaching through technology, New York, NY, USA, pp 79–84Google Scholar
  48. 48.
    Bryson JJ (2001) The study of sequential and hierarchical organisation of behaviour via artificial mechanisms of action selection. PhD thesis, University of EdinburghGoogle Scholar
  49. 49.
    Blumberg BM (1996) Old tricks, new dogs: ethology and interactive creatures. PhD thesis, Massachusetts Institute of TechnologyGoogle Scholar
  50. 50.
    Suh LH, Kim MJ, Lee S, Yi BJ (2004) A novel dynamic priority-based action-selection-mechanism integrating a reinforcement learning. In: Proceedings of IEEE international conference on robotics and automation (ICRA), New Orleans, LA, USA, pp 2639–2646Google Scholar
  51. 51.
    Parker LE (1998) ALLIANCE: an architecture for fault tolerant multirobot cooperation. IEEE Trans Robot Autom 14(2):220–240CrossRefGoogle Scholar
  52. 52.
    Kleinginna PR, Kleinginna AM (1981) A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv Emot 5(4):345–379CrossRefGoogle Scholar
  53. 53.
    Luis M, Manuel L, Alexandre B, Jose S-V (2008) Learning object affordance: from sensory-motor coordination to imitation. IEEE Trans Robot 24(1):15–26CrossRefGoogle Scholar
  54. 54.
    Abdo N, Kretzschmar H, Spinello L, Stachniss C (2013) Learning manipulation actions from a few demonstrations. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Karlsruhe, Germany, pp 1268–1275Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Hanyang UniversitySeoulKorea

Personalised recommendations