Robot Motor Skill Acquisition with Learning in Two Spaces

  • Jian FuEmail author
  • Ce Cao
  • Jinyu Du
  • Siyuan Shen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11745)


Motor skill acquisition and refinement is critical for the robot to step in human daily lives, which can endow it with the ability of autonomously performing unfamiliar tasks. However, how does the robot autonomously fulfill the new motion task with preassigned performance based on the demonstration task is still a challenge. We in this paper proposed a novel motor skill acquisition policy to conquer above problem, which is based on improved local weighted regression (iLWR), policy improvement with path integral (PI\(^2\)). Besides, the mixture Gaussian regression (GMR) guided self-reconstruction of basis function and the search of weight coefficient in the policy expression are performed alternately in basis function space and weight space to seek the optimal/suboptimal solution. In this way, robot can achieve the gradual acquisition of movement skills from similar tasks which is related to the demonstration to unsimilar task with different criterion. At last, the classical via-points trajectory planning experiment are performed with SCARA manipulator, NAO humanoid robot to verify that the proposed method is effective and feasible.


Alternate study in two spaces GMR-PI\(^2\) Motor skill acquisition 


  1. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)CrossRefGoogle Scholar
  2. Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. Artif. Intell. Rev. 11(1), 11–73 (1997)CrossRefGoogle Scholar
  3. Deisenroth, M., Neumann, G., Peters, J.: A survey on policy search for robotics. J. Intell. Rob. Syst. 15(1), 1–2 (2013)Google Scholar
  4. Gregory, M.D., Martin, S.V., Werner, D.H.: Improved electromagnetics optimization: the covariance matrix adaptation evolutionary strategy. IEEE Antennas Propag. Mag. 57(3), 48–59 (2015). Scholar
  5. Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013)MathSciNetCrossRefGoogle Scholar
  6. Khansari-Zadeh, S.M., Billard, A.: BM: an iterative algorithm to learn stable non-linear dynamical systems with Gaussian mixture models. In: 2010 IEEE International Conference on Robotics and Automation, Anchorage, USA, pp. 2381–2388 (2010)Google Scholar
  7. Khansari-Zadeh, S.M., Billard, A.: Learning stable nonlinear dynamical systems with Gaussian mixture models. IEEE Trans. Robot. 27(5), 943–957 (2011)CrossRefGoogle Scholar
  8. Parisi, S., Abdulsamad, H., Paraschos, A., Daniel, C., Peters, J.: Reinforcement learning vs human programming in tetherball robot games. In: 2015 IEEE International conference on Intelligent Robots and Systems, Hamburg, Germany, pp. 6428–6434 (2015)Google Scholar
  9. Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Netw. Off. J. Int. Neural Netw. Soc. 21(4), 682 (2008)CrossRefGoogle Scholar
  10. Peters, J., Mülling, K., Altun, Y.: Relative entropy policy search. In: 24th AAAI, Atlanta, Westin, USA, pp. 1607–1612 (2010)Google Scholar
  11. Pfeifer, R., Bongard, J.: How the Body Shapes the Way We Think: A New View of Intelligence. MIT Press, Cambridge (2006)CrossRefGoogle Scholar
  12. Rombokas, E., Malhotra, M., Theodorou, E.A., Todorov, E., Matsuoka, Y.: Reinforcement learning and synergistic control of the ACT hand. IEEE Trans. Mechatron. 18(2), 569–577 (2013). Scholar
  13. Theodorou, E., Buchli, J., Schaal, S.: A generalized path integral control approach to reinforcement learning. J. Mach. Learn. Res. 11, 3137–3181 (2010)MathSciNetzbMATHGoogle Scholar
  14. Ude, A., Asfour, G.T.A.: Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Trans. Robot. 26(5), 800–815 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of AutomationWuhan University of TechnologyWuhanChina

Personalised recommendations