Advertisement

Journal of Intelligent & Robotic Systems

, Volume 96, Issue 3–4, pp 457–475 | Cite as

Motor Skills Learning and Generalization with Adapted Curvilinear Gaussian Mixture Model

  • Huiwen Zhang
  • Yuquan LengEmail author
Article
  • 124 Downloads

Abstract

This paper is intended to solve the motor skills learning, representation and generalization problems in robot imitation learning. To this end, we present an Adapted Curvilinear Gaussian Mixture Model (AdC-GMM), which is a general extension of the GMM. The proposed model can encode data more compactly. More critically, it is inherently suitable for representing data with strong non-linearity. To infer the parameters of this model, a Cross Entropy Optimization (CEO) algorithm is proposed, where the cross entropy loss of the training data is minimized. Compared with the traditional Expectation Maximization (EM) algorithm, the CEO can automatically infer the optimal number of components. Finally, the generalized trajectories are retrieved by an Adapted Curvilinear Gaussian Mixture Regression (AdC-GMR) model. To encode observations from different frames, the sophisticated task parameterization (TP) technique is introduced. All above proposed algorithms are verified by comprehensive tasks. The CEO is evaluated by a hand writing task. Another goal-directed reaching task is used to evaluate the AdC-GMM and AdC-GMR algorithm. A novel hammer-over-a-nail task is designed to verify the task parameterization technique. Experimental results demonstrate the proposed CEO is superior to the EM in terms of encoding accuracy and the AdC-GMM can achieve more compact representation by reducing the number of components by up to 50%. In addition, the trajectory retrieved by the AdC-GMR is smoother and the approximation error is comparable to the Gaussian process regression (GPR) even far fewer parameters need to be estimated. Because of this, the AdC-GMR is much faster than the GPR. Finally, simulation experiments on the hammer-over-a-nail task demonstrates the proposed methods can be deployed and used in real-world applications.

Keywords

Skill learning Cross entropy Curvilinear Gaussian model Imitation learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

Special thanks to my supervisor Prof. Weijia Zhou for his support on this work and Guangyu Zhang for his help on experimental data collection. Thanks to Nailong Liu for his technical support. We also thank the anonymous reviewers for their comments on a preliminary version of this paper.

References

  1. 1.
    Billard, A.: Learning motor skills by imitation: a biologically inspired robotic model. Cybern. Syst. 32(1-2), 155–193 (2001)CrossRefGoogle Scholar
  2. 2.
    Billard, A.G., Calinon, S., Dillmann, R.: Learning from Humans. In: Springer handbook of robotics, pp. 1995–2014. Springer (2016)Google Scholar
  3. 3.
    Calinon, S.: A tutorial on task-parameterized movement learning and retrieval. Intell. Serv. Robot. 9(1), 1–29 (2016)CrossRefGoogle Scholar
  4. 4.
    Calinon, S., Lee, D.: Learning Control. In: Vadakkepat, P., Goswami, A. (eds.) Humanoid robotics: a reference. Springer (2018)Google Scholar
  5. 5.
    Dillmann, R., Kaiser, M., Ude, A.: Acquisition of elementary robot skills from human demonstration. In: International Symposium on Intelligent Robotics Systems, pp. 185–192 (1995)Google Scholar
  6. 6.
    Gams, A., Nemec, B., Ijspeert, A.J., Ude, A.: Coupling movement primitives: interaction with the environment and bimanual tasks. IEEE Trans. Robot. 30(4), 816–830 (2014)CrossRefGoogle Scholar
  7. 7.
    Gribovskaya, E., Khansari-Zadeh, S., Billard, A.: Learning non-linear multivariate dynamics of motion in robotic manipulators. Int. J. Robot. Res. 30(1), 80–117 (2011).  https://doi.org/10.1177/0278364910376251 CrossRefGoogle Scholar
  8. 8.
    Gribovskaya, E., Khansari-Zadeh, S.M., Billard, A.: Learning non-linear multivariate dynamics of motion in robotic manipulators. Int. J. Robot. Res. 30(1), 80–117 (2011)CrossRefGoogle Scholar
  9. 9.
    Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems, pp. 1547–1554 (2003)Google Scholar
  11. 11.
    Ju, Z., Liu, H.: Fuzzy Gaussian mixture models. Pattern Recogn. 45(3), 1146–1158 (2012).  https://doi.org/10.1016/j.patcog.2011.08.028 CrossRefzbMATHGoogle Scholar
  12. 12.
    Kaiser, M., Dillmann, R.: Building Elementary Robot Skills from Human Demonstration. In: Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 1996, vol. 3, pp. 2700–2705 (1996)Google Scholar
  13. 13.
    Khansari-Zadeh, S., Billard, A.: Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Trans. Robot. 27(5), 943–957 (2011).  https://doi.org/10.1109/TRO.2011.2159412 CrossRefGoogle Scholar
  14. 14.
    Khansari-Zadeh, S. M., Billard, A.: Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Trans. Robot. 27(5), 943–957 (2011)CrossRefGoogle Scholar
  15. 15.
    Khansari-Zadeh, S.M., Billard, A.: A dynamical system approach to realtime obstacle avoidance. Auton. Robot. 32(4), 433–454 (2012). The final publication is available at www.springerlink.com CrossRefGoogle Scholar
  16. 16.
    Kober, J., Peters, J.: Reinforcement learning in robotics: a survey, vol. 12, pp 579–610. Springer, Berlin (2012)CrossRefGoogle Scholar
  17. 17.
    Kronander, K., Khansari, M., Billard, A.: Incremental motion learning with locally modulated dynamical systems. Robot. Auton. Syst. 70(C), 52–62 (2015).  https://doi.org/10.1016/j.robot.2015.03.010 CrossRefGoogle Scholar
  18. 18.
    Lee, J.: A survey of robot learning from demonstrations for human-robot collaboration. arXiv:1710.08789 (2017)
  19. 19.
    Liu, S., Asada, H.: Teaching and Learning of Deburring Robots Using Neural Networks. In: Proceedings of the 1993 IEEE International Conference on Robotics and Automation, 1993, pp. 339–345 (1993)Google Scholar
  20. 20.
    Mayer, H., Nagy, I., Knoll, A.: Skill transfer and learning by demonstration in a realistic scenario of laparoscopic surgery. In: Proceedings of the IEEE International Conference on Humanoids CD-ROM. Munich, Germany (2003)Google Scholar
  21. 21.
    Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.: Learning from demonstration and adaptation of biped locomotion. Robot. Auton. Syst. 47(2), 79–91 (2004)CrossRefGoogle Scholar
  22. 22.
    Nguyen-Tuong, D., Peters, J.R., Seeger, M.: Local Gaussian process regression for real time online model learning. In: Advances in Neural Information Processing Systems, pp. 1193–1200 (2009)Google Scholar
  23. 23.
    Paraschos, A., Daniel, C., Peters, J.R., Neumann, G.: Probabilistic movement primitives. In: Advances in Neural Information Processing Systems, pp. 2616–2624 (2013)Google Scholar
  24. 24.
    Park, D.H., Hoffmann, H., Pastor, P., Schaal, S.: Movement Reproduction and Obstacle Avoidance with Dynamic Movement Primitives and Potential Fields. In: 2008 8Th IEEE-RAS International Conference on Humanoid Robots. Humanoids 2008, pp. 91–98 (2008)Google Scholar
  25. 25.
    Rasmussen, C.E.: Gaussian processes in machine learning. In: Advanced Lectures on Machine Learning, pp. 63–71. Springer (2004)Google Scholar
  26. 26.
    Schaal, S.: Scalable locally weighted statistical techniques for real time robot learning. Appl. Intell. Special Issue Scalable Robot. Appl. Neural Netw. 17, 49–60 (2002)zbMATHGoogle Scholar
  27. 27.
    Schaal, S.: Dynamic movement primitives-a framework for motor control in humans and humanoid robotics. In: Adaptive Motion of Animals and Machines, pp. 261–280. Springer (2006)Google Scholar
  28. 28.
    Sun, K., Mou, S., Qiu, J., Wang, T., Gao, H.: Adaptive fuzzy control for non-triangular structural stochastic switched nonlinear systems with full state constraints. IEEE Trans. Fuzzy Syst., pp. 1–1 (2018)Google Scholar
  29. 29.
    Tabor, J., Spurek, P.: Cross-entropy clustering. Pattern Recogn. 47(9), 3046–3059 (2014)CrossRefGoogle Scholar
  30. 30.
    Vakanski, A., Mantegh, I., Irish, A., Janabi-Sharifi, F.: Trajectory learning for robot programming by demonstration using hidden markov model and dynamic time warping. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 42(4), 1039–1052 (2012)CrossRefGoogle Scholar
  31. 31.
    Vijayakumar, S., D’Souza, A., Schaal, S.: Incremental online learning in high dimensions. Neural Comput. 17(12), 2602–2634 (2005).  https://doi.org/10.1162/089976605774320557 MathSciNetCrossRefGoogle Scholar
  32. 32.
    Zeestraten, M., Havoutis, I., Silvério, J., Calinon, S., Caldwell, D.G.: An approach for imitation learning on Riemannian manifolds. IEEE Robot. Autom. Lett. (RA-L) 2(3), 1240–1247 (2017)CrossRefGoogle Scholar
  33. 33.
    Zhang, B., Zhang, C., Yi, X.: Active curve axis gaussian mixture models. Pattern Recognition 38(12), 2351–2362 (2005).  https://doi.org/10.1016/j.patcog.2005.01.017 CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.State Key Laboratory of Robotics, Shenyang Institute of AutomationChinese Academy of SciencesShenyangChina
  2. 2.Institutes for Robotics and Intelligent ManufacturingChinese Academy of SciencesShenyangChina
  3. 3.University of Chinese Academy of SciencesBeijingChina
  4. 4.Southern University of Science and TechnologyShenzhenChina

Personalised recommendations