“Lucy, Take the Noodle Box!”: Domestic Object Manipulation Using Movement Primitives and Whole Body Motion

  • Alex MitrevskiEmail author
  • Abhishek Padalkar
  • Minh Nguyen
  • Paul G. Plöger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11531)


For robots acting - and failing - in everyday environments, a predictable behaviour representation is important so that it can be utilised for failure analysis, recovery, and subsequent improvement. Learning from demonstration combined with dynamic motion primitives is one commonly used technique for creating models that are easy to analyse and interpret; however, mobile manipulators complicate such models since they need the ability to synchronise arm and base motions for performing purposeful tasks. In this paper, we analyse dynamic motion primitives in the context of a mobile manipulator - a Toyota Human Support Robot (HSR) - and introduce a small extension of dynamic motion primitives that makes it possible to perform whole body motion with a mobile manipulator. We then present an extensive set of experiments in which our robot was grasping various everyday objects in a domestic environment, where a sequence of object detection, pose estimation, and manipulation was required for successfully completing the task. Our experiments demonstrate the feasibility of the proposed whole body motion framework for everyday object manipulation, but also illustrate the necessity for highly adaptive manipulation strategies that make better use of a robot’s perceptual capabilities.


Learning from demonstration Dynamic motion primitives Whole body motion Everyday object manipulation Toyota HSR 



We gratefully acknowledge the support by the b-it International Center for Information Technology. We would like to thank Sven Schneider for all useful discussions and insights.


  1. 1.
    Argall, B.D., Browning, B., Veloso, M.: Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot. Robot. Auton. Syst. 59(3–4), 243–255 (2011)CrossRefGoogle Scholar
  2. 2.
    Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)CrossRefGoogle Scholar
  3. 3.
    Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis: a survey. IEEE Trans. Robot. 30(2), 289–309 (2014)CrossRefGoogle Scholar
  4. 4.
    Chen, N., Bayer, J., Urban, S., van der Smagt, P.: Efficient movement representation by embedding dynamic movement primitives in deep autoencoders. In: IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 434–440 (November 2015)Google Scholar
  5. 5.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)CrossRefGoogle Scholar
  6. 6.
    Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Karlsson, M., Robertsson, A., Johansson, R.: Autonomous interpretation of demonstrations for modification of dynamical movement primitives. In: Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 316–321 (2017)Google Scholar
  8. 8.
    Kirc̀anski, M.V., Boric, M.D.: Symbolic singular value decomposition for a PUMA robot and its application to a robot operation near singularities. Int. J. Robot. Res. 12(5), 460–472 (1993)Google Scholar
  9. 9.
    Kober, J., Peters, J.: reinforcement learning in robotics: a survey, pp. 579–610 (2012)Google Scholar
  10. 10.
    Lee, J.: A survey of robot learning from demonstrations for human-robot collaboration. (2017)
  11. 11.
    Lim, B., Ra, S., Park, F.C.: Movement Primitives, Principal Component Analysis, and the Efficient Generation of Natural Motions. In: Proceedings of the 2005 IEEE international conference on Robotics and Automation, pp. 4630–4635 (April 2005)Google Scholar
  12. 12.
    Lioutikov, R., Kroemer, O., Maeda, G., Peters, J.: Learning manipulation by sequencing motor primitives with a two-armed robot. In: Menegatti, E., Michael, N., Berns, K., Yamaguchi, H. (eds.) Intelligent Autonomous Systems 13. AISC, vol. 302, pp. 1601–1611. Springer, Cham (2016). Scholar
  13. 13.
    Liu, W., et al.: SSD: single shot multibox detector. CoRR. arXiv:1512.02325 (2015)
  14. 14.
    Matamoros, M., Rascon, C., Hart, J., Holz, D., Beek, L.: RoboCup@Home 2018: Rules and Regulations (2018).
  15. 15.
    Nakamura, Y., Hanafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. J. Dyn. Syst. Meas. Control 108(3), 163–171 (1986)CrossRefGoogle Scholar
  16. 16.
    Paraschos, A., Daniel, C., Peters, J., Neumann, G.: Probabilistic movement primitives. In: Proceedings of 26th International Conference on Neural Information Processing Systems, vol. 2, pp. 2616–2624 (2013)Google Scholar
  17. 17.
    Park, D.H., Hoffmann, H., Pastor, P., Schaal, S.: Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In: Humanoids 2008–8th IEEE-RAS International Conference on Humanoid Robots, pp. 91–98 (December 2008)Google Scholar
  18. 18.
    Pettersson, O.: Execution monitoring in robotics: a survey. Robot. Auton. Syst. 53(2), 73–88 (2005)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Provan, G.: Diagnosing hybrid systems using consistency-based methods. In: 29th International Workshop Principles of Diagnosis DX 2018 (2018)Google Scholar
  20. 20.
    Ude, A., Gams, A., Asfour, T., Morimoto, J.: Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Trans. Robot. 26(5), 800–815 (2010)CrossRefGoogle Scholar
  21. 21.
    Wampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Trans. Syst. Man Cybern. 16(1), 93–101 (1986)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alex Mitrevski
    • 1
    Email author
  • Abhishek Padalkar
    • 1
  • Minh Nguyen
    • 1
  • Paul G. Plöger
    • 1
  1. 1.Hochschule Bonn-Rhein-SiegSankt AugustinGermany

Personalised recommendations