Intelligent Service Robotics

, Volume 12, Issue 1, pp 17–25 | Cite as

Hand-arm autonomous grasping: synergistic motions to enhance the learning process

  • Fanny FicucielloEmail author
Original Research Paper


In this paper, a reinforcement learning algorithm based on policy search methods has been developed for an anthropomorphic hand-arm system. The advantages of searching in the space of a policy are that the convergence and fastness of the algorithm rely on the proper choice of the policy and of its initial parameters together with the choice of the reward function. The main contribution of the paper consists in the computation of a suitable policy based on the synergies of the hand-arm system, and in the development of a supervised learning strategy to define the initial parameters of the policy. Finally, the reward function has been chosen to evaluate the goodness of the grasp in a synergy-based framework. The experiments executed on the robotic hand-arm system, constituted by the SCHUNK 5-Finger Hand and the KUKA LWR4+, demonstrate the convergence of the method and the promising results obtained for the execution of grasping tasks.


Hand-arm manipulation Dimensionality reduction Learning based on postural synergies 



The author wants to thank Damiano Zaccara which contributed to the development of the set-up and supported the experiments described in this paper.


  1. 1.
    Ficuciello F, Siciliano B (2016) Learning in robotic manipulation: the role of dimensionality reduction in policy search methods. Comment on Hand synergies: integration of robotics and neuroscience for understanding the control of biological and artificial hands by Marco Santello, Matteo Bianchi, Marco Gabiccini, Emiliano Ricciardi, Gionata Salvietti et al. Physics of Life ReviewsGoogle Scholar
  2. 2.
    Ficuciello F (2018) Synergy-based Control of Underactuated Anthropomorphic Hands. IEEE Trans Ind Inform (in press)Google Scholar
  3. 3.
    Suarez R, Rosell J, Garcia N (2015) Using synergies in dual-arm manipulation tasks. In: Proceedings of the 2000 IEEE international conference on robotics and automationGoogle Scholar
  4. 4.
    Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Siciliano B, Khatib O (eds) Springer handbook of robotic. Springer, BerlinGoogle Scholar
  5. 5.
    Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483CrossRefGoogle Scholar
  6. 6.
    Kormushev P, Calinon S, Caldwell D (2007) Reinforcement learning in robotics: applications and real-world challenges. Robotics 2:122–148CrossRefGoogle Scholar
  7. 7.
    Romero J, Feix T, Ek C, Kjellstrom H, Kragic D (2013) A metric for comparing the anthropomorphic motion capability of artificial hands. IEEE Trans Robot 29:1342–1352CrossRefGoogle Scholar
  8. 8.
    Theodorou E, Buchli J, Schaal S (2010) Learning policy improvements with path integrals. J Mach Learn Res 9:828–835zbMATHGoogle Scholar
  9. 9.
    Stulp F, Theodorou A, Schaal S (2012) Reinforcement learning with sequences of motion primitives for robust manipulation. IEEE Trans Robot 28(6):1360–1370CrossRefGoogle Scholar
  10. 10.
    Hansen N (2006) The CMA evolution strategy: a comparing review. In: Lozano J, Larranaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation. Springer, Berlin, pp 75–102CrossRefGoogle Scholar
  11. 11.
    Stulp F, Sigaud O (2012) Path integral policy improvement with covariance matrix adaptation, In: Proceedings of the 29th international conference on machine learning, Edinburgh, Scotland, pp 2112–2118Google Scholar
  12. 12.
    Theodorou E, Buchli J, Schaal F (2010) A generalized path integral control approach to reinforcement learning. J Mach Learn Res 11:3137–3181MathSciNetzbMATHGoogle Scholar
  13. 13.
    Kormushev P, Calinon S, Caldwell D (2010) Robot motor skill coordination with EM-based reinforcement learning, In: Proceedings IEEE/RSJ international conference on intelligent robotics systems, Taipei, Taiwan, pp 3232–3237Google Scholar
  14. 14.
    Rombokas E, Malhotra M, Theodorou E, Todorov E, Matsuoka Y (2012) Reinforcement learning and synergistic control of the act hand. IEEE/ASME Trans Mechatron 18(2):569–577CrossRefGoogle Scholar
  15. 15.
    Santello M, Flanders M, Soechting J (1998) Postural hand synergies for tool use. J Neurosci 18(23):10105–10115CrossRefGoogle Scholar
  16. 16.
    Bicchi A (1994) On the closure properties of robotic grasping. Int j Robot Res 14(4):319–334CrossRefGoogle Scholar
  17. 17.
  18. 18.
    Ficuciello F, Zaccara D, Siciliano B (2016) Synergy-based policy improvement with path integrals for anthropomorphic hands. In: IEEE/RSJ international conference on intelligent robots and systems, Daejeon, Korea, 9–14 SeptemberGoogle Scholar
  19. 19.
    Ficuciello F, Palli G, Melchiorri C, Siciliano B (2014) Postural synergies of the ub hand iv for human-like grasping. Robot Auton Syst 62:357–362CrossRefGoogle Scholar
  20. 20.
    Palli G, Melchiorri C, Vassura G, Scarcia U, Moriello L, Berselli G, Cavallo A, Maria GD, Natale C, Pirozzi S, May C, Ficuciello F, Siciliano B (2014) The dexmart hand: mechatronic design and experimental evaluation of synergy-based control for human-like grasping. Int J Robot Res 33:799–824CrossRefGoogle Scholar
  21. 21.
    Jarrasse N, Ribeiro AT, Sahbani A, Bachta W, Roby-Brami A (2014) Analysis of hand synergies in healthy subjects during bimanual manipulation of various objects. J Neuroeng Rehabil 11:113CrossRefGoogle Scholar
  22. 22.
    Ficuciello F, Zaccara D, Siciliano B (2016) Learning grasps in a synergy-based framework. In: 15th International symposium on experimental robotics, Tokyo, Japan, 3–6 OctoberGoogle Scholar
  23. 23.
  24. 24.
    Deisenroth M, Rasmussen CE (2011) Pilco: a model-based and dataefficient approach to policy search. In: Proceedings of the 28th international conference on machine learning, pp 465472Google Scholar
  25. 25.
    Saveriano M, Yin Y, Falco P, Lee D (2017) Data-efficient control policy search using residual dynamics learning. In: IEEE/RSJ international conference on intelligent robots and systems, Vancouver, BC, 2017, pp. 4709–4715Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Dipartimento di Ingegneria Elettrica e Tecnologie dell’InformazioneUniversità degli Studi di Napoli Federico IINaplesItaly

Personalised recommendations