Hand grasp is a complex system that plays an important role in the activities of daily living. Upper-limb neuroprostheses aim at restoring lost reaching and grasping functions on people suffering from neural disorders. However, the dimensionality and complexity of the upper-limb makes the neuroprostheses modeling and control challenging. In this work we present preliminary results for checking the feasibility of using a reinforcement learning (RL) approach for achieving grasp functions with a surface multi-field neuroprosthesis for grasping. Grasps from 20 healthy subjects were recorded to build a reference for the RL system and then two different award strategies were tested on simulations based on neuro-fuzzy models of hemiplegic patients. These first results suggest that RL might be a possible solution for obtaining grasp function by means of multi-field neuroprostheses in the near future.


Neuroprostheses Functional electrical stimulation Grasp Reinforcement learning Modeling and control 



Authors would like to thank to Intelligent Control Research Group of UPV/EHU for giving the means of carrying out this work and to the Health Division of Tecnalia for its continuous support.


  1. 1.
    Jones, L.A., Lederman, S.J.: Human Hand Function. Oxford University Press, Oxford (2006)CrossRefGoogle Scholar
  2. 2.
    Schieber, M.H., Santello, M.: Hand function: peripheral and central constraints on performance. J. Appl. Physiol. 96, 2293–2300 (2004)CrossRefGoogle Scholar
  3. 3.
    Ingram, J.N., Kording, K.P., Howard, I.S., Wolpert, D.M.: The statistics of natural hand movements. Exp. Brain Res. 188, 223–236 (2008)CrossRefGoogle Scholar
  4. 4.
    Castiello, U.: The neuroscience of grasping. Nat. Rev. Neurosci. 6, 726–736 (2005)CrossRefGoogle Scholar
  5. 5.
    Roh, J., Rymer, W.Z., Perreault, E.J., Yoo, S.B., Beer, R.F.: Alterations in upper limb muscle synergy structure in chronic stroke survivors. J. Neurophysiol. 109, 768–781 (2013)CrossRefGoogle Scholar
  6. 6.
    Nichols-Larsen, D.S., Clark, P.C., Zeringue, A., Greenspan, A., Blanton, S.: Factors influencing stroke survivors? quality of life during subacute recovery. Stroke 36, 1480–1484 (2005)CrossRefGoogle Scholar
  7. 7.
    Peckham, P.H., Knutson, J.S.: Functional electrical stimulation for neuromuscular applications. Annu. Rev. Biomed. Eng. 7, 327–360 (2005)CrossRefGoogle Scholar
  8. 8.
    Popović, D.B.: Advances in functional electrical stimulation (FES). J. Electromyogr. Kinesiol. 24, 795–802 (2014)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Keller, T., Kuhn, A.: Electrodes for transcutaneous (surface) electrical stimulation. J. Autom. Control. 18, 35–45 (2008)CrossRefGoogle Scholar
  10. 10.
    Westerveld, A.J., Schouten, A.C., Veltink, P.H., van der Kooij, H.: Selectivity and resolution of surface electrical stimulation for grasp and release. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 94–101 (2012)CrossRefGoogle Scholar
  11. 11.
    Rau, G., Disselhorst-Klug, C., Schmidt, R.: Movement biomechanics goes upwards: from the leg to the arm. J. Biomech. 33, 1207–1216 (2000)CrossRefGoogle Scholar
  12. 12.
    Jagodnik, K.M., Blana, D., van den Bogert, A.J., Kirsch, R.F.: An optimized proportional-derivative controller for the human upper extremity with gravity. J. Biomech. 48, 3692–3700 (2015)CrossRefGoogle Scholar
  13. 13.
    Ethier, C., Oby, E.R., Bauman, M.J., Miller, L.E.: Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485, 368–371 (2012)CrossRefGoogle Scholar
  14. 14.
    Popovic, M.B.: Control of neural prostheses for grasping and reaching. Med. Eng. Phys. 25, 41–50 (2003)CrossRefGoogle Scholar
  15. 15.
    Popovic, D.B., Popovic, M.B.: Automatic determination of the optimal shape of a surface electrode: selective stimulation. J. Neurosci. Meth. 178, 174–181 (2009)CrossRefGoogle Scholar
  16. 16.
    Freeman, C., Rogers, E., Burridge, J.H., Hughes, A., Meadmore, K.: Iterative Learning Control for Electrical Stimulation and Stroke Rehabilitation. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  17. 17.
    Wright, T.W., Glowczewskie, F., Cowin, D., Wheeler, D.L.: Radial nerve excursion and strain at the elbow and wrist associated with upper-extremity motion. J. Hand Surg. 30, 990–996 (2005)CrossRefGoogle Scholar
  18. 18.
    Gillen, G.: Stroke Rehabilitation: A Function-Based Approach. Elsevier Health Sciences, St. Louis (2015)Google Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (1998)zbMATHGoogle Scholar
  20. 20.
    Izawa, J., Kondo, T., Ito, K.: Biological arm motion through reinforcement learning. Biol. Cybern. 91, 10–22 (2004)CrossRefGoogle Scholar
  21. 21.
    Imatz-Ojanguren, E., Irigoyen, E., Valencia-Blanco, D., Keller, T.: Neuro-fuzzy models for hand movements induced by functional electrical stimulation in able-bodied and hemiplegic subjects. Med. Eng. Phys. (2016). (In Press)Google Scholar
  22. 22.
    Yozbatiran, N., Der-Yeghiaian, L., Cramer, S.C.: A standardized approach to performing the action research arm test. Neurorehab. Neural Repair 22, 78–90 (2008)CrossRefGoogle Scholar
  23. 23.
    Helwig, N.E., Hong, S., Hsiao-Wecksler, E.T., Polk, J.D.: Methods to temporally align gait cycle data. J. Biomech. 44, 561–566 (2011)CrossRefGoogle Scholar
  24. 24.
    Vergara, M., Sancho-Bru, J.L., Gracia-Ibanez, V., Perez-Gonzalez, A.: An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand Ther. 27, 225–234 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Eukene Imatz-Ojanguren
    • 1
    • 2
    Email author
  • Eloy Irigoyen
    • 1
  • Thierry Keller
    • 2
  1. 1.Intelligent Control Research GroupUPV/EHU - University of the Basque CountryBilbaoSpain
  2. 2.TECNALIA Research and InnovationDonostia-San SebastiánSpain

Personalised recommendations