Triggering Robot Hand Reflexes with Human EMG Data Using Spiking Neurons

  • J. Camilo Vasquez TieckEmail author
  • Sandro Weber
  • Terrence C. Stewart
  • Arne Roennau
  • Rüdiger Dillmann
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 867)


The interaction of humans and robots (HRI) is of great relevance for the field of neurorobotics as it can provide insights on motor control and sensor processing mechanisms in humans that can be applied to robotics. We propose a spiking neural network (SNN) to trigger motion reflexes on a robotic hand based on human EMG data. The first part of the network takes EMG signals to measure muscle activity, then classify the data to detect which finger is active in the human hand. The second part triggers single finger reflexes using the classification output. The finger reflexes are modeled with motion primitives activated with an oscillator and mapped to the robot kinematic. We evaluated the SNN by having users wear a non-invasive EMG sensor, record a training dataset, and then flex different fingers, one at a time. The muscle activity was recorded using a Myo sensor with eight channels. EMG signals were successfully encoded into spikes as input for the SNN. The classification could detect the active finger to trigger motion generation of finger reflexes. The SNN was able to control a real Schunk SVH robotic hand. Being able to map myo-electric activity to functions of motor control for a task, can provide an interesting interface for robotic applications, and also to study brain functioning. SNN provide a challenging but interesting framework to interact with human data. In future work the approach will be extended to control a robot arm at the same time.


Human-Robot-Interaction Humanoid robots Neurorobotics Motion representation EMG classification Spiking neural networks Anthropomorphic robot hand 



This research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 720270 (Human Brain Project SGA1) and No. 785907 (Human Brain Project SGA2).


  1. 1.
    Bekolay, T., Bergstra, J., Hunsberger, E., DeWolf, T., Stewart, T.C., Rasmussen, D., Choo, X., Voelker, A., Eliasmith, C.: Nengo: a python tool for building large-scale functional brain models. Front. Neuroinform. 7, 48 (2014)CrossRefGoogle Scholar
  2. 2.
    Bekolay, T., Kolbeck, C., Eliasmith, C.: Simultaneous unsupervised and supervised learning of cognitive functions in biologically plausible spiking neural networks. In: Cogsci (2013)Google Scholar
  3. 3.
    Bernstein, N.: The Co-ordination and Regulation of Movements. Pergamon-Press, Oxford (1967)Google Scholar
  4. 4.
    Bizzi, E., Cheung, V., d’Avella, A., Saltiel, P., Tresch, M.: Combining modules for movement. Brain Res. Rev. 57(1), 125–133 (2008)CrossRefGoogle Scholar
  5. 5.
    Chinellato, E., Pobil, A.: The Visual Neuroscience of Robotic Grasping: Achieving Sensorimotor Skills through Dorsal-Ventral Stream Integration. Cognitive Systems Monographs. Springer, Cham (2016)CrossRefGoogle Scholar
  6. 6.
    Chowdhury, R.H., Reaz, M.B., Ali, M.A.B.M., Bakar, A.A., Chellappan, K., Chang, T.G.: Surface electromyography signal processing and classification techniques. Sensors 13(9), 12431–12466 (2013)CrossRefGoogle Scholar
  7. 7.
    d’Avella, A., Saltiel, P., Bizzi, E.: Combinations of muscle synergies in the construction of a natural motor behavior. Nature Neurosci. 6(3), 300 (2003)CrossRefGoogle Scholar
  8. 8.
    Davies, M., Srinivasa, N., Lin, T.H., Chinya, G., Cao, Y., Choday, S.H., Dimou, G., Joshi, P., Imam, N., Jain, S.: Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 8(1), 82–99 (2018)CrossRefGoogle Scholar
  9. 9.
    dzhu: Myo python api (2018). Accessed 21 Feb 2018
  10. 10.
    Eliasmith, C., Anderson, C.H.: Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems. MIT Press, Cambridge (2003)Google Scholar
  11. 11.
    Furber, S., Temple, S., Brown, A.: High-performance computing for systems of spiking neurons. In: AISB 2006 Workshop. GC5: Archit. Brain Mind (2006)Google Scholar
  12. 12.
    Grüning, A., Bohte, S.M.: Spiking neural networks: principles and challenges. In: ESANN (2014)Google Scholar
  13. 13.
    Heppner, G.: schunk\_svh\_driver (2018). Accessed 21 Feb 2018
  14. 14.
    Johannes, M.S., Bigelow, J.D., Burck, J.M., Harshbarger, S.D., Kozlowski, M.V., Van Doren, T.: An overview of the developmental process for the modular prosthetic limb. Johns Hopkins APL Tech. Digest 30(3), 207–216 (2011)Google Scholar
  15. 15.
    Knierim, J.: Spinal reflexes and descending motor pathways. Neuroscience Online (2016). Accessed 11 Feb 2018Google Scholar
  16. 16.
    Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)CrossRefGoogle Scholar
  17. 17.
    MacNeil, D., Eliasmith, C.: Fine-tuning and the stability of recurrent neural networks. PLoS ONE 6(9), e22885 (2011)CrossRefGoogle Scholar
  18. 18.
    Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)CrossRefGoogle Scholar
  19. 19.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, Kobe, Japan (2009)Google Scholar
  20. 20.
    Tenore, F., Ramos, A., Fahmy, A., Acharya, S., Etienne-Cummings, R., Thakor, N.V.: Towards the control of individual fingers of a prosthetic hand using surface EMG signals. In: EMBC (2007)Google Scholar
  21. 21.
    ThalmicLabs: Myo Diagnostics (2018). Accessed 21 Feb 2018
  22. 22.
    Tieck, J.C.V., Donat, H., Kaiser, J., Peric, I., Ulbrich, S., Roennau, A., Zöllner, M., Dillmann, R.: Towards grasping with spiking neural networks for anthropomorphic robot hands. In: ICANN (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • J. Camilo Vasquez Tieck
    • 1
    Email author
  • Sandro Weber
    • 2
  • Terrence C. Stewart
    • 3
  • Arne Roennau
    • 1
  • Rüdiger Dillmann
    • 1
  1. 1.FZI Research Center for Information TechnologyKarlsruheGermany
  2. 2.TUM Technical University of MunichMünchenGermany
  3. 3.Centre for Theoretical NeuroscienceUniversity of WaterlooWaterlooCanada

Personalised recommendations