Advertisement

Towards the sEMG hand: internet of things sensors and haptic feedback application

  • Gongfa Li
  • Leilei Zhang
  • Ying Sun
  • Jianyi Kong
Article

Abstract

With the trend going on in ubiquitous computing, everything is going to be connected to the Internet and its data will be used for various progressive purposes, creating not only information from it, but also, knowledge and even wisdom. Internet of Things (IoT) is becoming important because the amount of data could make it possible to create more usefulness and develop smart applications for the users. Meanwhile, it mainly focuses on how to enable general objects to see, hear, and smell the physical world for themselves, and make them connected to share the observations. In this paper, we focus our attention on the integration of artificial sensory perception and haptic feedback in sEMG hands, which is an intelligent application of the IoT. Artificial sensory perception and haptic feedback are essential elements for amputees with myoelectric hands to restore the grasping function. They can provide information to users, such as forces of interaction and surface properties at points of contact between hands and objects. Recent advancements in robot tactile sensing led to development of many computational techniques that exploit this important sensory channel. At the same time, Surface electromyography (sEMG) is perhaps most useful for providing insight into how the neuromuscular system behaves. Therefore, integration of sEMG technology, artificial sensation and haptic feedback plays an important role in improving the manipulation performance and enhancing perceptual embodiment for users. This paper provides sEMG technologies that involve Multichannel sEMG electrodes array and processing methods, and then reviews current state-of-the-art of artificial sensation and haptic feedback. Drawing from advancements and taking into design considerations of each feedback modality and individual haptic technology, the paper outline challenging issues and future developments.

Keywords

Internet of things sensors sEMG signal sEMG hands Sensory perception Haptic feedback 

Notes

Acknowledgements

This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 51575412, 61273106) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

References

  1. 1.
    Antfolk C, D'Alonzo M, Controzzi M et al (2013) Artificial redirection of sensation from prosthetic fingers to the phantom hand map on transradial amputees: vibrotactile versus mechanotactile sensory feedback. IEEE Trans Neural Syst Rehab Eng 21(1):112–120CrossRefGoogle Scholar
  2. 2.
    Atieh A, Ahmadi R, Kalantari M et al (2011) A piezoresistive based tactile sensor for use in minimally invasive surgery. Northeast Bioeng Conf 7886(4):1–2Google Scholar
  3. 3.
    Balasubramanian R, Santos VJ (2014) The human hand as an inspiration for robot hand development. Springer Publishing Company, Incorporated 95Google Scholar
  4. 4.
    Bandari NM, Ahmadi R, Hooshiar A et al (2017) Hybrid piezoresistive-optical tactile sensor for simultaneous measurement of tissue stiffness and detection of tissue discontinuity in robot-assisted minimally invasive surgery. J Biomed Opt 22(7):77002CrossRefGoogle Scholar
  5. 5.
    Barbadillo G, Dautenhahn K, Wood L (2011) Using FSR sensors to provide tactile skin to the humanoid robot KASPAR. Comput Sci Tech Rep 511:147–153Google Scholar
  6. 6.
    Bhattacharyya S, Clerc M, Hayashibe M (2016) A study on the effect of electrical stimulation as a user stimuli for motor imagery classification in brain-machine interface. Eur J Translat Myol 26(2):165–168Google Scholar
  7. 7.
    Casini S, Morvidoni M, Bianchi M, et al. (2015) Design and realization of the CUFF-clenching upper-limb force feedback wearable device for distributed mechano-tactile stimulation of normal and tangential skin forces, IEEE Int Conf Intell Robots Syst: 1186–1193Google Scholar
  8. 8.
    Cipriani C, Zaccone F, Micera S et al (2008) On the shared control of an EMG-controlled prosthetic hand: analysis of user-prosthesis interaction. IEEE Trans Robot 24(1):170–184CrossRefGoogle Scholar
  9. 9.
    Cipriani C, D'Alonzo M, Carrozza MC (2012) A miniature vibrotactile sensory substitution device for multifingered hand prosthetics. IEEE Trans Biomed Eng 59(2):400–408CrossRefGoogle Scholar
  10. 10.
    Cloutier A, Yang J (2013) Design, control, and sensory feedback of externally powered hand prostheses: a literature review. Crit Rev Biomed Eng 41(2):161CrossRefGoogle Scholar
  11. 11.
    Cotton DPJ, Chappell PH, Cranny A et al (2007) A novel thick-film piezoelectric slip sensor for a prosthetic hand. IEEE Sensors J 7(5):752–761CrossRefGoogle Scholar
  12. 12.
    Cotton DPJ, Graz IM, Lacour SP (2009) A multifunctional capacitive sensor for stretchable electronic skins. IEEE Sensors J 9(12):2008–2009CrossRefGoogle Scholar
  13. 13.
    Dahiya RS, Metta G, Valle M (2010) Tactile sensing:from humans to humanoids. IEEE Trans Robot 26(1):1–20CrossRefGoogle Scholar
  14. 14.
    Dang H, Allen PK (2014) Stable grasping under pose uncertainty using tactile feedback. Auton Robot 36(4):309–330CrossRefGoogle Scholar
  15. 15.
    Englehart K, Hudgins B, Parker PA, Stevenson M (1999) Classification of the myoelectric signal using time-frequency based representations. Med Eng Phys 21(6):431–438CrossRefGoogle Scholar
  16. 16.
    Fang Y, Liu H, Li G et al. (2015) A multichannel surface EMG system for hand motion recognition. Int J Human Robot 12(2). doi: https://doi.org/10.1142/S0219843615500115.
  17. 17.
    Farina D, Holobar A (2015) Human-machine interfacing by decoding the surface electromyogram [life sciences]. IEEE Signal Process Mag 32(1):115–120CrossRefGoogle Scholar
  18. 18.
    Fishel JA, Loeb GE (2012) Bayesian exploration for intelligent identification of textures. Front Neurorobot 6(4):1–20Google Scholar
  19. 19.
    Gibson A, Artemiadis P (2015) Neural closed-loop control of a hand prosthesis using cross-modal haptic feedback. IEEE Int Conf Rehab Robot: 37–42Google Scholar
  20. 20.
    Gonzalez J, Soma H, Sekine M et al (2012) Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study. J Neuroeng Rehab 9(1):33CrossRefGoogle Scholar
  21. 21.
    Gonzalez J, Suzuki H, Natsumi N, et al. (2012) Auditory display as a prosthetic hand sensory feedback for reaching and grasping tasks’. Eng Med Biol Soc: 1789Google Scholar
  22. 22.
    Gonzalez MG, Sorichetti PA, Brazzano LC et al (2014) Electromechanical characterization of piezoelectric polymer thin films in a broad frequency range. Polym Test 37(8):163–169CrossRefGoogle Scholar
  23. 23.
    Hargrove L, Englehart K, Hudgins B. (2006) The effect of electrode displacements on pattern recognition based myoelectric control. Int Conf IEEE. Eng Med Biol Soc: 2203–2206Google Scholar
  24. 24.
    Hargrove LJ, Englehart K, Hudgins B (2007) A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans Biomed Eng 54(5):847–853CrossRefGoogle Scholar
  25. 25.
    He Y, Li GF, Sun Y et al (2017) Temperature intelligent prediction model of coke oven flue based on CBR and RBFNN. Int J Comput Sci Math.  https://doi.org/10.1007/s10586-017-1237-1
  26. 26.
    Heo JS, Kim JY, Lee JJ (2008) Tactile sensors using the distributed optical fiber sensors. Int Conf Sens Technol 382:486–490Google Scholar
  27. 27.
    Heyneman B, Cutkosky MR (2012) Biologically inspired tactile classification of object-hand and object-world interactions. 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO): 167–173Google Scholar
  28. 28.
    Hosoda K, Tada Y, Asada M (2006) Anthropomorphic robotic soft fingertip with randomly distributed receptors. Robot Auto Syst 54(2):104–109CrossRefGoogle Scholar
  29. 29.
    Huang H, Zhou P, Li G, Kuiken TA (2008) An analysis of EMG electrode configuration for targeted muscle reinnervation based neural machine interface. IEEE Trans Neural Syst Rehab Eng 16(1):37–45CrossRefGoogle Scholar
  30. 30.
    Hudgins B, Englehart K, Parker PA, Scott RN (1997) A microprocessor based multifunction myoelectric control system. 23rd Can Med Biol Eng Soc Conf: 138–139Google Scholar
  31. 31.
    Jara CA, Pomares J, Candelas FA et al (2014) Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors' feedback. Sensors 14(7):1787–1804CrossRefGoogle Scholar
  32. 32.
    Jiang N, Falla D, D'Avella A et al (2010) Myoelectric control in neurorehabilitation. Crit Rev Biomed Eng 38(4):381CrossRefGoogle Scholar
  33. 33.
    Jiang L, Yang B, Huang Q et al (2017) Biomechatronic integration of intelligent prosthetic hand. Robot 39(4):387–394Google Scholar
  34. 34.
    Kaczmarek KA, Webster JG, Bach-Y-Rita P et al (1991) Electrotactile and vibrotactile displays for sensory substitution systems. IEEE Trans Biomed Eng 38(1):1–16CrossRefGoogle Scholar
  35. 35.
    Kappassov Z, Corrales JA, Perdereau V (2015) Tactile sensing in dexterous robot hands-review. North-Holland Publishing Co 74:195–220Google Scholar
  36. 36.
    Kimoto A, Sugitani N, Fujisaki S (2010) A multifunctional tactile sensor based on PVDF films for identification of materials. IEEE Sensors J 10(9):1508–1513CrossRefGoogle Scholar
  37. 37.
    Konstantinova J, Stilli A, Althoefer K (2017) Fingertip fiber optical tactile array with two-level spring structure. Sensors 17(10):2337CrossRefGoogle Scholar
  38. 38.
    Kramer RK, Majidi C, Wood RJ (2011) Wearable tactile keypad with stretchable artificial skin. IEEE Int Conf Robot Auto 47:1103–1107Google Scholar
  39. 39.
    Lang SB, Muensit S (2006) Review of some lesser-known applications of piezoelectric and pyroelectric polymers. Appl Phys A 85(2):125–134CrossRefGoogle Scholar
  40. 40.
    Lee HK, Chung J, Chang SI et al (2011) Real-time measurement of the three-axis contact force distribution using a flexible capacitive polymer tactile sensor. J Micromech Microeng 21(3):035010CrossRefGoogle Scholar
  41. 41.
    Li C, Wu PM, Lee S et al (2008) Flexible dome and bump shape piezoelectric tactile sensors using PVDF-TrFE copolymer. J Microelectromech Syst 17(2):334–341CrossRefGoogle Scholar
  42. 42.
    Li G, Schultz AE, Kuiken TA (2010) Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehab Eng 18:185–192CrossRefGoogle Scholar
  43. 43.
    Li B, Sun Y, Li G et al (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput 3:1–10MathSciNetGoogle Scholar
  44. 44.
    Li K, Fang Y, Zhou Y et al (2017) Non-invasive stimulation-based tactile sensation for upper-extremity prosthesis: a review. IEEE Sensors J 17(9):2625–2635CrossRefGoogle Scholar
  45. 45.
    Liu P, Huang Y, Qiu H et al (2011) Strain and piezoresistive effect in working principle of pressure-sensitive conductive rubber. Polymer Mater Sci Eng 27(5):112–115Google Scholar
  46. 46.
    Liu H, Greco J, Song X, et al. (2012) Tactile image based contact shape recognition using neural network. IEEE Conf Multisensor Fusion Integra Intell Syst 138–143Google Scholar
  47. 47.
    Maereg AT, Nagar AK, Reid D et al (2017) Wearable Vibrotactile haptic device for stiffness discrimination during virtual interactions. Front Robot AI 4:42.  https://doi.org/10.3389/frobt.2017.00042 CrossRefGoogle Scholar
  48. 48.
    Martinez HU, Metta G, Dodd TJ et al. (2013) Active contour following to explore object shape with robot touch. World Haptics Conf IEEE: 341–346Google Scholar
  49. 49.
    Miao W, Li G, Jiang G et al (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Appl Comput Math 14(3):238–247MathSciNetzbMATHGoogle Scholar
  50. 50.
    Ohka M, Morisawa N, Suzuki H et al (2008) A robotic finger equipped with an optical three-axis tactile sensor. IEEE Int Conf Robot Auto 74(740):3425–3430Google Scholar
  51. 51.
    Peerdeman B, Boere D, Witteveen H (2011) Myoelectric forearm prostheses: state of the art from a user-centered perspective. J Rehab Res Dev 48(6):719–737CrossRefGoogle Scholar
  52. 52.
    Rafiee J, Rafiee MA et al (2011) Feature extraction of forearm EMG signals for prosthetics. Expert Syst Appl 38(4):4058–4067MathSciNetCrossRefGoogle Scholar
  53. 53.
    Schmitz A, Maiolino P, Maggiali M et al (2011) Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Trans Robot 27(3):389–400CrossRefGoogle Scholar
  54. 54.
    Schofield JS, Evans KR, Carey JP et al (2014) Applications of sensory feedback in motorized upper extremity prosthesis: a review. Expert Rev Med Dev 11(5):499–511CrossRefGoogle Scholar
  55. 55.
    Seminara L, Pinna L, Valle M et al (2013) Piezoelectric polymer transducer arrays for flexible tactile sensors. IEEE Sensors J 13(10):4022–4029CrossRefGoogle Scholar
  56. 56.
    Shair EF, Zawawi TNST, Abdullah AR et al.(2015) sEMG signals analysis using time-frequency distribution for symmetric and asymmetric lifting. Int Sym Technol Manag Emerg Technol. IEEE: 233–237Google Scholar
  57. 57.
    Stanley AA, Gwilliam JC, Okamura AM (2013) Haptic jamming: a deformable geometry, variable stiffness tactile display using pneumatics and particle jamming. IEEE World Haptics Conf: 25–30Google Scholar
  58. 58.
    Stassi S, Cauda V, Canavese G et al (2014) Flexible tactile sensing based on piezoresistive composites: a review. Sensors 14(3):5296–5332CrossRefGoogle Scholar
  59. 59.
    Sun Y, Li C, Li G et al (2018) Gesture recognition based on Kinect and sEMG signal fusion. Mobile Netw Appl.  https://doi.org/10.1007/s11036-018-1008-0
  60. 60.
    Tegin J, Wikander J (2013) Tactile sensing in intelligent robotic manipulation-a review. Ind Robot 32(1):64–70CrossRefGoogle Scholar
  61. 61.
    Tejeiro C, Stepp CE, Malhotra M et al. (2012) Comparison of remote pressure and vibrotactile feedback for prosthetic hand control. IEEE Ras Embs Int Conf Biomed Robot Biomechatron: 521–525Google Scholar
  62. 62.
    Teshigawara S, Tadakuma K, Ming A et al (2010) High sensitivity initial slip sensor for dexterous grasp. IEEE Int Conf Robot Auto 58(8):4867–4872Google Scholar
  63. 63.
    Tiwana MI, Redmond SJ, Lovell NH (2012) A review of tactile sensing technologies with applications in biomedical engineering. Sens Actuators A Phys 179(3):17–31CrossRefGoogle Scholar
  64. 64.
    Ulrika W, Ingela C (2015) Forearm amputees' views of prosthesis use and sensory feedback. J Hand Therap Off J Am Soc Hand Therap 28(3):269CrossRefGoogle Scholar
  65. 65.
    Wang H, Boer GD et al (2016) Design methodology for magnetic field-based soft tri-axis tactile sensors. Sensors 16(9):1356CrossRefGoogle Scholar
  66. 66.
    Xu D, Loeb GE, Fishel JA (2013) Tactile identification of objects using Bayesian exploration. IEEE Int Conf Robot Auto: 3056–3061Google Scholar
  67. 67.
    Yousef H, Boukallel M, Althoefer K (2011) Tactile sensing for dexterous in-hand manipulation in robotics-a review. Sens Actua A Phys 167(2):171–187CrossRefGoogle Scholar
  68. 68.
    Zardoshti KM, Wheeler BC, Badie K et al (1995) EMG feature evaluation for movement control of upper extremity prostheses. IEEE Trans Rehab Eng 3(4):324–333CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of EducationWuhan University of Science and TechnologyWuhanChina
  2. 2.Research Center for Biomimetic Robot and Intelligent Measurement and ControlWuhan University of Science and TechnologyWuhanChina
  3. 3.Institute of Precision ManufacturingWuhan University of Science and TechnologyWuhanChina
  4. 4.Hubei Key Laboratory of Mechanical Transmission and Manufacturing EngineeringWuhan University of Science and TechnologyWuhanChina

Personalised recommendations