Advertisement

Training of Hand Rehabilitation Using Low Cost Exoskeleton and Vision-Based Game Interface

  • Yassine BouteraaEmail author
  • Ismail Ben Abdallah
  • Ahmed M. Elmogy
Article
  • 31 Downloads

Abstract

Motivating game-based training have the potential to improve therapy for people with neurological impairments. In recent years, the serious games have become extremely useful tools in rehabilitation field. They aim to stimulate the mobility of the body through an immersive experience that puts the user in interactive virtual environment. This paper is concerned about developing a customized augmented reality system for stroke rehabilitation. This will be done through integrating an interactive serious game interface with a hand exoskeleton device. This game-based rehabilitation system allows users to carry out physical rehabilitation therapies using a natural user interface based on Kinect’s skeletal tracking features and the electromyography (EMG) sensor. During game playing, the interactive user interface provides useful real-time feedback information such as the time required to grasp a desired dynamic virtual object, and the assigned score and thus the ability of the proposed system to provide a compensatory action regarding the dynamic behavior of the virtual target. The main goal of the developed virtual environment is to create positive influences on the rehabilitation process. Patient movement information and signals obtained from the developed exoskeleton device are used together to monitor the rehabilitation progress. The developed exoskeleton hand is a 3D printed low cost device suitable for grasping tasks that can be used even for domestic stroke patients. The developed exoskeleton device is not only a mechanical system able to perform the rehabilitation act but also it presents an effective tracking and traceability software solution. The EMG signals measured during hand motion are used to detect the intention of hand opening or closing which in turn will actuate the mechanical structure to accomplish the desired task. Parameters and results of patients’ exercises are stored and analyzed when needed to evaluate patients’ progress. The developed system is tested experimentally and it is able to restore the functions of the upper limb and mainly give patients more motivation to undergo the rehabilitation exercises.

Keywords

Stroke rehabilitation Robotic exoskeleton 3D printing EMG control Kinect sensor 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Supplementary material

(MP4 9.46 MB)

10846_2018_966_MOESM2_ESM.mp4 (412 kb)
(MP4 411 KB)

References

  1. 1.
    Aguilar-Pereyra, J.F.: Design of a reconfigurable robotic system for flexoextension fitted to hand fingers size. In: Applied Bionics and Biomechanics (2016)Google Scholar
  2. 2.
    Ajoudani, A., Tsagarakis, N., Bicchi, A.: Tele-impedance: teleoperation with impedance regulation using a body–machine interface. Int. J. Robot. Res. 31, 1642–1656 (2012)CrossRefGoogle Scholar
  3. 3.
    Zhang, X., Chen, X., Li, Y., Lantz, V., Wang, K., Yang, J.: A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Syst. Hum. 41(6), 1064–1076 (2011)CrossRefGoogle Scholar
  4. 4.
    Baoguo, X., Aiguo, S., Guopu, Z., Jia, L., Guozheng, X., Lizheng, P., Renhuan, Y., Huijun, L., Jianwei, C.: Eeg-modulated robotic rehabilitation system for upper extremity. Biotechnol. Biotechnol. Equip. 32 (3), 795–803 (2018)CrossRefGoogle Scholar
  5. 5.
    Benabdallah, I., Bouteraa, Y., Boucetta, R., Rekik, C.: Kinect-based computed torque control for lynxmotion robotic arm. In: 2015 7th International Conference on Modelling, Identification and Control (ICMIC), pp. 1–6 (2015)Google Scholar
  6. 6.
    Borboni, A., Mor, M., Faglia, R.: Gloreha-hand robotic rehabilitation: design, mechanical model, and experiments. J. Dyn. Syst. Meas. Control 138(11), 17–28 (2016)CrossRefGoogle Scholar
  7. 7.
    Bouteraa, Y., Abdallah, I.B.: Exoskeleton robots for upper-limb rehabilitation. In: 2016 13th International Multi-conference on Systems, Signals Devices (SSD), pp. 1–6 (2016)Google Scholar
  8. 8.
    Bouteraa, Y., Ben Abdallah, I.: A gesture-based telemanipulation control for a robotic arm with biofeedback-based grasp. Ind. Robot: Int. J. 44, 575–587 (2017)CrossRefGoogle Scholar
  9. 9.
    Bouteraa, Y., Ghommam, J., Derbel, N., Poisson, G.: Non-linear adaptive synchronisation control of multi-agent robotic systems. Int. J. Syst. Control Commun. 4(1–2), 55–71 (2012)CrossRefGoogle Scholar
  10. 10.
    Bouteraa, Y., Ben Abdallah, I., Ghommam, J.: Task-space region-reaching control for medical robot manipulator. Ind. Robot: Int. J. 67, 629–645 (2018)Google Scholar
  11. 11.
    Brown, E.V.D., McCoy, S.W., Amber S Fechko, R.P., Gilbertson, T., Moritz, C.T.: Preliminary investigation of an electromyography-controlled video game as a home program for persons in the chronic phase of stroke recovery. Arch. Phys. Med. Rehabil. 95, 1461–1469 (2012)CrossRefGoogle Scholar
  12. 12.
    Cameirao, M., Badia, B., Duarte, E., Frisoli, A., Verschure, P.: The combined impact of virtual reality neurorehabilitation and its interfaces on upper extremity functional recovery in patients with chronic stroke. STROKE 43(10), 2720 (2012)CrossRefGoogle Scholar
  13. 13.
    Cesqui, B., Tropea, P., Micera, S., Krebs, H.I.: Emg-based pattern recognition approach in post stroke robot-aided rehabilitation: a feasibility study. J. Neuroeng. Rehabil. 10(1), 75 (2013)CrossRefGoogle Scholar
  14. 14.
    Chaudhary, A., Raheja, J.L., Das, K., Raheja, S., Intelligent approaches to interact with machines using hand gesture recognition in natural way: a survey. ArXiv e-prints (2013)Google Scholar
  15. 15.
    Tsai, C.H., Kuo, Y.H., Chu, K.C., Yen, J.C.: Development and evaluation of game-based learning system using the Microsoft Kinect sensor. International Journal of Distributed Sensor Networks 11(7), 498560 (2015)CrossRefGoogle Scholar
  16. 16.
    Cram, J.R., Kasman, G.S., Holtz, J.: Introduction to surface electromyography, 2nd edn. Jones and Bartlett Publishers, Sudbury (2010)Google Scholar
  17. 17.
    De Luca, C., Donald, L., Mikhail, K., Serge, H.: Filtering the surface emg signal: Movement artifact and baseline noise contamination. J. Biochem. 43(8), 1573–1579 (2010)Google Scholar
  18. 18.
    Dipietro, L., Sabatini, A., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 38, 461–482 (2008)CrossRefGoogle Scholar
  19. 19.
    Dovat, L., Lambercy, O., Gassert, R., Maeder, T., Milner, T., Leong, T.C., Burdet, E.: HandCARE: a cable-actuated rehabilitation system to train hand function after stroke. IEEE Trans. Neural Syst. Rehabil. Eng. 16(6), 582–591 (2008).  https://doi.org/10.1109/TNSRE.2008.2010347 CrossRefGoogle Scholar
  20. 20.
    Gabriele, B., Sami, H., Alberto, S.: A critical analysis of a hand orthosis reverse engineering and 3d printing process. In: Applied Bionics and Biomechanics (2016)Google Scholar
  21. 21.
    Ho, N.S.K., Tong, K.Y., Hu, X.L., Fung, K.L., Wei, X.J., Rong, W., Susanto, E.A.: An emg-driven exoskeleton hand robotic training device on chronic stroke subjects: task training system for stroke rehabilitation. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–5 (2011)Google Scholar
  22. 22.
    Hyesuk, K., Incheol, K.: Dynamic arm gesture recognition using spherical angle features and hidden markov models. In: Advances in Human-Computer Interaction (2015)Google Scholar
  23. 23.
    Zannatha, J.M.I., Tamayo, A.J.M., Sánchez, Á.D.G., Delgado, J.E.L., Cheu, L.E.R., Arévalo, W.A.S.: Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation. Comput. Methods Programs Biomed. 112(2), 239–249 (2013)CrossRefGoogle Scholar
  24. 24.
    Díaz, I., Catalan, J.M., Badesa, F.J., Justo, X., Lledo, L.D., Ugartemendia, A., Gil, J.J., Díez, J., García-Aracil, N.: Development of a robotic device for post-stroke home tele-rehabilitation. Advances in Mechanical Engineering 10(1), 1687814017752302 (2018)CrossRefGoogle Scholar
  25. 25.
    Ismail, B.A., Yassine, B., Chokri, R.: Kinect-based sliding mode control for lynxmotion robotic arm. Adv. Human-Comput. Interact. 7921295:1–7921295:10 (2016)Google Scholar
  26. 26.
    Kiguchi, K.: A study on emg-based human motion prediction for power assist exoskeletons. In: International Symposium on Computational Intelligence in Robotics and Automation, pp. 190–195 (2007)Google Scholar
  27. 27.
    Kofman, J., Verma, S., Wu, X.: Robot-manipulator teleoperation by markerless vision-based handarm tracking. Int. J. Optomechatronics 1, 331–357 (2007)CrossRefGoogle Scholar
  28. 28.
    Kortier, H.G., Sluiter, V.I., Roetenberg, D., Veltink, P.H.: Assessment of hand kinematics using inertial and magnetic sensors. J. Neuroeng. Rehabil. 11(1), 70 (2014)CrossRefGoogle Scholar
  29. 29.
    Kwon, J.S., Park, M.J., Yoon, I.J., Park, S.H.: Effects of virtual reality on upper extremity function and activities of daily living performance in acute stroke: a double-blind randomized clinical trial. NeuroRehabilitation 31, 379–85 (2012)Google Scholar
  30. 30.
    Le, C.H., Vander Sloten, J., Hung, L.T., Khanh, L., Soe, S., Zlatov, N., Phuoc, L., Trung, D.P.: Medical reverse engineering applications and methods. In: 2nd International Conference on Innovations, Recent Trends and Challenges in Mechatronics, Mechanical Engineering and New High-Tech Products Development, MECAHITECH, pp. 232–246 (2010)Google Scholar
  31. 31.
    Lee, G.: Effects of training using video games on the muscle strength, muscle tone, and activities of daily living of chronic stroke patients. J. Phys. Ther. Sci. 25, 595–597 (2013)CrossRefGoogle Scholar
  32. 32.
    Lee, S.W., Wilson, K.M., Lock, B.A., Kamper, D.G.: Subject-specific myoelectric pattern classification of functional hand movements for stroke survivors. IEEE Trans. Neural Syst. Rehabil. Eng. 19(5), 558–566 (2011)CrossRefGoogle Scholar
  33. 33.
    Lee, H.W., Liu, C.H., Chu, K.T., Mai, Y.C., Hsieh, P.C., Hsu, K.C., Tseng, H.C.: Kinect who’s coming—applying kinect to human body height measurement to improve character recognition performance. Smart Science 3, 117–121 (2015)CrossRefGoogle Scholar
  34. 34.
    Mello, R.G.T., Oliveira, L., Nadal, J.: Digital butterworth filter for subtracting noise from low magnitude surface electromyogram. Comput. Methods Prog. Biomed. 87, 28–35 (2007)CrossRefGoogle Scholar
  35. 35.
    Mouawad, M.R., Doust, C.G., Max, M., McNulty, P.: Wii-based movement therapy to promote improved upper extremity function post-stroke: a pilot study. J. Rehabil. Med. 43, 527–533 (2011)CrossRefGoogle Scholar
  36. 36.
    Naik, G.R., Al-Timemy, A.H., Nguyen, H.T.: Transradial amputee gesture classification using an optimal number of semg sensors: an approach using ica clustering. IEEE Trans. Neural Syst. Rehabil. Eng. 24(8), 837–846 (2016)CrossRefGoogle Scholar
  37. 37.
    Naik, G.R., Kumar, D.K.: Twin svm for gesture classification using the surface electromyogram. IEEE Trans. Inf. Technol. Biomed. 14(2), 301–308 (2010)CrossRefGoogle Scholar
  38. 38.
    Phinyomark, A., Phukpattaranont, P., Limsakul, C.: Feature reduction and selection for emg signal classification. Exp. Syst. Appl. 39, 7420–7431 (2012)CrossRefGoogle Scholar
  39. 39.
    Phinyomark, A., Phukpattaranont, P., Limsakul, C.: Fractal analysis features for weak and single-channel upper-limb emg signal. Exp. Syst. Appl. 39, 11156–11163 (2012)CrossRefGoogle Scholar
  40. 40.
    Prochnow, D., Bermúdez i Badia, S., Schmidt, J., Duff, A., Brunheim, S., Kleiser, R., Seitz, R.J., Verschure, P.F.: A functional magnetic resonance imaging study of visuomotor processing in a virtual reality-based paradigm: rehabilitation gaming system. Eur. J. Neurosci. 37, 1441–1447 (2013)CrossRefGoogle Scholar
  41. 41.
    Ferguson, P.W., Dimapasoc, B., Shen, Y., Rosen, J.: Design of a hand exoskeleton for use with upper limb exoskeletons. In: International Symposium on Wearable Robotics, pp. 276–280. Springer, Cham (2018)Google Scholar
  42. 42.
    Sin, H., Lee, G.: Additional virtual reality training using xbox kinect in stroke survivors with hemiplegia. Am. J. Phys. Med. Rehab./Association of Academic Physiatrists 92, 871–80 (2013)CrossRefGoogle Scholar
  43. 43.
    Sushant, N., Suresh, D., Rajesh, K.S.: Basics and applications of rapid prototyping medical models. Rapid Prototyp. J. 20(3), 256–267 (2014)CrossRefGoogle Scholar
  44. 44.
    Takahashi, C.D., Der-Yeghiaian, L., Le, V., Motiwala, R.R., Cramer, S.C.: Robot-based hand motor therapy after stroke. Brain 131(2), 425–437 (2008)Google Scholar
  45. 45.
    Turolla, A., Dam, M., Ventura, L., Tonin, P., Agostini, M., Zucconi, C., Kiper, P., Cagnin, A., Piron, L.: Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. J. Neuroeng. Rehabil. 10, 85 (2013)CrossRefGoogle Scholar
  46. 46.
    Zhang, X., Yue, Z., Wang, J.: Robotics in lower-limb rehabilitation after stroke. Behav. Neurol. 2017, 13 (2017). Article ID 3731802Google Scholar
  47. 47.
    Yasemin, C., Abdullah, Y., Mihriban, Y., Ayse Esra, K., Gokhan, K., Alpaslan, G., Gul, G., Huseyin, U., Meliha, K.: Evaluation of invasive and noninvasive methods for the diagnosis of helicobacter pylori infection. Asian Pac. J. Cancer Prev.: APJCP: APJCP 17(12), 5265–5272 (2016)Google Scholar
  48. 48.
    Yates, M., Kelemen, A., Sik-Lányi, C.: Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Inj. 30, 1–9 (2016)CrossRefGoogle Scholar
  49. 49.
    Yeow, C.H., Baisch, A.T., Talbot, S.G., Walsh, C.J.: Cable-driven finger exercise device with extension return springs for recreating standard therapy exercises. ASME J. Med. Devices 8, 014502 (2014)CrossRefGoogle Scholar
  50. 50.
    Yue, Z., Zhang, X., Wang, J.: Hand rehabilitation robotics on poststroke motor recovery. Behav. Neurol. 2017, 20 (2017). Article ID 3908135CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  • Yassine Bouteraa
    • 1
    • 2
    Email author
  • Ismail Ben Abdallah
    • 2
  • Ahmed M. Elmogy
    • 1
    • 3
  1. 1.Prince Sattam Bin Abdulaziz UniversityAl KharjSaudi Arabia
  2. 2.Digital Research Center of Sfax, CEM Lab-ENISUniversity of SfaxSfaxTunisia
  3. 3.Faculty of EngineeringTanta UniversityTantaEgypt

Personalised recommendations