Skip to main content

Advertisement

SpringerLink
Log in

A therapy-driven gamification framework for hand rehabilitation

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation centers or hospitals, leading to slower recovery times for corresponding patients. Therefore, there is a high demand for the development of technology-based personalized solutions to guide and encourage patients towards performing online rehabilitation program that can help them live independently at home. This paper introduces an innovative e-health framework that develops adaptive serious games for people with hand disabilities. The aim of this work is to provide a patient-adaptive environment for the gamification of hand therapies in order to facilitate and encourage rehabilitation issues. Theoretical foundations (i.e., therapy and patient models) and algorithms to match therapy-based hand gestures to navigational movements in 3D space within the serious game environment have been developed. A novel game generation module is introduced, which translates those movements into a 3D therapy-driven route on a real-world map and with different levels of difficulty based on the patient profile and capabilities. In order to enrich the user navigation experience, a 3D spatio-temporal validation region is also generated, which tracks and adjusts the patient movements throughout the session. The gaming environment also creates and adds semantics to different types of attractive and repellent objects in space depending on the difficulty level of the game. Relevant benchmarks to assess the patient interaction with the environment along with a usability and performance testing of our framework are introduced to ensure quantitative as well as qualitative improvements. Trial tests in one disability center were conducted with a total number of five subjects, having hand motor controls problems, who used our gamified physiotherapy solution to help us in measuring the usability and users’ satisfaction levels. The obtained results and feedback from therapists and patients are very encouraging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. https://en.wikipedia.org/wiki/Goniometer

  2. https://dev.windows.com/en-us/kinect.

  3. http://research.microsoft.com/en-us/projects/digits.

  4. https://www.myo.com.

  5. https://www.leapmotion.com.

  6. https://developer.microsoft.com/en-us/windows/kinect/hardware.

  7. https://www.vicon.com/products/camera-systems.

  8. www.mirarehab.com/.

  9. www.jintronix.com.

  10. www.reflexionhealth.com.

  11. www.yugonow.com.

  12. www.gesturetekhealth.com.

  13. http://www.who.int/topics/disabilities/en/.

  14. https://unity3d.com/.

  15. www.jqplot.com/.

  16. https://plot.ly/.

References

  • Afyouni, I., Rehman, F.U., Qamar, A., Ahmad, A., Abdur, M., Basalamah, S.: A GIS-based serious game recommender for online physical therapy. In: Proceedings of the Third International ACM SIGSPATIAL Workshop on HealthGIS (HealthGIS’14), ACM (2014)

  • Ayala, D., Wolfson, O., Xu, B., Dasgupta, B., Lin, J.: Parking slot assignment games. In: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, pp. 299–308 (2011)

  • Bartolomé, N.A., Zorrilla, A.M., Zapirain, B.G.: Can game-based therapies be trusted? Is game-based education effective? A systematic review of the serious games for health and education. In: Proceedings of 2011 16th International Conference on Computer Games (CGAMES), IEEE, pp. 275–282 (2011)

  • Boian, R., Sharma, A., Han, C., Merians, A., Burdea, G., Adamovich, S., Recce, M., Tremaine, M., Poizner, H., et al.: Virtual Reality-Based Post-Stroke Hand Rehabilitation. Studies in Health Technology and Informatics, pp. 64–70. IOS press, Newport beach, CA (2002)

  • Boulanger, C., Boulanger, A., de Greef, L., Kearney, A., Sobel, K., Transue, R., Sweedyk, Z., Dietz, P.H., Bathiche, S.: Stroke rehabilitation with a sensing surface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 1243–1246 (2013)

  • Cameirão, M.S., i Badia, S.B., Oller, E.D., Verschure, P.F.: Neurorehabilitation using the virtual reality based rehabilitation gaming system: methodology, design, psychometrics, usability and validation. J. Neuroeng. Rehabil. 7(1), 1 (2010)

    Article  Google Scholar 

  • Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res. Dev. Disabil. 32(6), 2566–2570 (2011)

    Article  Google Scholar 

  • Chen, Y., Xu, W., Sundaram, H., Rikakis, T., Liu, S.M.: A dynamic decision network framework for online media adaptation in stroke rehabilitation. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 5(1), 4 (2008)

    Google Scholar 

  • Chittaro, L., Carchietti, E., De Marco, L., Zampa, A.: Personalized emergency medical assistance for disabled people. User Model. User-adap. Interact. 21(4–5), 407–440 (2011)

    Article  Google Scholar 

  • Crawford, S., Fuhrmann, S., Li, B., MacEachren, A., Gahegan, M., Downs, R.: Mapstats for kids: developing and assessing web-based tools to foster geographic and statistical literacy in children. In: Proceedings of the Annual National Conference on Digital Government Research, Digital Government Society of North America (2003)

  • Eurostat: healthcare personnel statistics—dentists, pharmacists and physiotherapists. http://ec.europa.eu/eurostat/statistics-explained/index.php/Healthcare_personnel_statistics-_dentists,_pharmacists_and_physiotherapists (2017)

  • Gajdosik, R.L., Bohannon, R.W.: Clinical measurement of range of motion. Phys. Ther. 67(12), 1867–1872 (1987)

    Article  Google Scholar 

  • Gouaïch, A., Hocine, N., Van Dokkum, L., Mottet, D.: Digital-pheromone based difficulty adaptation in post-stroke therapeutic games. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, ACM, pp. 5–12 (2012)

  • Grant, J.A., Mohtadi, N.G., Maitland, M.E., Zernicke, R.F.: Comparison of home versus physical therapy-supervised rehabilitation programs after anterior cruciate ligament reconstruction a randomized clinical trial. Am. J. Sports Med. 33(9), 1288–1297 (2005)

    Article  Google Scholar 

  • Hashemian, Y., Wang, D.: Touchpoints: an exertion game with strategy. In: Proceedings of the first ACM SIGCHI Annual Symposium on Computer–Human Interaction in Play, ACM, pp. 419–420 (2014)

  • Heuser, A., Kourtev, H., Hentz, V., Forducey, P.: Tele-rehabilitation using the rutgers master II glove following carpal tunnel release surgery. In: 2006 International Workshop on Virtual Rehabilitation, IEEE, pp. 88–93 (2006)

  • Hocine, N., Gouaïch, A., Cerri, S.A., Mottet, D., Froger, J., Laffont, I.: Adaptation in serious games for upper-limb rehabilitation: an approach to improve training outcomes. User Model. User-adap. Interact. 25(1), 65–98 (2015)

    Article  Google Scholar 

  • Jacob, R., Mooney, P., Corcoran, P., Winstanley, A.C.: Haptic-GIS: exploring the possibilities. SIGSPATIAL Special 2(3), 13–18 (2010)

    Article  Google Scholar 

  • Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., Cramer, S.C.: Free-hand interaction with leap motion controller for stroke rehabilitation. In: Proceedings of the CHI’14 Extended Abstracts on Human Factors in Computing Systems, ACM, pp. 1663–1668 (2014)

  • Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)

    Article  Google Scholar 

  • Kim, D., Hilliges, O., Izadi, S., Butler, A.D., Chen, J., Oikonomidis, I., Olivier, P.: Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, ACM, pp. 167–176 (2012)

  • Komatireddy, R., Hutchins, S., Shah, M.: Systems, apparatus and methods for non-invasive motion tracking to augment patient administered physical rehabilitation. US Patent App. 13/524,708 (2013)

  • Ma, M., Bechkoum, K.: Serious games for movement therapy after stroke. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, IEEE, pp. 1872–1877 (2008)

  • Ma, M., McNeill, M., Charles, D., McDonough, S., Crosbie, J., Oliver, L., McGoldrick, C.: Adaptive virtual reality games for rehabilitation of motor disorders. In: International Conference on Universal Access in Human–Computer Interaction, Springer, pp. 681–690 (2007)

  • Mayo, M.J.: Games for science and engineering education. Commun. ACM 50(7), 30–35 (2007)

    Article  Google Scholar 

  • Michael, D.R., Chen, S.L.: Serious Games: Games that Educate, Train, and Inform. Muska & Lipman, Cincinnati (2005)

    Google Scholar 

  • Mihelj, M., Novak, D., Milavec, M., Ziherl, J., Olenšek, A., Munih, M.: Virtual rehabilitation environment using principles of intrinsic motivation and game design. Presence Teleoperators Virtual Environ. 21(1), 1–15 (2012)

    Article  Google Scholar 

  • Norkin, C.C., White, D.J.: Measurement of Joint Motion A Guide to Goniometry, chap 6, 4th edn. F. A. Davis, Duxbury (2009)

    Google Scholar 

  • Numrich, S.K.: Culture, models, and games: incorporating warfare’s human dimension. IEEE Intell. Syst. 4(23), 58–61 (2008)

    Article  Google Scholar 

  • Omelina, L., Jansen, B., Bonnechere, B., Van Sint Jan, S., Cornelis, J.: Serious games for physical rehabilitation: designing highly configurable and adaptable games. In: Proceedings of the 9th International Conference on Disability, Virutal Reality and Associated Technologies, pp. 195–201 (2012)

  • Peng, S.T., Hsu, S.Y., Hsieh, K.: An interactive immersive serious game application for Kunyu Quantu world map. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2(5), 221 (2015)

    Article  Google Scholar 

  • Pirovano, M., Mainetti, R., Baud-Bovy, G., Lanzi, P.L., Borghese, N.A.: Self-adaptive games for rehabilitation at home. In: 2012 IEEE Conference on Computational Intelligence and Games (CIG), IEEE, pp. 179–186 (2012)

  • Qamar, A., Afyouni, I., Hossain, D., Ur Rehman, F., Toonsi, A., Abdur Rahman, M., Basalamah, S.: A multimedia e-health framework towards an interactive and non-invasive therapy monitoring environment. In: Proceedings of the ACM International Conference on Multimedia, ACM, pp. 745–746 (2014a)

  • Qamar, A., Afyouni, I., Rahman, M.A., Rehman, F.U., Hussain, D., Basalamah, S., Lbath, A.: A “gis-based” serious game interface for therapy monitoring. In: Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Dallas/Fort Worth, TX, USA, November 4–7, 2014, ACM, pp. 589–592 (2014b)

  • Rabin, B., Burdea, G., Hundal, J., Roll, D., Damiani, F.: Integrative motor, emotive and cognitive therapy for elderly patients chronic post-stroke: a feasibility study of the brightarm rehabilitation system. In: 2011 International Conference on Virtual Rehabilitation, IEEE, pp. 1–8 (2011)

  • Rego, P., Moreira, P.M., Reis, L.P.: Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: Proceedings of the 2010 5th Iberian Conference on Information Systems and Technologies (CISTI), IEEE, pp. 1–6 (2010)

  • Saini, S., Rambli, D.R.A., Sulaiman, S., Zakaria, M.N., Shukri, S.R.M.: A low-cost game framework for a home-based stroke rehabilitation system. In: Proceedings of the International Conference on Computer and Information Science (ICCIS), IEEE, vol. 1, pp. 55–60 (2012)

  • Sawyer, B.: From cells to cell processors: the integration of health and video games. IEEE Comput. Graph. Appl. 28(6), 83–85 (2008)

    Article  Google Scholar 

  • Stone, E.E., Skubic, M.: Passive in-home measurement of stride-to-stride gait variability comparing vision and kinect sensing. In: Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp. 6491–6494 (2011)

  • Stone, E.E., Skubic, M.: Mapping kinect-based in-home gait speed to tug time: a methodology to facilitate clinical interpretation. In: Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, IEEE, pp. 57–64 (2013)

  • Wang, C., Phua, K.S., Ang, K.K., Guan, C., Zhang, H., Lin, R., Chua, K.S.G., Ang, B.T., Kuah, C.W.K.: A feasibility study of non-invasive motor-imagery bci-based robotic rehabilitation for stroke patients. In: Proceedings of the 4th International IEEE/EMBS Conference on Neural Engineering, IEEE, pp. 271–274 (2009)

  • Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013)

    Article  Google Scholar 

  • Zhao, W., Chai, J., Xu, Y.Q.: Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, pp. 33–42 (2012)

Download references

Acknowledgements

This project was supported by the KACST Research program (M-S-37-60) in the Kingdom of Saudi Arabia. We would like to thank all the therapists and doctors that gave us useful feedback of the system and medical specific knowledge in the Disability Children Association in Makkah. Particularly, we thank Dr. Shehnaz Hassan of physiotherapy department of Umm Al-Qura University for her time and efforts. We would also like to thank all the patients who agree to use our framework and participated in the user study and also gave valuable feedback. We kindly acknowledge the useful suggestions from Dr. Mohamed F. Mokbel of University of Minnesota, Dr. Sohaib Khan and Akhlaq Ahmad from Umm Al-Qura University, and Dr. Ahmed Lbath of University of Grenoble Alpes, France. We would like to thank Wadi Makkah Technology Innovation Center at Umm Al-Qura University, Saudi Arabia for providing the necessary resources.

Author information

Authors and Affiliations

  1. Wadi Makkah Technology Innovation Center, UQU, Mecca, Kingdom of Saudi Arabia

    Imad Afyouni, Syed Osama Hussain, Bilal Sadiq & Abdullah Murad

  2. Science and Technology Unit, UQU, Mecca, Kingdom of Saudi Arabia

    Faizan Ur Rehman & Sohaib Ghani

  3. College of Computer and Information Systems, UQU, Mecca, Kingdom of Saudi Arabia

    Ahmad Muaz Qamar, Mohamed Abdur Rahman & Saleh Basalamah

Authors
  1. Imad Afyouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Faizan Ur Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Muaz Qamar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sohaib Ghani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Syed Osama Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bilal Sadiq
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mohamed Abdur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Abdullah Murad
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Saleh Basalamah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imad Afyouni.

Additional information

This paper or a similar version is not currently under review by a journal or conference, nor will it be submitted to such within the next three months. This paper is void of plagiarism or self-plagiarism as defined in Sect. 1 of ACM’s Policy and Procedures on Plagiarism.

Appendices

Appendix 1

Questionnaire for the therapist

 

Questions

Strongly agree

Agree

Not sure

Disagree

Strongly disagree

1

I feel that non-invasive therapy monitoring systems are more convenient for my patients than invasive therapy based systems

     

2

I feel that the proposed system is helpful for in-home rehabilitation in terms of monitoring the correctness and quality of the therapy

     

3

I feel quite comfortable using the system

     

4

I found the overall framework intuitive and easy to use

     

5

The kinematic data provided by the graph plots is useful in the therapy monitoring process

     

6

The ability to increase the level of difficulty in the game according to the patient’s improvement level is helpful in therapy planning

     

7

According to my understanding, providing user specific serious games will have positive impact on patient’s engagement with the therapy process

     

8

I like cloud based storage because it keeps patient data accessible to me all the time

     

9

Provided that the privacy issue is handled, I am willing to use this therapy framework for my patients

     

10

The ability to access patient improvement data helps in evaluating the effectiveness of the therapy

     

11

The framework provides me all the required features for monitoring the complete therapy process

     

12

The ability to produce live kinematic data will be helpful in guiding the patient to perform the therapy in the correct manner

     

Extra notes:

Appendix 2

Questionnaire for the patient

 

Questions

Strongly agree

Agree

Not sure

Disagree

Strongly disagree

1

I have used multimedia based systems for therapy previously

     

2

I find the system easy to use

     

3

I did not face much difficulty in learning how to use the system the first time

     

4

I enjoy seeing my avatar performing exercise on screen

     

5

I find myself motivated to do exercise when using the framework

     

6

I find the serious game appropriate for my rehabilitation

     

7

I feel less fatigue when performing exercise with the system as compared to doing without it

     

8

I enjoy playing the game more than doing the routine exercise

     

9

I am encouraged to exercise more when I observe a measurable quantitative increase in my improvement through graph plots (engaging)

     

10

I find interacting with the system very easy

     

11

I like the fact that more improvement statistics are always available

     

12

I do not feel that I have to exert extra mental effort to perform with the system as compared to doing so without it

     

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Afyouni, I., Rehman, F.U., Qamar, A.M. et al. A therapy-driven gamification framework for hand rehabilitation. User Model User-Adap Inter 27, 215–265 (2017). https://doi.org/10.1007/s11257-017-9191-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-017-9191-4

Keywords

Search

Navigation