AR-PETS: Development of an Augmented Reality Supported Pressing Evaluation Training System

  • Alexander PlopskiEmail author
  • Ryosuke Mori
  • Takafumi Taketomi
  • Christian Sandor
  • Hirokazu Kato
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10927)


With age, changes to the human nervous system lead to a decrease in control accuracy of extremities. Especially, reduced control over the fingers gravely affects a person’s quality of life and self-reliance. It is possible to recover the ability to accurately control the amount of power applied with one’s fingers through training with the Pressing Evaluation Training System (PETS). When training with the PETS users have to focus on guidance provided on a monitor and lose sight of their fingers. This could lead to increased mental workload and reduced training efficiency. In this paper we explore if presenting the guidelines closer to the user’s fingers provides better guidance to the user, thus improving the performance results. In particular, we use a video-see-through head-mounted display to present guidance next to the user’s fingers through augmented reality (AR), and a haptic device to replicate the tasks during PETS training. We test our implementation with 18 university students. Although the results of our study indicate that presenting information closer to the interaction area does not improve the performance, several participants preferred guidance presented in AR.


Video See-Through Head-Mounted Display Augmented Reality Haptic device Force sensing Elderly training Pressing Evaluation Training System 



This research was supported by Japan Science and Technology Agency as part of Infrastructure Development for Promoting International S&T Cooperation.


  1. 1.
  2. 2.
    Asghar, Z., Keränen, N., Yamamoto, G., Taketomi, T., Sandor, C., Kato, H.: Remote Assistance for Elderly to Find Hidden Objects in a Kitchen. EAI Endorsed Transactions on Pervasive Health and Technology 3(12), 1–7 (2017)CrossRefGoogle Scholar
  3. 3.
    Buker, T.J., Vincenzi, D.A., Deaton, J.E.: The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation. Hum. factors 54(2), 235–249 (2012)CrossRefGoogle Scholar
  4. 4.
    Burke, J.W., McNeill, M., Charles, D., Morrow, P.J., Crosbie, J., McDonough, S.: Augmented reality games for upper-limb stroke rehabilitation. In: International Conference on Games and Virtual Worlds for Serious Applications, pp. 75–78 (2010)Google Scholar
  5. 5.
    Grabiner, M.D., Enoka, R.M.: Changes in movement capabilities with aging. Exerc. Sport Sci. Rev. 23(1), 65–104 (1995)Google Scholar
  6. 6.
    Huber, M., Pustka, D., Keitler, P., Echtler, F., Klinker, G.: A system architecture for ubiquitous tracking environments. In: Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 211–214 (2007)Google Scholar
  7. 7.
    Hyry, J., Krichenbauer, M., Yamamoto, G., Taketomi, T., Sandor, C., Kato, H., Pulli, P.: Design of assistive tabletop projector-camera system for the elderly with cognitive and motor skill impairments. ITE Trans. Media Technol. Appl. 5(2), 57 (2017)CrossRefGoogle Scholar
  8. 8.
    Keogh, J.W., Morrison, S., Barrett, R.: Strength training improves the tri-digit finger-pinch force control of older adults. Arch. Phys. Med. Rehabil. 88(8), 1055–1063 (2007)CrossRefGoogle Scholar
  9. 9.
    Kim, S., Dey, A.K.: Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 133–142 (2009)Google Scholar
  10. 10.
    Kwok, T.C., Lam, K., Wong, P., Chau, W., Yuen, K.S., Ting, K., Chung, E.W., Li, J.C., Ho, F.K.: Effectiveness of coordination exercise in improving cognitive function in older adults: a prospective study. Clin. Interv. Aging 6, 261–267 (2011)Google Scholar
  11. 11.
    Luo, X., Kline, T., Fischer, H.C., Stubblefield, K.A., Kenyon, R.V., Kamper, D.G.: Integration of augmented reality and assistive devices for post-stroke hand opening rehabilitation. In: International Annual Conference of the Engineering in Medicine and Biology Society, pp. 6855–6858 (2005)Google Scholar
  12. 12.
    Mousavi, H.H., Khademi, M., Dodakian, L., Cramer, S., Lopes, C.: A spatial augmented reality rehab system for post-stroke hand rehabilitation. Stud. Health Technol. Inf. 184, 279–285 (2013)Google Scholar
  13. 13.
    Oculus VR: Oculus Rift. Accessed 28 Feb 2018.
  14. 14.
    Olafsdottir, H.B., Zatsiorsky, V.M., Latash, M.L.: The effects of strength training on finger strength and hand dexterity in healthy elderly individuals. J. Appl. Physiol. 105(4), 1166–1178 (2008)CrossRefGoogle Scholar
  15. 15.
    Razer Inc.: Orbweaver Chroma. Accessed 28 Feb 2018.
  16. 16.
    Shen, Y., Gu, P., Ong, S.K., Nee, A.Y.: A novel approach in rehabilitation of hand-eye coordination and finger dexterity. Virtual Real. 16(2), 161–171 (2012)CrossRefGoogle Scholar
  17. 17.
    Shinohara, M., Li, S., Kang, N., Zatsiorsky, V.M., Latash, M.L.: Effects of age and gender on finger coordination in MVC and submaximal force-matching tasks. J. Appl. Physiol. 94(1), 259–270 (2003)CrossRefGoogle Scholar
  18. 18.
    Tsai, R.Y., Lenz, R.K.: Real time versatile robotics hand/eye calibration using 3D machine Cision. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 554–561 (1988)Google Scholar
  19. 19.
    Unity Technologies: Unity 2017.1.0. Accessed 28 Feb 2018.
  20. 20.
    Wizapply: Ovrvision Pro. Accessed 28 Feb 2018.
  21. 21.
    Wu, Y.H., Pazin, N., Zatsiorsky, V.M., Latash, M.L.: Improving finger coordination in young and elderly persons. Exp. Brain Res. 226(2), 273–283 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Alexander Plopski
    • 1
    Email author
  • Ryosuke Mori
    • 1
  • Takafumi Taketomi
    • 1
  • Christian Sandor
    • 1
  • Hirokazu Kato
    • 1
  1. 1.Nara Institute of Science and TechnologyNaraJapan

Personalised recommendations