Skip to main content

Advertisement

SpringerLink
Log in

An environment for motor skill transfer based on wearable haptic communication

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

We propose an environment for assisting in the transfer of motor skills by supporting nonverbal communication with wearable haptic technology. The proposed setting comprises a human expert and several learners, all endowed with wearable devices. Learners benefit from the automated feedback obtained via wearable technology regarding a desired move relevant to a specific motor skill. Similarly, the expert benefits from automated means to provide instructions to the learner as well as from monitoring data provided by wearable devices. We focus on tactile communication as the basis for this interaction, thus supporting users who are not able to rely on other senses that are relevant to most motor skill-related tasks, such as sight and hearing. We discuss leader-follower dance as a motor skill context in order to instantiate and validate critical components of our skill transfer environment, using accelerometers and gyroscopes as the basis for sensing movements. In this particular scenario, our approach focuses on the haptic communication elements needed to perform a leader-follower dance, assessing movements by the learners in response to the intent of the expert. In order to validate our approach, we have prototyped and conducted experiments with the three critical components of our environment that enables tactile communication between experts and learners. Our development is informed by user research conducted with expert dance teachers, through which we gathered expert dance moves that resulted in a reference database that makes up one of the components that are critical to demonstrate the potential of our proposed environment. We also implemented a tactile encoder that is used to send movement instructions to learners through the sense of touch, as well as an autonomous classifier that uses the reference database as input. Overall, our results provide a roadmap for building an environment for skill transfer using nonverbal communication based on haptic wearable devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Grosshauser T, Hermann T (2009) Augmented haptics–an interactive feedback system for musicians. In International Conference on Haptic and Audio Interaction Design. Springer, Heidelberg, pp 100–108

    Google Scholar 

  2. Fogtmann MH, Fritsch J, Kortbek KJ (2008) Kinesthetic interaction: revealing the bodily potential in interaction design. In Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat (pp. 89-96). ACM

  3. Ludvigsen M, Fogtmann MH, Grønbæk K (2010) TacTowers: an interactive training equipment for elite athletes. In Proceedings of the 8th ACM Conference on Designing Interactive Systems. ACM, pp 412-415

  4. Grønbæk K, Iversen OS, Kortbek KJ, Nielsen KR, Aagaard L (2007) Interactive floor support for kinesthetic interaction in children learning environments. In: IFIP Conference on Human-Computer Interaction. Springer, Heidelberg, pp 361–375

    Google Scholar 

  5. Yokokohji Yasuyoshi, Hollis Ralph L., Kanade Takeo, Henmi Kazuyuki, and Yoshikawa Tsuneo (1996) Toward machine mediated training of motor skills. Skill transfer from human to human via virtual environment. Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN’96 TSUKUBA. IEEE

  6. Chellali A, Dumas C, Milleville-Pennel I (2010) WYFIWIF: a haptic communication paradigm for collaborative motor skills learning. In Web Virtual Reality and Three-Dimensional Worlds 2010. IADIS, pp 301-308

  7. Asadipour A (2020) A technology-aided multi-modal training approach to assist abdominal palpation training and its assessment in medical education. Int J Hum Comput Stud:102394

  8. Lutz J, Memmert D, Raabe D, Dornberger R, Donath L (2020) Wearables for integrative performance and tactic analyses: opportunities, challenges, and future directions. Int J Environ Res Public Health 17(1):59

    Article  Google Scholar 

  9. Georgiou T, Holland S, van der Linden J (2016) Wearable haptic devices for post-stroke gait rehabilitation. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM, pp 1114-1119

  10. Tsetserukou D (2010) HaptiHug: a novel haptic display for communication of hug over a distance. In: International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, Heidelberg, pp 340–347

    Google Scholar 

  11. Van Hooren B, Goudsmit J, Restrepo J, Vos S (2020) Real-time feedback by wearables in running: current approaches, challenges and suggestions for improvements. J Sports Sci 38(2):214–230

    Article  Google Scholar 

  12. Jensen MM, Mueller FF (2014) Running with technology: where are we heading?. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design. ACM, pp 527-530

  13. Yang Chenguang, Liang Peidong, Ajoudani Arash, Li Zhijun, Bicchi Antonio (2016) Development of a robotic teaching interface for human to human skill transfer. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE

  14. Pedemonte N, Laliberté T, Gosselin C (2016) Bidirectional haptic communication: application to the teaching and improvement of handwriting capabilities. Machines 4(1):6

    Article  Google Scholar 

  15. Furukawa M, Kajimoto H, Tachi S. (2012) KUSUGURI: a shared tactile interface for bidirectional tickling. In Proceedings of the 3rd Augmented Human International Conference. ACM, p 9

  16. Damian I, André E (2016) Exploring the potential of realtime haptic feedback during social interactions. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, pp 410-416

  17. Eaves DL, Breslin G, Van Schaik P, Robinson E, Spears IR (2011) The short-term effects of real-time virtual reality feedback on motor learning in dance. Presence Teleop Virt 20(1):62–77

    Article  Google Scholar 

  18. Raina A, Lakshmi TG, Murthy S (2017) CoMBaT: wearable technology based training system for novice badminton players. In 2017 IEEE 17th international Conference on Advanced Learning Technologies (ICALT). IEEE, pp 153-157

  19. Narazani Marla, Seaborn Katie, Hiyama Atsushi, Inami Masahiko (2018) StepSync: wearable skill transfer system for real-time foot-based interaction” Proceedings of the 23rd Annual Meeting of the Virtual Reality Society of Japan

  20. Aggravi, Marco, Gionata Salvietti, Domenico Prattichizzo (2016) Haptic assistive bracelets for blind skier guidance.” Proceedings of the 7th Augmented Human International Conference 2016. ACM

  21. Havlucu H, Eskenazi T, Akgün B, Onbaşlı MC, Coşkun A, Özcan O (2018) Flow state feedback through sports wearables: a case study on tennis. In Proceedings of the 2018 on Designing Interactive Systems Conference 2018. ACM, pp 1025-1039

  22. Gardner, Howard (2005) Intelligence in seven steps. New Horizons For Learning, Creating the Future. Report retrieved December21 (2003)

  23. Camarillo-Abad HM, Sánchez JA, Starostenko O (2019) Organizing knowledge on nonverbal communication mediated through haptic technology. In: Iberoamerican Workshop on Human-Computer Interaction. Springer, Cham, pp 269–283

    Chapter  Google Scholar 

  24. Camarillo-Abad HM, Sanchez JA, Starostenko O, Sandoval MG (2019) A basic tactile language to support leader-follower dancing. J Intell Fuzzy Syst 36(5):5011–5022

    Article  Google Scholar 

  25. Valsted Frederik Mørch, Nielsen Christopher VH, Jensen Jacob Qvist, Sonne Tobias, Jensen Mads Møller (2017) Strive: exploring assistive haptic feedback on the run. Proceedings of the 29th Australian Conference on Computer-Human Interaction. ACM

  26. Matsuyama H, Hiroi K, Kaji K, Yonezawa T, Kawaguchi N (2019) Ballroom dance step type recognition by random forest using video and wearable sensor. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. pp 774-780

  27. Guerreiro J, Sato D, Ahmetovic D, Ohn-Bar E, Kitani KM, Asakawa C (2020) Virtual navigation for blind people: transferring route knowledge to the real-world. Int J Hum Comput Stud 135:102369

    Article  Google Scholar 

  28. Trajkova M (2019) Towards AI-enhanced ballet learning. In Proceedings of the 6th International Conference on Movement and Computing. pp 1-5

  29. Levine D, Cheng A, Olaleye D, Leonardo K, Shifrin M, Ishii H (2019). AUFLIP-an auditory feedback system towards implicit learning of advanced motor skills. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. pp 1-6

  30. Brewster, Stephen, Lorna M. Brown (2004) Tactons: structured tactile messages for non-visual information display. Proceedings of the fifth conference on Australasian user interface-Volume 28. Australian Computer Society, Inc.

  31. Fogtmann MH, Grønbæk K, Ludvigsen MK (2011). Interaction technology for collective and psychomotor training in sports. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology. ACM, p 13

  32. Parikh SP, Esposito JM, Searock J (2014) The role of verbal and nonverbal communication in a two-person, cooperative manipulation task. Adv Hum Comput Interact 2014:9

    Google Scholar 

  33. Gentry S, Murray-Smith R (2003) Haptic dancing: human performance at haptic decoding with a vocabulary. In SMC’03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme-System Security and Assurance (Cat. No. 03CH37483) Vol. 4. IEEE, pp 3432-3437

  34. Barczewska K, Drozd A (2013) Comparison of methods for hand gesture recognition based on dynamic time warping algorithm. In 2013 Federated Conference on Computer Science and Information Systems. IEEE, pp 207-210

  35. Yurtman A, Barshan B (2014) Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals. Comput Methods Prog Biomed 117(2):189–207

    Article  Google Scholar 

  36. Chan JC, Leung H, Tang JK, Komura T (2010) A virtual reality dance training system using motion capture technology. IEEE Trans Learn Technol 4(2):187–195

    Article  Google Scholar 

  37. Matsuyama H, Hiroi K, Kaji K, Yonezawa T, Kawaguchi N (2019) Hybrid activity recognition for ballroom dance exercise using video and wearable sensor. In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR). IEEE, pp 112-117

Download references

Author information

Authors and Affiliations

  1. Universidad de las Américas Puebla, Ex-Hacienda Sta. Catarina Mártir S/N., San Andrés Cholula, Mexico

    Hector M. Camarillo-Abad

  2. National Laboratory of Advanced Informatics (LANIA), Rébsamen 80, 91090, Xalapa, Mexico

    J. Alfredo Sánchez

  3. Universidad de las Américas Puebla, Ex-Hacienda Sta. Catarina Mártir S/N., San Andrés Cholula, Mexico

    Oleg Starostenko

Authors
  1. Hector M. Camarillo-Abad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Alfredo Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oleg Starostenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hector M. Camarillo-Abad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Camarillo-Abad, H.M., Sánchez, J.A. & Starostenko, O. An environment for motor skill transfer based on wearable haptic communication. Pers Ubiquit Comput 25, 411–435 (2021). https://doi.org/10.1007/s00779-020-01425-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01425-z

Keywords

Search

Navigation