Skip to main content
SpringerLink
Log in

Organizing Knowledge on Nonverbal Communication Mediated Through Haptic Technology

  • Conference paper
  • First Online:
Human-Computer Interaction (HCI-COLLAB 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1114))

Included in the following conference series:

Abstract

Nonverbal communication (NVC) can benefit from any human sense. The sense of touch, or haptic sense, is often used as a channel of NVC. This paper organizes existing knowledge on the use of technology to communicate nonverbally through the haptic sense (HNVC). The analysis of reported work and ongoing projects in the area during the last five years has resulted in an initial taxonomy that is based on three major dimensions, based upon the intent of the messages exchanged: interpretive, affective, and active communication. Thus, haptic devices are used to convey meaning in interpretive HNVC, to convey or to generate emotional reactions in affective HNVC, or to request specific actions or tasks in active HNVC. We characterize existing work using these major categories and various subcategories. This initial organization provides a general overview of all the areas that benefit from technology as a NVC channel. Analysis shows that using the haptic sense as a communication means still has many open research areas and potential new applications, and has proven to date to be an effective mechanism to communicate most of what humans would want to: messages, emotions, and actions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Schroeder, M.L.: A research framework for second nonverbal code acquisition. Globe: J. Lang. Cult. Commun. 5, 103–124 (2017). https://doi.org/10.5278/ojs.globe.v5i0.1943

    Article  Google Scholar 

  2. Hayward, V., Astley, O.R., Cruz-Hernandez, M., Grant, D., Robles-De-La-Torre, G.: Haptic interfaces and devices. Sens. Rev. 24(1), 16–29 (2004). https://doi.org/10.1108/02602280410515770

    Article  Google Scholar 

  3. Huisman, G.: Social touch technology: a survey of haptic technology for social touch. IEEE Trans. Haptics 10(3), 391–408 (2017). https://doi.org/10.1109/TOH.2017.2650221

    Article  Google Scholar 

  4. Mann, S.: Humanistic computing: “WearComp” as a new framework and application for intelligent signal processing. Proc. IEEE 86(11), 2123–2151 (1998). https://doi.org/10.1109/5.726784

    Article  Google Scholar 

  5. MacLean, K.E., Pasquero, J., Smith, J.: Building a haptic language: communication through touch. Computer Science Department, University of British Columbia TR-2005-29, pp. 1–16 (2005). ftp://ftp.cs.ubc.ca/local/techreports/2005/TR-2005-29.pdf. Accessed 11 October 2019

    Google Scholar 

  6. Enriquez, M., MacLean, K., Chita, C.: Haptic phonemes: basic building blocks of haptic communication. In: 8th International Conference Proceedings on Multimodal Interfaces, pp. 302–309. ACM, Canada (2006). https://doi.org/10.1145/1180995.1181053

  7. Chen, J., Turcott, R., Castillo, P., Wahyudinata, S., Lau, F., Israr, A.: Learning to feel words: a comparison of learning approaches to acquire haptic words. In: 15th ACM Symposium Proceedings on Applied Perception, pp. 1–7. ACM, Canada (2018). https://doi.org/10.1145/3225153.3225174

  8. Brewster, S., Brown, L.M.: Tactons: structured tactile messages for non-visual information display. In: Cockburn, A. (eds.) 5th Conference on Australasian User Interface. AUIC 2004 Proceedings, vol. 28, pp. 15–23. Australian Computer Society Inc. (2004)

    Google Scholar 

  9. de J. Oliveira, V.A., Maciel, A.: Assessment of tactile languages as navigation aid in 3D environments. In: Auvray, M., Duriez, C. (eds.) EUROHAPTICS 2014. LNCS, vol. 8619, pp. 104–111. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44196-1_14

    Chapter  Google Scholar 

  10. Schelle, K.J., Gomez, N.C., Bhömer, M.T., Tomico, O., Wensveen, S.: Tactile dialogues: personalization of vibrotactile behavior to trigger interpersonal communication. In: 9th International Conference Proceedings on Tangible, Embedded, and Embodied Interaction, pp. 637–642. ACM, USA (2015). https://doi.org/10.1145/2677199.2687894

  11. Velázquez, R., Pissaloux, E.: Constructing tactile languages for situational awareness assistance of visually impaired people. In: Pissaloux, E., Velázquez, R. (eds.) Mobility of Visually Impaired People, pp. 597–616. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-54446-5_19

    Chapter  Google Scholar 

  12. Brave, S., Nass, C.: Emotion in human-computer interaction. In: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 81–96. L. Erlbaum Associates Inc., Hillsdale (2003). https://doi.org/10.1201/b10368-6

    Google Scholar 

  13. Morrison, A., Knoche, H., Manresa-Yee, C.: Designing a vibrotactile language for a wearable vest. In: Marcus, A. (ed.) DUXU 2015. LNCS, vol. 9187, pp. 655–666. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20898-5_62

    Chapter  Google Scholar 

  14. Haritaipan, L., Mougenot, C.: Cross-cultural study of tactile interactions in technologically mediated communication. In: Rau, P.-L.P. (ed.) CCD 2016. LNCS, vol. 9741, pp. 63–69. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40093-8_7

    Chapter  Google Scholar 

  15. Hirano, T., et al.: Communication cues in a human-robot touch interaction. In: 4th International Conference Proceedings on Human Agent Interaction, pp. 201–206. ACM, USA (2016). https://doi.org/10.1145/2974804.2974809

  16. Ahmed, I., Harjunen, V., Jacicci, G., Hoggan, E.: Reach out and touch me: effects of four distinct haptic technologies on affective touch in virtual reality. In: 18th International Conference Proceedings on Multimodal Interaction, pp. 341–348. ACM, USA (2016). https://doi.org/10.1145/2993148.2993171

  17. Mazzoni, A., Bryan-Kinns, N.: Mood glove: a haptic wearable prototype system to enhance mood music in film. Entertain. Comput. 17, 9–17 (2016). https://doi.org/10.1016/j.entcom.2016.06.002

    Article  Google Scholar 

  18. Goedschalk, L., Bosse, T., Otte, M.: Get your virtual hands off me! – developing threatening IVAs using haptic feedback. In: Verheij, B., Wiering, M. (eds.) BNAIC 2017. CCIS, vol. 823, pp. 61–75. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76892-2_5

    Chapter  Google Scholar 

  19. Bosse, T., Hartmann, T., Blankendaal, R.A.M., Dokter, N., Otte, M., Goedschalk, L.: Virtually bad: a study on virtual agents that physically threaten human beings. In: 17th International Conference Proceedings on Autonomous Agents and MultiAgent Systems, pp. 1258–1266. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2018)

    Google Scholar 

  20. Houde, M.S., Landry, S.P., Pagé, S., Maheu, M., Champoux, F.: Body perception and action following deafness. Neural Plast. 2016, 1–7 (2016). https://doi.org/10.1155/2016/5260671

    Article  Google Scholar 

  21. Bălan, O., Moldoveanu, A., Moldoveanu, F., Butean, A.: Auditory and haptic spatial cognitive representation in the case of the visually impaired people. In: 22nd International Congress Proceedings on Sound and Vibration, Italy, pp. 1–5 (2015)

    Google Scholar 

  22. Kruijff, E., et al.: On your feet!: enhancing vection in leaning-based interfaces through multisensory stimuli. In: 2016 Symposium Proceedings on Spatial User Interaction, pp. 149–158. ACM, Japan (2016). https://doi.org/10.1145/2983310.2985759

  23. Feng, M., Dey, A., Lindeman, R.W.: An initial exploration of a multi-sensory design space: tactile support for walking in immersive virtual environments. In: 2016 IEEE Symposium Proceedings on 3D User Interfaces, pp. 95–104. IEEE, USA (2016). https://doi.org/10.1109/3dui.2016.7460037

  24. Bernardet, U., et al.: m+ m: a novel middleware for distributed, movement based interactive multimedia systems. In: 3rd International Symposium Proceedings on Movement and Computing, pp. 21–30. ACM, USA (2016)

    Google Scholar 

  25. Saunders, W., Vogel, D.: The performance of indirect foot pointing using discrete taps and kicks while standing. In: 41st Graphics Interface Conference Proceedings, pp. 265–272. Canadian Information Processing Society, Canada (2015)

    Google Scholar 

  26. Costes, A., Argelaguet, F., Danieau, F., Guillotel, P., Lécuyer, A.: Touchy: a visual approach for simulating haptic effects on touchscreens. Front. ICT 6, 1–11 (2019). https://doi.org/10.3389/fict.2019.00001

    Article  Google Scholar 

  27. Bufalo, F., Olivari, M., Geluardi, S., Gerboni, C.A., Pollini, L., Bülthoff, H.H.: Variable force-stiffness haptic feedback for learning a disturbance rejection task. In: 2017 IEEE International Conference Proceedings on Systems, Man, and Cybernetics, pp. 1517–1522. IEEE, Canada (2017). https://doi.org/10.1109/smc.2017.8122829

  28. D’Intino, G., et al.: Evaluation of haptic support system for training purposes in a tracking task. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics Proceedings, pp. 2169–2174. IEEE, Hungary (2016). https://doi.org/10.1109/smc.2016.7844560

  29. Van Oosterhout, J., et al.: Haptic shared control in tele-manipulation: effects of inaccuracies in guidance on task execution. IEEE Trans. Haptics 8(2), 164–175 (2015). https://doi.org/10.1109/TOH.2015.2406708

    Article  Google Scholar 

  30. Xu, Y., Yang, C., Liang, P., Zhao, L., Li, Z.: Development of a hybrid motion capture method using MYO armband with application to teleoperation. In: 2016 IEEE International Conference Proceedings on Mechatronics and Automation, pp. 1179–1184. IEEE, China (2016). https://doi.org/10.1109/icma.2016.7558729

  31. Robotti, E.: Designing innovative learning activities to face difficulties in algebra of dyscalculic students: exploiting the functionalities of AlNuSet. In: Leung, A., Baccaglini-Frank, A. (eds.) Digital Technologies in Designing Mathematics Education Tasks. MEDE, vol. 8, pp. 193–214. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-43423-0_10

    Chapter  Google Scholar 

  32. Magana, A.J., et al.: Exploring multimedia principles for supporting conceptual learning of electricity and magnetism with visuohaptic simulations. Comput. Educ. J. 8(2), 8–23 (2017)

    Google Scholar 

  33. Yokokohji, Y., Hollis, R., Kanade, T., Henmi, K., Yoshikawa, T.: Toward machine mediated training of motor skills-skill transfer from human to human via virtual environment. In: 5th IEEE International Workshop Proceedings on Robot and Human Communication, pp. 32–37. IEEE, Japan (1996). https://doi.org/10.1109/roman.1996.568646

  34. Schirmer, M., Hartmann, J., Bertel, S., Echtler, F.: Shoe me the way: a shoe-based tactile interface for eyes-free urban navigation. In: 17th International Conference Proceedings on Human-Computer Interaction with Mobile Devices and Services, pp. 327–336. ACM, USA (2015). https://doi.org/10.1145/2785830.2785832

  35. Prasad, M., Taele, P., Olubeko, A., Hammond, T.: HaptiGo: a navigational ‘tap on the shoulder’. In: 2014 IEEE Haptics Symposium Proceedings, pp. 1–7. IEEE (2014). https://doi.org/10.1109/haptics.2014.6775478

  36. Narazani, M., Seaborn, K., Hiyama, A., Inami, M.: StepSync: wearable skill transfer system for real-time foot-based interaction. In: 23rd Annual Meeting Proceedings, pp. 1–4. Virtual Reality Society of Japan (2018)

    Google Scholar 

  37. Ros, R., Coninx, A., Demiris, Y., Patsis, G., Enescu, V., Sahli, H.: Behavioral accommodation towards a dance robot tutor. In: 9th ACM/IEEE International Conference Proceedings on Human-Robot Interaction, pp. 278–279. ACM, USA (2014). https://doi.org/10.1145/2559636.2559821

  38. Aggravi, M., Salvietti, G., Prattichizzo, D.: Haptic assistive bracelets for blind skier guidance. In: 7th Augmented Human International Conference Proceedings, pp. 1–4. ACM, USA (2016). https://doi.org/10.1145/2875194.2875249

  39. Valsted, F.M., Nielsen, C.V.H., Jensen, J.Q., Sonne, T., Jensen, M.M.: Strive: exploring assistive haptic feedback on the run. In: 29th Australian Conference Proceedings on Computer-Human Interaction, pp. 275–284. ACM, USA (2017). https://doi.org/10.1145/3152771.3152801

  40. Yang, C., Liang, P., Ajoudani, A., Li, Z., Bicchi, A.: Development of a robotic teaching interface for human to human skill transfer. In: 2016 IEEE/RSJ International Conference Proceedings on Intelligent Robots and System, pp. 710–716. IEEE, USA (2016). https://doi.org/10.1109/iros.2016.7759130

  41. Pedemonte, N., Laliberté, T., Gosselin, C.: Bidirectional haptic communication: application to the teaching and improvement of handwriting capabilities. Machines 4(1), 1–15 (2016). https://doi.org/10.3390/machines4010006

    Article  Google Scholar 

  42. Rajanna, V., et al.: KinoHaptics: an automated, wearable, haptic assisted, physio-therapeutic system for post-surgery rehabilitation and self-care. J. Med. Syst. 40(3), 1–12 (2016). https://doi.org/10.1007/s10916-015-0391-3

    Article  Google Scholar 

  43. Ostadabbas, S., et al.: Tongue-controlled robotic rehabilitation: a feasibility study in people with stroke. J. Rehabil. Res. Dev. 53(6), 989–1006 (2016). https://doi.org/10.1682/JRRD.2015.06.0122

    Article  Google Scholar 

  44. Choi, K.-S., Chan, S.-H., Pang, W.-M.: Virtual suturing simulation based on commodity physics engine for medical learning. J. Med. Syst. 36(3), 1781–1793 (2012). https://doi.org/10.1007/s10916-010-9638-1

    Article  Google Scholar 

  45. Uribe, D.O., Schoukens, J., Stroop, R.: Improved tactile resonance sensor for robotic assisted surgery. Mech. Syst. Signal Process. 99, 600–610 (2018). https://doi.org/10.1016/j.ymssp.2017.07.007

    Article  Google Scholar 

  46. Guo, J., Yu, Y., Guo, S., Du, W.: Design and performance evaluation of a novel master manipulator for the robot-assist catheter system. In: 2016 IEEE International Conference Proceedings on Mechatronics and Automation, pp. 937–942. IEEE, USA (2016). https://doi.org/10.1109/icma.2016.7558688

Download references

Author information

Authors and Affiliations

  1. Universidad de las Américas Puebla, Ex-Hacienda Sta. Catarina Mártir S/N. San Andrés, Cholula, Mexico

    Hector M. Camarillo-Abad & Oleg Starostenko

  2. National Laboratory of Advanced Informatics (LANIA), Rébsamen 80, 91090, Xalapa, Mexico

    J. Alfredo Sánchez

Authors
  1. Hector M. Camarillo-Abad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Alfredo Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oleg Starostenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hector M. Camarillo-Abad .

Editor information

Editors and Affiliations

  1. Corporación Universitaria Comfacauca - Unicomfacauca and Universidad del Cauca, Popayán, Colombia

    Pablo H. Ruiz

  2. Corporación Universitaria Comfacauca - Unicomfacauca and Universidad del Cauca, Popayán, Colombia

    Vanessa Agredo-Delgado

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Camarillo-Abad, H.M., Alfredo Sánchez, J., Starostenko, O. (2019). Organizing Knowledge on Nonverbal Communication Mediated Through Haptic Technology. In: Ruiz, P., Agredo-Delgado, V. (eds) Human-Computer Interaction. HCI-COLLAB 2019. Communications in Computer and Information Science, vol 1114. Springer, Cham. https://doi.org/10.1007/978-3-030-37386-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37386-3_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37385-6

  • Online ISBN: 978-3-030-37386-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation