Skip to main content
SpringerLink
Log in

A Vibrothermal Haptic Display for Socio-emotional Communication

  • Conference paper
  • First Online:
HCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13095))

Included in the following conference series:

  • 1703 Accesses

Abstract

Touch plays a vital role in maintaining human relationships through social and emotional communication. The proposed haptic display prototype generates stimuli in vibrotactile and thermal modalities toward simulating social touch cues between remote users. High-dimensional spatiotemporal vibrotactile-thermal (vibrothermal) patterns were evaluated with ten participants. The device can be wirelessly operated to enable remote communication. In the future, such patterns can be used to richly simulate social touch cues. A research study was conducted in two parts: first, the identification accuracy of vibrothermal patterns was explored; and second, the relatability of vibrothermal patterns to social touch experienced during social interactions was evaluated. Results revealed that while complex patterns were difficult to identify, simpler patterns, such as SINGLE TAP and HOLD, were highly identifiable and highly relatable to social touch cues. Directional patterns were less identifiable and less relatable to the social touch cues experienced during social interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cang, X.L., Israr, A.: Communicating socio-emotional sentiment through haptic messages. In: IEEE Haptics Symposium HAPTICS (2020). https://research.fb.com/publications/communicating-socio-emotional-sentiment-through-haptic-messages/

  2. Caswell, N.A., Yardley, R.T., Montandon, M.N., Provancher, W.R.: Design of a forearm-mounted directional skin stretch device. In: Proceedings of Haptics Symposium 2012, HAPTICS 2012, pp. 365–370 (2012). https://doi.org/10.1109/HAPTIC.2012.6183816

  3. Chen, Z., Peng, W., Peiris, R., Minamizawa, K.: ThermoReality: Thermally enriched head-mounted displays for virtual reality. In: ACM SIGGRAPH 2017 Posters, SIGGRAPH 2017 1, 1–2 (2017). https://doi.org/10.1145/3102163.3102222

  4. Gallace, A., Spence, C.: The science of interpersonal touch: an overview. Neurosci. Biobehav. Rev. 34(2), 246–259 (2010)

    Article  Google Scholar 

  5. Graham, W., Martin Halvey, S.A.B., Stephen, A.: Huges: thermal icons: evaluating structured thermal feedback for mobile interaction, p. 452 (2012)

    Google Scholar 

  6. Haans, A., IJsselsteijn, W.: Mediated social touch: a review of current research and future directions. Virtual Real. 9(2–3), 149–159 (2006). https://doi.org/10.1007/s10055-005-0014-2

  7. He, L., Xu, C., Xu, D., Brill, R.: Pneuhaptic: delivering haptic cues with a pneu- matic armband. In: ISWC 2015 - Proceedings 2015 ACM Internationa Symposium Wearable Computer, pp. 47–48 (2015). https://doi.org/10.1145/2802083.2802091

  8. Ion, A., Wang, E., Baudisch, P.: Skin drag displays: dragging a physical tac- tor across the user’s skin produces a stronger tactile stimulus than vibrotactile. In: Proceedings of Conference Human Factors Computing System, pp. 2501–2504 (2015). https://doi.org/10.1145/2702123.2702459

  9. Israr, A., Poupyrev, I.: Tactile brush: drawing on skin with a tactile grid display. In: Proceedings of Conference Human Factors Computing Systems, pp. 2019–2028 (2011). https://doi.org/10.1145/1978942.1979235

  10. Israr, A., Zhao, S., Schwalje, K., Klatzky, R., Lehman, J.: Feel effects: enriching storytelling with haptic feedback. ACM Trans. Appl. Percept. 11(3), 1–17 (2014). https://doi.org/10.1145/2641570

    Article  Google Scholar 

  11. Jones, L.A., Ho, H.N.: Warm or cool, large or small? the challenge of thermal displays. IEEE Trans. Haptics 1(1), 53–70 (2008). https://doi.org/10.1109/TOH.2008.2

    Article  Google Scholar 

  12. Ju, Y., Hynds, D., Chernyshov, G., Kunze, K., Zheng, D., Minamizawa, K.: Haptic empathy: conveying emotional meaning through vibrotactile feedback. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (2021)

    Google Scholar 

  13. Katja Suhonen, S.M., Rantala, J., Vaaanaanen-Vainio-Mattila, K., Raisamo, R., Lantz, V.: Haptically augmented remote speech communication: a study of user practices and experiences. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, p. 834 (2012)

    Google Scholar 

  14. Knapp, M.L., Hall, J.A., Horgan, T.G.: Nonverbal Communication in Human Interaction. Cengage Learning (2013)

    Google Scholar 

  15. Krishna, S., Bala, S., McDaniel, T., McGuire, S., Panchanathan, S.: VibroGlove: an assistive technology aid for conveying facial expressions. In: Proceedings of Conference Human Factors Computing Systems, pp. 3637–3642 (2010). https://doi.org/10.1145/1753846.1754031

  16. Lee, W., Lim, Y.K.: Explorative research on the heat as an expression medium: Focused on interpersonal communication. Pers. Ubiquitous Comput. 16(8), 1039–1049 (2012). https://doi.org/10.1007/s00779-011-0424-y

  17. Maeda, T., Kurahashi, T.: Thermodule: wearable and modular thermal feedback system based on a wireless platform. In: ACM International Conference Proceeding Series Association for Computing Machinery, March 2019. https://doi.org/10.1145/3311823.3311826

  18. McDaniel, T., Bala, S., Rosenthal, J., Tadayon, R., Tadayon, A., Panchanathan, S.: Affective haptics for enhancing access to social interactions for individuals who are blind. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014. LNCS, vol. 8513, pp. 419–429. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07437-5_40

    Chapter  Google Scholar 

  19. Niijima, A., Takeda, T., Mukouchi, T., Satou, T.: ThermalBitDisplay: Haptic dis play providing thermal feedback perceived differently depending on body parts. In: Proceedings of Conference Human Factors Computing System Association for Computing Machinery, April 2020. https://doi.org/10.1145/3334480.3382849

  20. Peiris, R.L., Peng, W., Chen, Z., Chan, L., Minamizawa, K.: ThermoVR: exploring integrated thermal haptic feedback with head-mounted displays, pp. 5452–5456 (2017). https://doi.org/10.1145/3025453.3025824

  21. Peiris, R.L., Feng, Y.L., Chan, L., Minamizawa, K.: Thermalbracelet: Exploring thermal haptic feedback around the wrist. In: Proceedings of Conference Human Factors Computing System. Association for Computing Machinery, May 2019. https://doi.org/10.1145/3290605.3300400

  22. Salminen, K., et al.: Emotional and behavioral responses to haptic stimulation. In: Proceddings of Conference Human Factors Computing System, pp. 1555–1562 (2008). https://doi.org/10.1145/1357054.1357298

  23. Salminen, K., et al.: Emotional responses to thermal stimuli. ACM (2011)

    Google Scholar 

  24. Singhal, A., Jones, L.A.: Creating thermal icons - a model-based approach. ACM Trans. Appl. Percept. 15(2), 1–22 (2018). https://doi.org/10.1145/3182175

  25. Song, S., Noh, G., Yoo, J., Oakley, I., Cho, J., Bianchi, A.: Hot & tight: exploring thermo and squeeze cues recognition on wrist wearables. In: ISWC 2015 - Proceedings 2015 ACM International Symposium Wearable Computing, pp. 39–42. Association for Computing Machinery, Inc. September 2015. https://doi.org/10.1145/2802083.2802092

  26. Tewell, J., Bird, J., Buchanan, G.R.: The heat is on: a temperature display for conveying effective feedback. In: Proceedings of Conference Human Factors Computing System, pp. 1756–1767. Association for Computing Machinery, May 2017. https://doi.org/10.1145/3025453.3025844

  27. Trnka, R., Laˇcevlaˇlaˇcev, A., Balcar, K., Kuˇska, M., Tavel, P.: Modeling semantic emotion space using a 3D hypercube-projection: an innovative analytical approach for the psychology of emotions. Front. Psychol 7, 522 (2016). https://doi.org/10.3389/fpsyg.2016.00522, www.frontiersin.org

  28. Turchet, L., Zanotto, D., Minto, S., Roda, A., Agrawal, S.K.: Emotion rendering in plantar vibro-tactile simulations of imagined walking styles. IEEE Trans. Affect. Comput. 8(3), 340–354 (2017). https://doi.org/10.1109/TAFFC.2016.2552515

    Article  Google Scholar 

  29. Wilson, G., Brewster, S., Halvey, M., Hughes, S.: Thermal feedback identification in a mobile environment. In: Oakley, I., Brewster, S. (eds.) HAID 2013. LNCS, vol. 7989, pp. 10–19. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41068-0_2

    Chapter  Google Scholar 

  30. Wilson, G., Davidson, G., Brewster, S.: In the heat of the moment: subjective interpretations of thermal feedback during interaction. In: Conference Human Factors Computing System, pp. 2063–2072. Association for Computing Machinery (2015). https://doi.org/10.1145/2702123.2702219

  31. Wilson, G., Dobrev, D., Brewster, S.A.: Hot under the collar: mapping thermal feedback to dimensional models of emotion. In: Proceedings of Conference Human Factors Computing System, pp. 4838–4849. Association for Computing Machinery, May 2016. https://doi.org/10.1145/2858036.2858205

  32. Wilson, G., Freeman, E., Brewster, S.A.: Multimodal effective feedback: Combining thermal, vibrotactile, audio and visual signals. In: ICMI 2016 – Proceedings of 18th ACM International Conference Multimodal Interaction, pp. 400–401. Association for Computing Machinery, Inc, October 2016. https://doi.org/10.1145/2993148.2998522

  33. Yang, G.H., Kwon, D.S.: Effect of temperature in perceiving tactile stimulus using a thermo-tactile display. In: 2008 International Conference Control Automation System, ICCAS 2008, pp. 266–271 (2008). https://doi.org/10.1109/ICCAS.2008.4694562

  34. Yang, G.H., Kyung, K.U., Srinivasan, M.A., Kwon, D.S.: Development of quantitative tactile display device to provide both pin-array-type tactile feedback and thermal feedback. In: Proceedings - Second Jt. EuroHaptics Conference Symposium Haptic Interfaces Virtual Environment Teleoperator System World Haptics 2007, pp. 578–579 (2007). https://doi.org/10.1109/WHC.2007.41

  35. Yang, G.H., Yang, T.H., Kim, S.C., Kwon, D.S., Kang, S.C.: Compact tactile display for fingertips with multiple vibrotactile actuator and thermoelectric module. In: Proceeidngs of - IEEE International Conference Robotics Automation, pp. 491–496 (2007). https://doi.org/10.1109/ROBOT.2007.363834

  36. Yoo, Y., Lee, H., Choi, H., Choi, S.: Emotional responses of vibrotactile-thermal stimuli: effects of constant-temperature thermal stimuli. In: 2017 7th International Conference Affective Computing Intelligent Interaction. ACII 2017, pp. 273–278 (2018). https://doi.org/10.1109/ACII.2017.8273612

  37. Yoo, Y., Yoo, T., Kong, J., Choi, S.: Emotional responses of tactile icons: effects of amplitude, frequency, duration, and envelope. IEEE World Haptics Conf. WHC 2015, 235–240 (2015). https://doi.org/10.1109/WHC.2015.7177719

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Arizona State University for their funding support through MORE (Mater’s Opportunity for Research in Engineering) Scholarship and GPSA-GRSP (Graduate Research Support Program) for this research project. This work was also supported in part by the National Science Foundation (Grant No. 1828010).

Author information

Authors and Affiliations

  1. School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USA

    Shubham Shriniwas Gharat

  2. The Polytechnic School, Arizona State University, Mesa, AZ, USA

    Yatiraj Shetty & Troy McDaniel

Authors
  1. Shubham Shriniwas Gharat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yatiraj Shetty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Troy McDaniel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yatiraj Shetty .

Editor information

Editors and Affiliations

  1. University of Creteand Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  3. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  4. U.S. Army Combat Capabilities Development Command Soldier Center, Orlando, FL, USA

    Gino Fragomeni

  5. Smart Future Initiative, Frankfurt am Main, Hessen, Germany

    Norbert Streitz

  6. Faculty of Arts and Science, Kyushu University, Fukuoka, Japan

    Shin'ichi Konomi

  7. Siemens (United States), Princeton, NJ, USA

    Helmut Degen

  8. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gharat, S.S., Shetty, Y., McDaniel, T. (2021). A Vibrothermal Haptic Display for Socio-emotional Communication. In: Stephanidis, C., et al. HCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence. HCII 2021. Lecture Notes in Computer Science(), vol 13095. Springer, Cham. https://doi.org/10.1007/978-3-030-90963-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90963-5_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90962-8

  • Online ISBN: 978-3-030-90963-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation