Skip to main content
SpringerLink
Log in

Multisensory Experiences: Where the Senses Meet Technology

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2021 (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12932))

Included in the following conference series:

Abstract

Multisensory experiences, that is, experiences that involve more than one of our senses, are part of our everyday life. We often tend to take them for granted, at least when our different senses function normally (normal sight functioning) or are corrected-to-normal (using glasses). However, closer inspection to any, even the most mundane experiences, reveals the remarkable sensory world in which we live in. While we have built tools, experiences and computing systems that have played to the human advantages of hearing and sight (e.g., signage, modes of communication, visual and musical arts, theatre, cinema and media), we have long neglected the opportunities around touch, taste, or smell as interface/interaction modalities. Within this keynote I will share my vision for the future of computing/HCI and what role touch, taste, and smell can play in it.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brianza, G., Tajadura-Jiménez, A., Maggioni, E., Pittera, D., Bianchi-Berthouze, N., Obrist, M.: As light as your scent: effects of smell and sound on body image perception. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) Human-Computer Interaction – INTERACT 2019. INTERACT 2019. Lecture Notes in Computer Science, vol. 11749. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29384-0

  2. Carter, T., Seah, S.A., Long, B., Drinkwater, B., Subramanian, S.: UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In: Proceedings of UIST 2013, pp. 505–514. ACM (2013)

    Google Scholar 

  3. Cornelio, P., Velasco, C., Obrist, M.: Multisensory integration as per technological advances: a review. Front. Neurosci. (2021). DOI: https://doi.org/10.3389/fnins.2021.652611

  4. Hajas, D., Ablart, D., Schneider, O., Obrist, M.: I can feel it moving: science communicators talking about the potential of mid-air haptics. In: Frontiers in Computer Science (2020). https://doi.org/10.3389/fcomp.2020.534974

  5. Lin, Y.-J., et al.: FoodFab: creating food perception tricks using food 3D printing. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020), pp. 1–13. ACM, New York (2020). https://doi.org/10.1145/3313831.3376421

  6. Maggioni, E., Cobden, R., Obrist, M.: OWidgets: a toolkit to enable smell-based experience design. Int. J. Hum. Comput. Stud. 130, 248–260 (2019)

    Article  Google Scholar 

  7. Maggioni, E., Cobden, R., Dmitrenko, D., Hornbæk, K., Obrist, M.: SMELL SPACE: mapping out the olfactory design space for novel interactions. ACM Trans. Comput. Hum. Interact. 27(5), 26 (2020). DOI: https://doi.org/10.1145/3402449. Article 36

  8. Mueller, F., et al.: Next steps in human-computer integration. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020), pp. 1–15. ACM, New York (2020). DOI: https://doi.org/10.1145/3313831.3376242

  9. Obrist, M., Seah, S.A., Subramanian, S.: Talking about Tactile Experiences. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1659–1668. ACM, New York (2013). https://doi.org/10.1145/2470654.2466220

  10. Pittera, D., Gatti, E., Obrist, M.: I'm sensing in the rain: illusion of raindrops through multiple incongruent mid-air tactile stimulation. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI 2019), Paper 132, 15 p. ACM, New York (2019)

    Google Scholar 

  11. Trotta, R., Hajas, D., Camargo-Molina, J.E., Cobden, R., Maggioni, E., Obrist, M.: Communicating cosmology with multisensory metaphorical experiences. In: JCOM 2019, p. 2 (2020). https://doi.org/10.22323/2.19020801. N01

  12. Velasco, C., Obrist, M.: Multisensory Experiences: Where the Senses Meet Technology. Oxford University Press (2020)

    Google Scholar 

  13. Velasco, C., Obrist, M.: Multisensory experiences: a primer. Front. Comput. Sci. (2021). DOI: https://doi.org/10.3389/fcomp.2021.614524

  14. Vi, T.C., Ablart, D., Gatti, E., Velasco, C., Obrist, M.: Not just seeing, but also feeling art: mid-air haptic experiences integrated in a multisensory art exhibition. Int. J. Hum. Comput Stud. 108, 1–14 (2017). https://doi.org/10.1016/j.ijhcs.2017.06.004

    Article  Google Scholar 

  15. Vi, C.T., et al.: LeviSense: a platform for the multisensory integration in levitating food and insights into its effect on flavour perception. Int. J. Hum. Comput. Stud. (2020). https://doi.org/10.1016/j.ijhcs.2020.102428

    Article  Google Scholar 

  16. Weiser, M.: The Computer for the 21st Century. Scientific American (1991)

    Google Scholar 

Download references

Acknowledgements

This work was mainly funded by the European Research Council (ERC) under the European Unions Horizon 2020 Research and Innovation Program under Grant No: 638605. I would like to thank all members of the SCHI Lab for their fantastic contributions to multisensory HCI. Special thanks go to Dr Carlos Velasco, co-author of our book on Multisensory Experiences, that provides the foundation for this keynote.

Author information

Authors and Affiliations

  1. Department of Computer Science, University College London, Gower Street, London, WC1E 6BT, UK

    Marianna Obrist

Authors
  1. Marianna Obrist
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marianna Obrist .

Editor information

Editors and Affiliations

  1. Department of Electrical and Information Engineering, Polytechnic University of Bari, Bari, Italy

    Carmelo Ardito

  2. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Rosa Lanzilotti

  3. Computer Science Department, University of Pisa, Pisa, Italy

    Alessio Malizia

  4. Department of Computer Science, University of York, York, UK

    Helen Petrie

  5. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  6. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Giuseppe Desolda

  7. Microsoft Research, Redmond, WA, USA

    Kori Inkpen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Obrist, M. (2021). Multisensory Experiences: Where the Senses Meet Technology. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12932. Springer, Cham. https://doi.org/10.1007/978-3-030-85623-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85623-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85622-9

  • Online ISBN: 978-3-030-85623-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation