Skip to main content
SpringerLink
Log in

Lemma 4: Haptic Input + Auditory Display = Musical Instrument?

  • Conference paper
Haptic and Audio Interaction Design (HAID 2006)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 4129))

Included in the following conference series:

Abstract

In this paper we look at some of the design issues that affect the success of multimodal displays that combine acoustic and haptic modalities. First, issues affecting successful sonification design are explored and suggestions are made about how the language of electroacoustic music can assist. Next, haptic interaction is introduced in the light of this discussion, particularly focusing on the roles of gesture and mimesis. Finally, some observations are made regarding some of the issues that arise when the haptic and acoustic modalities are combined in the interface. This paper looks at examples of where auditory and haptic interaction have been successfully combined beyond the strict confines of the human-computer application interface (musical instruments in particular) and discusses lessons that may be drawn from these domains and applied to the world of multimodal human-computer interaction. The argument is made that combined haptic-auditory interaction schemes can be thought of as musical instruments and some of the possible ramifications of this are raised.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Eisenstein, S., Pudovkin, V., Alexandrov, G.: Statement on sound. In: Taylor, R., Christie, I. (eds.) The Film Factory: Russian and Soviet Cinema in Documents, 1896-1939, pp. 234–235. Harvard University Press, Cambridge (1988)

    Google Scholar 

  2. Hudson, S.E., Smith, I.: Techniques for addressing fundamental privacy and disruption tradeoffs in awareness support systems. In: CSCW 1996: Proceedings of the 1996 ACM conference on Computer supported cooperative work, pp. 248–257. ACM Press, New York (1996)

    Chapter  Google Scholar 

  3. Gutwin, C., Greenberg, S.: Support for group awareness in real time desktop conferences. In: Proceedings of The Second New Zealand Computer Science Research Students Conference, Hamilton, New Zealand, University of Waikato (1995)

    Google Scholar 

  4. Kilander, F., Lönnqvist, P.: A whisper in the woods – an ambient soundscape for peripheral awareness of remote processes. In: ICAD 2002 – International Conference on Auditory Display (2002)

    Google Scholar 

  5. Sawhney, N., Schmandt, C.: Nomadic radio: Scaleable and contextual notification for wearable audio messaging. In: Proceedings of the CHI 1999 Conference on Human factors in computing systems: the CHI is the limit, pp. 96–103. ACM Press, New York (1999)

    Chapter  Google Scholar 

  6. Sawhney, N., Schmandt, C.: Nomadic radio: speech and audio interaction for contextual messaging in no- madic environments. ACM Transactions on Computer-Human Interaction (TOCHI) 7(3), 353–383 (2000)

    Article  Google Scholar 

  7. Pedersen, E.R., Sokoler, T.: Aroma: abstract representation of presence supporting mutual awareness. In: CHI 1997: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 51–58. ACM Press, New York (1997)

    Chapter  Google Scholar 

  8. Somers, E.: A pedagogy of creative thinking based on sonification of visual structures and visualization of aural structures. In: Brewster, S.A., Edwards, A.D.N. (eds.) ICAD 1998 Fifth International Conference on Auditory Display. Electronic Workshops in Computing. British Computer Society, Glasgow (1998)

    Google Scholar 

  9. Fishwick, P. (ed.): Aesthetic Computing. MIT Press, Cambridge (2006)

    Google Scholar 

  10. Pedersen, E.R.: People presence or room activity supporting peripheral awareness over distance. In: CHI 1998: CHI 1998 conference summary on Human factors in computing systems, pp. 283–284. ACM Press, New York (1998)

    Chapter  Google Scholar 

  11. Cohen, J.: Monitoring background activities. In: Kramer, G. (ed.) Auditory Display. Santa Fe Institute, Studies in the Sciences of Complexity Proceedings, vol. XVIII, pp. 499–532. Addison-Wesley, Reading (1994)

    Google Scholar 

  12. Buxton, W.: Introduction to this special issue on nonspeech audio. Human-Computer Interaction 4, 1–9 (1989)

    Google Scholar 

  13. Wrightson, K.: An introduction to acoustic ecology. Soundscape: The Journal of Acoustic Ecology 1(1), 10–13 (2000)

    MathSciNet  Google Scholar 

  14. Schafer, R.M.: The Tuning of the World. Random House (1977)

    Google Scholar 

  15. Søråsen, S.: Monitoring continuous activities with rhythmic music. In: Proceedings of the Student Interaction Design Research Conference, Sønderborg, Mads Clausen Institute, University of Southern Denmark (2005)

    Google Scholar 

  16. Vickers, P., Hogg, B.: Sonification abstraite/sonification concrète: An ‘æsthetic perspective space’ for classifying auditory displays in the ars musica domain. In: Edwards, A.D.N., Stockman, T. (eds.) ICAD 2006 - The 12th Meeting of the International Conference on Auditory Display, London, UK (2006)

    Google Scholar 

  17. Smalley, D.: Spectro-morphology and structuring processes. In: Emmerson, S. (ed.) The Language of Electroacoustic Music, pp. 61–96. Macmillan, London (1986)

    Google Scholar 

  18. Emmerson, S.: The relation of language to materials. In: Emmerson, S. (ed.) The Language of Electroacoustic Music, pp. 17–39. Macmillan, London (1986)

    Google Scholar 

  19. Vickers, P., Alty, J.L.: Musical program auralization: Empirical studies. ACM Trans. Appl. Percept. 2(4), 477–489 (2005)

    Article  Google Scholar 

  20. Gaver, W.W.: Auditory icons: Using sound in computer interfaces. Human Computer Interaction 2, 167–177 (1986)

    Article  Google Scholar 

  21. Barra, M., Cillo, T., De Santis, A., Petrillo, U., Negro, A., Scarano, V.: Multimodal monitoring of web servers. IEEE Multimedia 9(3), 32–41 (2002)

    Article  Google Scholar 

  22. Hayward, C.: Listening to the earth sing. In: Kramer, G. (ed.) Auditory Display. Santa Fe Institute, Studies in the Sciences of Complexity Proceedings, vol. XVIII, pp. 369–404. Addison-Wesley, Reading (1994)

    Google Scholar 

  23. Vickers, P., Alty, J.L.: Siren songs and swan songs: Debugging with music. Communications of the ACM 46(7), 86–92 (2003)

    Article  Google Scholar 

  24. Kramer, G.: An introduction to auditory display. In: Kramer, G. (ed.) Auditory Display. Santa Fe Institute, Studies in the Sciences of Complexity Proceedings, vol. XVIII, pp. 1–78. Addison-Wesley, Reading (1994)

    Google Scholar 

  25. Brewster, S.A., Brown, L.M.: Tactons: Structured tactile messages for non-visual information display. In: Proceedings of Australasian User Interface Conference, pp. 15–23. Austalian Computer Society, Dunedin (2004)

    Google Scholar 

  26. McGee, K., Harup, A.: Contact expressions for touching technologies. In: Mitchell, G. (ed.) Proc. 3rd Conf. Computational Semiotics in Games and New Media, pp. 68–76. University of Teesside, UK (2003)

    Google Scholar 

  27. Magnusson, T.: Screen-based musical interfaces as semiotic machines. In: Schnell, N., Bevilacqua, F., Lyons, M., Tanaka, A. (eds.) Proceedings of the 2006 International Conference on New Interfaces for Musical Expression (NIME 2006), pp. 162–167. IRCAM – Centre Pompidou, Paris (2006)

    Google Scholar 

  28. Winograd, T., Flores, F.: Understanding Computers and Cognition. Ablex Publishing Corp., Norwood (1986)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Engineering, and Information Sciences, Northumbria University, Pandon Building, Camden St., Newcastle upon Tyne, NE2 1XE, UK

    Paul Vickers

Authors
  1. Paul Vickers
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computing Science, University of Glasgow, G12 8QQ, Glasgow, UK

    David McGookin

  2. Glasgow Interactive Systems Group, Department of Computing Science, University of Glasgow, G12 8QQ, Glasgow, UK

    Stephen Brewster

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Vickers, P. (2006). Lemma 4: Haptic Input + Auditory Display = Musical Instrument?. In: McGookin, D., Brewster, S. (eds) Haptic and Audio Interaction Design. HAID 2006. Lecture Notes in Computer Science, vol 4129. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11821731_6

Download citation

  • DOI: https://doi.org/10.1007/11821731_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-37595-1

  • Online ISBN: 978-3-540-37596-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation