Skip to main content
SpringerLink
Log in

Human Factors Research in Audio Augmented Reality

  • Chapter
  • First Online:
Human Factors in Augmented Reality Environments

Abstract

Within the burgeoning field of augmented reality (AR), audio AR is a medium whereby 2D or 3D spatial audio is registered to the user’s surroundings, using their position and head-orientation. In locative audio applications, such as several that have been released for smart-phones, audio is presented only in relationship to the user’s location. Audio AR that uses both these sensor inputs has the advantage that both the semantic and directional information content of the aural presentation may be perceived without paying attention to the hardware device. In comparison to visual AR, which often requires engaging with a screen on a mobile device, audio AR enables hands-free, passive reception of AR content. Human spatial hearing allows awareness of the direction and distance of virtual sonic objects throughout the full sphere, in comparison to the limited momentary field of view for visual AR. This chapter situates the author’s research within a wider survey of human factors research in audio AR systems. It provides a taxonomy of audio AR systems, and a framework for considering human factors in terms of users’ abilities and affordances of the systems. Research is reviewed that examines the perceptual and task-related performance shortfalls between ideal audio AR and what is achievable with practical, current technology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Azuma, R.: A survey of augmented reality. Presence: Teleoperators and Virtual Environment 6(4), 355–385 (1997)

    Google Scholar 

  2. Bederson, B.B.: Audio augmented reality: A prototype automated tour guide. In: CHI. pp. 210–211 (May 7–11 1995)

    Google Scholar 

  3. Begault, D.R.: 3-D Sound for Virtual Reality and Multimedia. Academic Press Professional, Cambridge, MA (1994)

    Google Scholar 

  4. Begault, D.R., Wenzel, E.M., Anderson, M.R.: Direct comparison of the impact of head tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. J of the Audio Eng Soc 49(10), 904–916 (2001)

    Google Scholar 

  5. Best, V.: Spatial Hearing with Simultaneous Sound Sources: A Psychophysical Investigation. Ph.D. thesis, Faculty of Medicine, University of Sydney (April 2004)

    Google Scholar 

  6. Blauert, J.: Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge, MA (1983)

    Google Scholar 

  7. Blauert, J.: Spatial hearing: The psychophysics of human sound localization (rev. ed.). MIT Press, Cambridge, MA (1997)

    Google Scholar 

  8. Brungart, D.S., Simpson, B.D.: Effects of temporal fine structure on the localization of broadband sounds: potential implications for the design of spatial audio displays. In: Int Conf on Aud Disp. Paris, France (June 24–27 2008)

    Google Scholar 

  9. Brungart, D.S., Simpson, B.D., McKinley, R.L., Kordik, A.J., Dallman, R.C., Ovenshire, D.A.: The interaction between head-tracker latency, source duration, and response time in the localization of virtual sound sources. In: Int Conf on Aud Disp. Sydney, Australia (July 6–9 2004)

    Google Scholar 

  10. Carlile, S., Best, V.: Discrimination of sound source velocity in human listeners. J of the Acoustical Soc of America 111(2), 1026 (February 2002)

    Article  Google Scholar 

  11. Cohen, M.: Augmented Audio Reality: Design for a Spatial Sound GPS PGS. In: Center on Disabilities Virtual Reality Conf. California State University, Northridge (1994)

    Google Scholar 

  12. Cohen, M., Aoki, S., Koizumi, N.: Augmented audio reality: Telepresence/ar hybrid acoustic environments. In: IEEE Int Workshop on Robot and Human Communication. pp. 361–364 (1993)

    Google Scholar 

  13. Fitts, P.: The information capacity of the human motor system in controlling the amplitude of movement. J of Experimental Psychology 47(6), 381–91 (1954)

    Article  Google Scholar 

  14. Galloway, A., Ward, M.: Locative Media as Socialising and Spatialising Practice: Learning from Archaeology. Leonardo Electronic Almanac 14(3) (2006)

    Google Scholar 

  15. Gibson, J.J.: In Perceiving, Acting, and Knowing: Toward an Ecological Psychology, chap. The theory of affordances. Lawrence Erlbaum Associates, Hillsdale, NJ (1977)

    Google Scholar 

  16. Härmä, A.: Ambient human-to-human communication. Handbook of Ambient Intelligence and Smart Environments pp. 795–823 (2010)

    Google Scholar 

  17. Härmä, A., Jakka, J., Tikander, M., Karjalainen, M., Lokki, T., Nironen, H., Vesa, S.: Techniques and applications of wearable augmented reality audio. In: 114th Conv Audio Eng Soc. Amsterdam, The Netherlands (March 22–25 2003)

    Google Scholar 

  18. Helyer, N.: Sonic landscapes (22/8/2006 1999–2001), http://www.sonicobjects.com/index.php/sonicobjects/more/sonic_landscapes/

  19. Holland, S., Morse, D.R., Gedenryd, H.: AudioGPS: spatial audio in a minimal attention interface. Personal and Ubiquitous Computing 6(4), 253–259 (2002)

    Article  Google Scholar 

  20. Jeffress, L.A., Taylor, R.W.: Lateralization vs Localization. J of the Acoustical Soc of America 32, 936 (1960)

    Article  Google Scholar 

  21. Jin, C., Corderoy, A., Carlile, S., van Schaik, A.: Contrasting monaural and interaural spectral cues for human sound localization. J of the Acoustical Soc of America 115, 3124 (2004)

    Article  Google Scholar 

  22. Jones, K.S.: What is an affordance? Ecological Psychology 15(2), 107–114 (2003)

    Article  Google Scholar 

  23. Jones, M., Bradley, G., Jones, S., Holmes, G.: Navigation-by-Music for Pedestrians: an Initial Prototype and Evaluation. Proc of the Int Symp on Intelligent Environments: Improving the quality of life in a changing world (April 5–7 2006)

    Google Scholar 

  24. Kinayoglu, G.: Using audio-augmented reality to assess the role of soundscape in environmental perception. In: eCAADe27. Istanbul (2009)

    Google Scholar 

  25. Kyriakakis, C.: Fundamental and technological limitations of immersive audio systems. Proc of the IEEE 86(5), 941–951 (1998)

    Article  Google Scholar 

  26. Lindeman, R.W., Noma, H.: A classification scheme for multi-sensory augmented reality. In: Proc 2007 ACM symposium on Virtual reality software and technology. pp. 175–178. ACM (2007)

    Google Scholar 

  27. Livingston, M.A.: Evaluating human factors in augmented reality systems. Computer Graphics and Applications, IEEE 25(6), 6–9 (November/December 2005)

    Google Scholar 

  28. Lokki, T., Grohn, M., Savioja, L., Takala, T.: A case study of auditory navigation in virtual acoustic environments. In: Int Conf on Aud Disp (2000)

    Google Scholar 

  29. Loomis, J.M., Golledge, R.G., Klatzky, R.L.: Personal Guidance System for the Visually Impaired using GPS, GIS, and VR Technologies. In: VR Conf. California State University, Northridge (1993)

    Google Scholar 

  30. Loomis, J.: Digital map and navigation system for the visually impaired (July 25 1985), http://www.geog.ucsb.edu/pgs/papers/loomis_1985.pdf

  31. Loomis, J., Hebert, C., Cicinelli, J.: Active localization of virtual sounds. J of the Acoustical Soc of America 88(4), 1757–1764 (October 1990)

    Article  Google Scholar 

  32. Lyons, K., Gandy, M., Starner, T.: Guided by voices: An audio augmented reality system (2000)

    Google Scholar 

  33. Mallem, M.: Augmented reality: Issues, trends and challenges. In: 2nd Int Conf on Image Processing Theory Tools and Applications (IPTA). p. 8 (July 2010)

    Google Scholar 

  34. Mantell, J., Rod, J., Kage, Y., Delmotte, F., Leu, J.: Navinko: Audio augmented reality-enabled social navigation for city cyclists. In: Pervasive (2010)

    Google Scholar 

  35. Mariette, N.: CMMR/ICAD 2009, chap. Navigation Performance Effects of Render Method and Head-Turn Latency in Mobile Audio Augmented Reality, pp. 239–265. Lecture Notes in Computer Science 5954, Springer, Berlin/Heidelberg (2010)

    Google Scholar 

  36. Mariette, N.: Perceptual Evaluation of Personal, Location-Aware Spatial Audio. Ph.D. thesis, University of New South Wales, Sydney, Australia (March 2011)

    Google Scholar 

  37. Mariette, N.: From backpack to handheld: The recent trajectory of personal location aware spatial audio. In: 7th Int Digital Arts and Culture Conf. Perth, Australia (15th–18th September 2007)

    Google Scholar 

  38. Mariette, N.: Mitigation of binaural front–back confusions by body motion in audio augmented reality. In: 13th Int Conf on Aud Disp. Montréal, Canada (June 26–29 2007)

    Google Scholar 

  39. Martens, W.L.: Perceptual evaluation of filters controlling source direction: Customized and generalized HRTFs for binaural synthesis. Acoustical Science and Technology 24(5), 220–232 (2003)

    Article  Google Scholar 

  40. Martin, A., Jin, C., Schaik, A.: Psychoacoustic evaluation of systems for delivering spatialized augmented-reality audio. J of the Audio Eng Soc 57(12), 1016–1027 (December 2009)

    Google Scholar 

  41. Moustakas, N., Floros, A., Grigoriou, N.: Interactive audio realities: An augmented / mixed reality audio game prototype. In: 130th Conv Audio Eng Soc (May 13–16 2011)

    Google Scholar 

  42. Mynatt, E.D., Back, M., Want, R., Baer, M., Ellis, J.B.: Designing audio aura. In: Proc SIGCHI Conf on Human factors in Comp Sys. pp. 566–573. ACM Press/Addison-Wesley Publishing Co. (1998)

    Google Scholar 

  43. Novo, P.: Communication Acoustics, chap. Auditory Virtual Environments, pp. 277–297. Springer, Berlin–Heidelberg–New York (2005)

    Google Scholar 

  44. Pernaux, J.M., Emerit, M., Nicol, R.: Perceptual evaluation of binaural sound synthesis: the problem of reporting localization judgments. In: 114th Conv Audio Eng Soc (March 22–25 2003)

    Google Scholar 

  45. Philbeck, J.W., Mershon, D.H.: Knowledge about typical source output influences perceived auditory distance. J of the Acoustical Soc of America 111, 1980–1983 (2002)

    Article  Google Scholar 

  46. Rozier, J., Karahalios, K., Donath, J.: Hear&There: An Augmented Reality System of Linked Audio. In: Int Conf on Aud Disp. Atlanta, Georgia (April 2000)

    Google Scholar 

  47. Rutherford, P., Withington, D.: The application of virtual acoustic techniques for the development of an auditory navigation beacon used in building emergency egress. In: Proc 2001 Int Conf on Aud Disp. Espoo, Finland (July 29–August 1 2001)

    Google Scholar 

  48. Sawhney, N., Schmandt, C.: Design of Spatialized Audio in Nomadic Environments. In: Proc Int Conf on Aud Disp (1997)

    Google Scholar 

  49. Speigle, J.M., Loomis, J.M.: Auditory distance perception by translating observers. In: Proc, IEEE Symp on Research Frontiers in Virtual Reality. pp. 92–99 (1993)

    Google Scholar 

  50. Tikander, M.: Usability issues in listening to natural sounds with an augmented reality audio headset. J Audio Eng Soc 57(6), 430–441 (2009)

    Google Scholar 

  51. Walker, B.N., Lindsay, J.: Navigation performance in a virtual environment with bonephones. In: Proc 11th Int Conf on Aud Disp. pp. 260–263. Limerick, Ireland (2005)

    Google Scholar 

  52. Walker, B., Lindsay, J.: Navigation performance with a virtual auditory display: Effects of beacon sound, capture radius, and practice. Human Factors 48(2), 265 (2006)

    Article  Google Scholar 

  53. Warusfel, O., Eckel, G.: LISTEN – augmenting everyday environments through interactive soundscapes. In: IEEE Virtual Reality (2004)

    Google Scholar 

  54. Wenzel, E.M.: Analysis of the role of update rate and system latency in interactive virtual acoustic environments. In: 103rd Conv Audio Eng Soc. New York, NY, USA (September 26–29 1997)

    Google Scholar 

  55. Wightman, F.L., Kistler, D.J.: Resolution of front-back ambiguity in spatial hearing by listener and source movement. J of the Acoustical Soc of America 105(5), 13 (1999)

    Google Scholar 

  56. Williams, M., Jones, O., Fleuriot, C., Wood, L.: Children and emerging wireless technologies: investigating the potential for spatial practice. In: Proc SIGCHI Conf on Human factors in Comp Sys, April. pp. 02–07 (2005)

    Google Scholar 

  57. Woo, D., Mariette, N., Helyer, N., Rizos, C.: Syren – a ship based location-aware audio experience. J of global positioning systems, Int Assoc of Chinese Professionals in GPS 4(1–2), 41–47 (2005), alberta, Canada

    Google Scholar 

  58. Woo, D., Mariette, N., Salter, J., Rizos, C., Helyer, N.: Audio Nomad. In: ION GNSS. Fort Worth, Texas (September 26–29 2006)

    Google Scholar 

  59. Zhou, Z., Cheok, A.D., Qiu, Y., Yang, X.: The role of 3-D sound in human reaction and performance in augmented reality environments. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 37(2), 262–272 (March 2007)

    Article  Google Scholar 

  60. Zwiers, M., Van Opstal, A., Paige, G.: Plasticity in human sound localization induced by compressed spatial vision. Nature Neuroscience 6(2), 175–181 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of New South Wales, Sydney, NSW, Australia

    Nicholas Mariette

Authors
  1. Nicholas Mariette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicholas Mariette .

Editor information

Editors and Affiliations

  1. CSIRO ICT Centre, Pembroke Road, Marsfield, 1710, New South Wales, Australia

    Weidong Huang

  2. CSIRO ICT Centre, Pembroke Road, Marsfield, 1710, New South Wales, Australia

    Leila Alem

  3. Mail Code 5580, Washington, 20375, District of Columbia, USA

    Mark A. Livingston

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Mariette, N. (2013). Human Factors Research in Audio Augmented Reality. In: Huang, W., Alem, L., Livingston, M. (eds) Human Factors in Augmented Reality Environments. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4205-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4205-9_2

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4204-2

  • Online ISBN: 978-1-4614-4205-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation