Abstract
By convolving an audio stream with a given pair of impulse responses between a source position and the two ears, virtual sound scenes can be created over headphones. Typically, the set of these filters for an ensemble of spatial positions, termed the Head-Related Impulse Response (HRIR) is used to render position information of a sound object to a listener. However, HRIRs are measured in free-field conditions, ignoring room reflections. In the real world, multiple reflections and reverberation exist, producing complex rich sound spaces. Including room reflections and reverberation with the HRIR results in a binaural room impulse response (BRIR). The length of a given BRIR depend on the shape and volume of the room, with BRIRs having typical duration of several seconds, resulting in computationally long processing. When the virtual environment is updated in response to head/body movement, BRIRs need to be updated according to the relative direction of a sound object within the perceptual detection threshold of system latency. This poses complications for mobile devices where processing power is limited, such as the case of augmented reality. In this paper, the architecture of a new signal processing method by distributed computers is proposed for convolution of BRIRs applicable to such conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Assenmacher, I., Kuhlen, T., Lentz, T., Vorlaender, M.: Integrating real-time binaural acoustics into VR applications. In: Eurographics Conference on Virtual Environments, pp. 129–136 (2004)
Beranek, L.: Concert and Opera Halls: Music, Acoustics, and Architecture, 2nd edn. Springer, New York (2004). https://doi.org/10.1007/978-0-387-21636-2
Berkhout, A.J., de Vries, D., Vogel, P.: Acoustic control by wave field synthesis. J. Acoust. Soc. Am. 93(5), 2764–2778 (1993)
Blauert, J.: Spatial Hearing. MIT Press, Cambridge (1997)
Funkhouser, T., Min, P., Carlbom, I.: Real-time acoustic modeling for distributed virtual environments. In: Conference on Computer Graphics and Interactive Techniques. SIGGRAPH, pp. 365–374 (1999). https://doi.org/10.1145/311535.311590
Hendrickx, E., Stitt, P., Messonnier, J.C., Lyzwa, J.M., Katz, B.F., de Boishraud, C.: Improvement of externalization by listener and source movement, using a “binauralized" microphone array. J. Audio Eng. Soc., pp. 589–599 (2017). https://doi.org/10.17743/jaes.2017.0018
Hendrickx, E., Stitt, P., Messonnier, J.C., Lyzwa, J.M., Katz, B.F., de Boishraud, C.: Influence of head tracking on the externalization of speech stimuli for non-individualized binaural synthesis. J. Acoust. Soc. Am. 141(3), 2011–2023 (2017). https://doi.org/10.1121/1.4978612
Iiboshi, H., Yanagida, M.: Double overlap processing for convolving a long impulse response with an infinite length signal. In: Proceedings of FIT, pp. 11–12 (2004). (in Japanese)
Ise, S.: A principle of sound field control based on the Kirchhoff-Helmholtz integral equation and the theory of inverse systems. Acta Acust. United Acust. 85(1), 78–87 (1999)
Iwaya, Y., Otani, M., Tsuchiya, T., Li, J.: Virtual auditory display on a smartphone for high-resolution acoustic space by remote rendering. In: International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP), pp. 368–371 (2015)
Iwaya, Y., Suzuki, Y.: Sound localization and virtual auditory display. J. IEICE 89(1), 1091–1095 (2006). (in Japanese)
Katz, B.F., et al.: NAVIG: augmented reality guidance system for the visually impaired. Virtual Real. 16(4), 253–269 (2012). https://doi.org/10.1007/s10055-012-0213-6
Lopez, P.G., et al.: Edge-centric computing: vision and challenges. ACM SIGCOMM Comput. Commun. Rev. 45(5), 37–42 (2015)
Mariette, N., Katz, B., Boussetta, K., Guillerminet, O.: SoundDelta: a study of audio augmented reality using WiFi-distributed Ambisonic cell rendering. In: Audio Engineering Society Convention 128, London, pp. 1–15 (2010). http://www.aes.org/e-lib/browse.cfm?elib=15420
Poletti, M.: Three-dimensional surround sound systems based on spherical harmonics. J. Audio Eng. Soc. 23, 1004–1025 (2005)
Proakis, J.G., Manolakis, D.K.: Digital Signal Processing, 3rd edn. Prentice-Hall, Englewood Cliffs (1996)
Schissler, C., Manocha, D.: Interactive sound rendering on mobile devices using ray-parameterized reverberation filters. CoRR abs/1803.00430 (2018). http://arxiv.org/abs/1803.00430
Stitt, P., Hendrickx, E., Messonnier, J.C., Katz, B.F.G.: The influence of head tracking latency on binaural rendering in simple and complex sound scenes. In: Audio Engineering Society Convention 140, Paris, pp. 9591:1–8, June 2016. http://www.aes.org/e-lib/browse.cfm?elib=18289
Warusfel, O., Eckel, G.: LISTEN - augmenting everyday environments through interactive soundscapes. In: Workshop VR for Public Consumption, IEEE VR, Chicago (2004). http://resumbrae.com/vr04/warusfel.pdf
Yairi, S., Iwaya, Y., Suzuki, Y.: Estimation of detection threshold of system latency of virtual auditory display. Appl. Acoust. 68(8), 851–863 (2007)
Acknowledgements
The authors would like to thank Mr. S. Ito, Mr. H. Nojiri, K. Saito, and Ms. J. Chiba for their contributions in implementation of the system. A part of this study was supported by JSPS Kakenhi (26280078) and by SCOPE from Ministry of Internal Affairs and Communications, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Iwaya, Y., Katz, B.F.G. (2018). Distributed Signal Processing Architecture for Real-Time Convolution of 3D Audio Rendering for Mobile Applications. In: Bourdot, P., Cobb, S., Interrante, V., kato, H., Stricker, D. (eds) Virtual Reality and Augmented Reality. EuroVR 2018. Lecture Notes in Computer Science(), vol 11162. Springer, Cham. https://doi.org/10.1007/978-3-030-01790-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-01790-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01789-7
Online ISBN: 978-3-030-01790-3
eBook Packages: Computer ScienceComputer Science (R0)