Audition

Sounds are part of our real-life experience and offer rich information about our environment. Sound perceived as coming from different spatial locations as we hear it in everyday life is also known as spatial sound. Spatial sound is a key aspect in producing realistic virtual environments. It has been shown that the combination of sound and graphics enhances the sense of presence [222].

Sound can play different roles in a Virtual Reality system.

Complementary information: sound allows to provide additional and richer information about the simulated environment. Slight echoes and reverberations in the surrounding environment give the brain cues about the direction and distance of objects as well as information about the size of the environment. For example, an office has fewer echoes than a cathedral. The use of sound contributes to enhance the user awareness of the simulated space, for example, sounds coming from parts of the environment beyond the field of view let the user know that “there is more than meets the eye.” Sound can convey simulated properties of the entities within the virtual environment, for example, surface features (different sounds produced when touching soft or rough objects), weight, and impact force.

Alternative feedback: sound feedback can enhance user interfaces (e.g. sounds can indicate the reception of user commands or confirm selection of objects). Sound can help people with visual impairments.

Alternative interaction modality: in the form of voice, sound can be an efficient communication channel. Although it is still not completely reliable, speech recognition has been recognized as a convenient input for humancomputer interaction in VR (see the VIEW system in Section 1.3). Speech synthesis can be used as an alternative way to provide information to the user within a virtual environment.