Serious Games: Customizing the Audio-Visual Interface
Serious games are gaining in popularity within a wide range of educational and training applications given their ability to engage and motivate learners in the educational process. Recent hardware and computational advancements are providing developers the opportunity to develop applications that employ a high level of fidelity (realism) and novel interaction techniques. However, despite these great advances in hardware and computational power, real-time high fidelity rendering of complex virtual environments (found in many serious games) across all modalities is still not feasible. Perceptual-based rendering exploits various aspects of the multi-modal perceptual system to reduce computational requirements without any resulting perceptual effects on the resulting scene. A series of human-based experiments demonstrated a potentially strong effect of sound on visual fidelity perception, and task performance. However, the resulting effects were subjective whereby the influence of sound was dependent on various individual factors including musical listening preferences. This suggests the importance of customizing (individualizing) a serious game’s virtual environment with respect to audio-visual fidelity, background sounds, etc. In this paper details regarding this series of audio-visual experiments will be provided followed by a description of current work that is examining the customization of a serious game’s virtual environment by each user through the use of a game-based calibration method.
KeywordsSerious games virtual simulation audio-visual interaction audio-visual fidelity calibration
Unable to display preview. Download preview PDF.
- 2.Farmer, E., von Rooij, J., Riemersma, J., Joma, P., Morall, J.: Handbook of simulator based training. Ashgate Publishing Limited, Surrey (1999)Google Scholar
- 5.Blascovich, J., Bailenson, J.: Infinite Reality. Harper Collins, New York (2011)Google Scholar
- 6.Mastoropoulou, G., Debattista, K., Chalmers, A., Troscianco, T.: The influence of sound effects on the perceived smoothness of rendered animations. In: Proceedings of the Symposium on Applied Perception in Graphics and Visualization 2005, La Coruna, Spain, August 26-28, pp. 9–15 (2005)Google Scholar
- 7.Larsson, P., Vstjll, D., Kleiner, M.: On the quality of experience: A multi-modal approach to perceptual ego-motion and sensed presence in virtual environments. In: Proceedings of the First International Speech Communications Association Tutorial and Research Workshop on Auditory Quality of Systems, Akademie Mont-Cenis, Germany (2003)Google Scholar
- 14.Rojas, D., Kapralos, B., Cristancho, S., Collins, K., Conati, C., Dubrowski, A.: The effect of background sound on visual fidelity perception. In: Proceedings of ACM Audio Mostly 2011 (Extended Abstracts), Coimbra, Portugal, September 7-9, pp. 1–7 (2011)Google Scholar
- 15.Rojas, D., Kapralos, B., Cristancho, S., Collins, K., Hogue, A., Conati, C., Dubrowski, A.: Developing effective serious games: The effect of background sound on visual fidelity perception with varying texture resolution. Studies in Health Technology and Informatics 173, 386–392 (2013)Google Scholar
- 16.Rojas, D., Kapralos, B., Hogue, A., Collins, K., Nacke, L., Cristancho, S., Conati, C., Dubrowski, A.: The effect of sound on visual fidelity perception in stereoscopic 3-d. IEEE Transactions on System Man and Cybernetics Part B 43(6), 1572–1583 (2013)Google Scholar
- 17.Rojas, D., Kapralos, B., Collins, K., Dubrowski, A.: The effect of contextual sound cues on visual fidelity perception. Studies in Health Technology and Informatics (to appear, 2014)Google Scholar
- 18.Cowan, B., Rojas, D., Kapralos, B., Collins, K., Dubrowski, A.: Spatial sound and its effect on visual quality perception and task performance within a virtual environment. In: Proceedings of the 21st International Congress on Acoustics, Montreal, Canada, June 2-7, pp. 1–7 (2013)Google Scholar
- 20.Kerr, D.S.: The refractor/phoropter an important tool in vision correction (December 7, 2010), http://dougkerr.net/pumpkin/articles/Refractor.pdf
- 21.Tawadrous, M., Hogue, A., Kapralos, B., Collins, K.: An interactive in-game approach to user adjustment of stereoscopic 3D settings. In: Proceedings of Stereoscopic Displays and Applications XXIV, San Francisco, CA, USA, February 3-7, pp. 1–7 (2007)Google Scholar