Abstract
Most sensory systems are able to inform people about the spatial structure of their environment, their place in that environment, and their movement through it. We discuss these various sources of sensory information by dividing them into three general categories: external (vision, audition, somatosensory), internal (vestibular, kinesthetic) and efferent (efference copy, attention). Research on the roles of these sensory systems in the creation of environmental knowledge has shown, with few exceptions, that information from a single sensory modality is often sufficient for acquiring at least rudimentary knowledge of one’s immediate environment and one’s movement through it. After briefly discussing the ways in which sources of sensory information commonly covary in everyday life, we examine the types and quality of sensory information available from contemporary virtual environments, including desktop, CAVE, and HMD-based systems. Because none of these computer mediated systems is yet able to present a perfectly full and veridical sensory experience to its user, it is important for researchers and VE developers to understand the circumstances, tasks, and goals for which different sensory information sources are most critical. We review research on these topics, as well as research on how the omission, limitation, or distortion of different information sources may affect the perception and behavior of users. Finally, we discuss situations in which various types of virtual environment systems may be more or less useful.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Desktop VEs are rarely constructed to match the physical viewing angle (the visual extent taken up by the monitor) and the virtual field of view (the amount of the VE that is visible). A user’s physical viewing angle can vary greatly depending on her distance from the display, and it has been shown both that users do not notice this discrepancy [56] and that a wider field of view is advantageous in many tasks (e.g., [5]).
References
Alais D, Burr D (2004) No direction-specific bimodal facilitation for audiovisual motion detection. Cogn Brain Res 19:185–194
Alfano PL, Michel GF (1990) Restricting the field of view: perceptual and performance effects. Percept Motor Skills 70:25–45
Alibali MW (2005) Gesture in spatial cognition: expressing, communicating and thinking about spatial information. Spatial Cogn Comput 5:307–331
Allen GL, Siegel AW, Rosinski R (1978) The role of perceptual context in structuring spatial knowledge. J Experim Psychol Human Learn Mem 4:617–630
Arsenault R, Ware C (2002) Frustum view angle, observer view angle and VE navigation. In: Vidal CA, Kimer C (eds) Proceedings of the V Simposio de Realidade Virtual, Brazilian Computer Society, pp 15–25
Bachmann E, Calusdian J, Hodgson E, Yun X, Zmuda M (2012) Going anywhere anywhere—creating a low cost portable immersive VE system. Manuscript submitted for publication
Bakker NH, Werkhoven PJ, Passenier PO (1999) The effects of proprioceptive and visual feedback on geographical orientation in virtual environments. Presence Teleoperators Virtual Environ 8:36–53
Becker W, Nasios G, Raab S, Jürgens R (2002) Fusion of vestibular and podokinesthetic information during self-turning towards instructed targets. Exp Brain Res 144:458–474
Berger DR, Schulte-Pelkum J, Bülthoff HH (2010) Simulating believable forward accelerations on a Steward motion platform. ACM Trans Appl Percept 7(1):1–27 (Article 5)
Besson P, Richiardi J, Bourdin C, Bringoux L, Mestre DR, Vercher J (2010) Bayesian networks and information theory for audio-visual perception modeling. Biol Cybern 103:213–226
Biocca F, Kim J, Choi Y (2000) Visual touch in virtual environments: an exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence Teleoperators Virtual Environ 10:247–265
Blakemore SJ (2003) Deluding the motor system. Conscious Cogn 12:647–655
Bruggeman H, Piuneu VS, Rieser JJ (2009) Biomechanical versus inertial information: stable individual differences in perception of self-rotation. J Exp Psychol Human Percept Perform 35:1472–1480
Butler JS, Smith ST, Campos JL, Bülthoff HH (2010) Bayesian integration of visual and vestibular signals for heading. J Vis 10:1–13
Campos JL, Byrne P, Sun HJ (2010) The brain weights body-based cues higher than vision when estimating walked distances. Eur J Neurosci 31:1889–1898
Chance SS, Gaunet F, Beall AC, Loomis JM (1998) Locomotion mode affects the updating of objects encountered during travel: the contribution of vestibular and proprioceptive inputs to path integration. Presence Teleoperators Virtual Environ 7:168–178
Cheng K, Shettleworth SJ, Huttenlocher J, Rieser JJ (2007) Bayesian integration of spatial information. Psychol Bull 133:625–637
Chrastil ER, Warren WH (2012) Active and passive contributions to spatial learning. Psychon Bull Rev 19:1–23
Christou CG, Bülthoff HH (1999) View dependence in scene recognition after active learning. Memory Cogn 27:996–1007
Cohen HS (2000) Vestibular disorders and impaired path integration along a linear trajectory. J Vestib Res 10:7–15
Couclelis H (1996) Verbal directions for way-finding: space, cognition, and language. In: Portugali J (ed) The construction of cognitive maps. Kluwer, Dordrecht, The Netherlands, pp 133–153
Cruz-Neira C, Reiners D, Springer JP (2010) An affordable surround-screen virtual reality display. Soc Inf Disp 18:836–843
Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC (1992) The cave: audio visual experience automatic virtual environment. Commun ACM 35:64–72
Cutting JE, Vishton PM (1995) Perceiving layout: the integration, relative dominance, and contextual use of different information about depth. In: Epstein W, Rogers S (eds) Handbook of perception and cognition: Vol. 5: perception of space and motion. Academic Press, New York
Darken RP, Cockayne WR, Carmein D (1997) The omni-directional treadmill: a locomotion device for virtual worlds. In: Proceedings of ACM symposium on user interface software and technology, pp 213–221
Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433
Fortenbaugh FC, Hicks JC, Turano KA (2008) The effect of peripheral visual field loss on representations of space: evidence for distortion and adaptation. Investig Ophthalmol Visual Sci 49:2765–2772
Frenz H, Bremmer F, Lappe M (2003) Discrimination of travel distance from ‘situated’ optic flow. Vis Res 43:2173–2183
Gallistel CR (1990) The organization of learning. MIT, Cambridge
Gibson JJ (1958) Visually controlled locomotion and visual orientation in animals. Br J Psychol 49:182–194
Gibson JJ (1979) The ecological approach to visual perception. Houghton Mifflin, Boston
Glasauer S, Amorim MA, Vitte E, Berthoz A (1994) Goal directed linear locomotion in normal and labyrinthine-defective subjects. Exp Brain Res 98:323–335
Goldin SE, Thorndyke PW (1982) Simulating navigation for spatial knowledge acquisition. Human Factors 24:457–471
Golledge R, Klatzky R, Loomis J (1996) Cognitive mapping and wayfinding by adults without vision. In: Portugali J (ed) The construction of cognitive maps. Kluwer, The Netherlands, pp 215–246
Grant SC, Magee LE (1998) Contributions of proprioception to navigation in virtual environments. Human Factors 40:489–497
Guidice NA, Bakdash JZ, Legge GE (2007) Wayfinding with words: spatial learning and navigation using dynamically updated verbal descriptions. Psychol Res 71:347–358
Harm DL (2002) Motion sickness neurophysiology, physiological correlates, and treatment. In: Stanney KM (ed) Handbook of virtual environments: design, implementation, and applications. Erlbaum, Mahwah, pp 637–661
Harris LR, Jenkin M, Zikovitz DC (2000) Visual and non-visual cues in the perception of linear self-motion. Exp Brain Res 135:12–21
Hay JC, Pick HL Jr (1965) Visual capture produced by prism spectacles. Psychon Sci 2:215–216
Heilig M (1992) El cine de futro: the cinema of the future. Presence Teleoperators Virtual Environ 1:279–294 (Original work published 1955)
Heft H (1996) The ecological approach to navigation: A Gibsonian perspective. In: Portugali J (ed) The construction of cognitive maps. Kluwer Academic Publishers, Dordrect, pp 105–132
Heft H (1997) The relevance of Gibson’s ecological approach for environment-behavior studies. In: Moore GT, Marans RW (eds) Advances in environment, behavior, and design, vol 4. Plenum, New York, pp 71–108
Hettinger LJ (2002) Illusory self-motion in virtual environments. In: Stanney KM (ed) Handbook of Virtual Environments. Erlbaum, New Jersey, pp 471–492
Hilsendeger A, Brandauer S, Tolksdorf J, Fröhlich C (2009) Navigation in virtual reality with the wii balanceboard\(^{TM}\). In: GI-Workshop Virtuelle und Erweiterte Realität, ACM
Hodgson E, Bachmann E, Waller D (2011) Redirected walking to explore virtual environments: assessing the potential for spatial interference. ACM Trans Appl Percept 8, Article 22
Hollerbach JM (2002) Locomotion interfaces. In: Stanney KM (ed) Handbook of virtual environments: design, implementation, and applications. Erlbaum, New Jersey, pp 239–254
von Holst E, Mittlestaedt H (1950) Das reafferenz princip: (Wedlselwirkungen zwischen Zentrainervensystem und Peripherie.) Die Naturwissenschften, 37, 464–476. Translated in: Dodwell PC, ed (1971) Perceptual processing: stimulus equivalence and pattern recognition (pp. 41–72). Appleton-Century-Crofts, New York
Huang J, Chiu W, Lin Y, Tsai M, Bai H, Tai C, Gau C, Lee H (2000) The gait sensing disc: a compact locomotion device for the virtual environment. In: Proceedings of the international conference in central Europe on computer graphics, visualization and interactive digital media. Pilsen, Czech Republic
Iwata H (2000) Locomotion interface for virtual environments. In: Hollerbach J, Koditshek D (eds) Robotics research: the ninth international symposium. Springer-Verlag, London, pp 275–282
Iwata H, Yano H, Fukushima H, Noma H (2005) CirculaFloor. IEEE Comput Graph Appl 25:64–67
Iwata I, Yano H, Nakaizumi F (2001) Gait master: a versatile locomotion interface for uneven virtual terrain. In: Proceedings of IEEE virtual reality conference, pp. 131–137
Jürgens R, Becker W (2006) Perception of angular displacement without landmarks: evidence for Bayesian fusion of vestibular, optokinetic, podokinesthetic, and cognitive information. Exp Brain Res 174:528–543
Jürgens R, Boß T, Becker W (1999) Estimation of self-turning during active and passive rotation in the dark. Exp Brain Res 128:491–504
Kearns MJ, Warren WH, Duchon AP, Tarr MJ (2002) Path integration from optic flow and body senses in a homing task. Perception 31:349–374
Klatzky RL, Loomis JM, Beall AC, Chance SS, Golledge RG (1998) Updating an egocentric spatial representation during real, imagined, and virtual locomotion. Psychol Sci 9:293–298
Kubovy M (1986) The psychology of linear perspective and renaissance art. Cambridge University Press, Cambridge
Lackner JP, DiZio P (2005) Vestibular, proprioceptive, and haptic contributions to spatial orientation. Ann Rev Psychol 56:115–147
Lappe M, Bremmer F, van den Berg AV (1999) Perception of self-motion from visual flow. Trends Cogn Sci 3:329–336
Larish JF, Flach JM (1990) Sources of optical information useful for perception of speed of rectilinear self-motion. J Exp Psychol Human Percept Perform 16:295–302
Lederman SJ, Klatzky RL (2009) Haptic perception: a tutorial. Atten Percept Psychophys 71:1439–1459
Lepecq J, Giannopulu I, Baudonnière P (1995) Cognitive effects on visually induced body motion in children. Perception 24:435–449
Loomis JM, Klatzky RL, Golledge RG, Philbeck JW (1999) Human navigation by path integration. In: Golledge RG (ed) Wayfinding: cognitive mapping and other spatial processes. Johns Hopkins, Baltimore, pp 125–151
Loomis JM, Klatzky RL, Avraamides M, Lippa Y, Golledge RG (2007) Functional equivalence of spatial images produced by perception and spatial language. In: Mast F, Jäncke L (eds) Spatial processing in navigation, imagery, and perception. Springer, New York, pp 29–48
May M, Klatzky RL (2000) Path integration while ignoring irrelevant movement. J Exp Psychol Learn Memory Cogn 26:169–186
Medina E, Fruland R, Weghorst S (2008) Virtusphere: walking in a human size VR “hamster ball”. In: Proceedings of the human factors and ergonomics society, pp 2102–2106
Mittelstaedt H (1996) Somatic graviception. Biol Psychol 42:53–74
Mittelstaedt ML, Mittelstaedt H (2001) Idiothetic navigation in humans: estimation of path length. Exp Brain Res 139:318–332
Nardini M, Jones P, Bedford R, Braddick O (2008) Development of cue integration in human navigation. Curr Biol 18:689–693
Neider MB, Gaspar JG, McCarley JS, Crowell J, Kaczmarski H, Kramer AF (2011) Walking and talking: dual-task effects on street crossing behavior in older adults. Psychology and Aging (Advance online publication)
Nielsen TI (1963) Volition: a new experimental approach. Scand J Psychol 4:225–230
Nitzshe N, Hanebeck UD, Schmidt G (2004) Motion compression for telepresent walking in large target environments. Presence Teleoperators Virtual Environ 13:44–60
Noë A (2004) Action in perception. MIT Press, Boston
Ohmi M (1996) Egocentric perception through interaction among many sensory systems. Cogn Brain Res 5:87–96
Peck TC, Whitton MC, Fuchs H (2009) Evaluation of reorientation techniques for walking in large virtual environments. IEEE Trans Vis Comput Graph 15:121–127
Péruch P, Borel L, Magnan J, Lacour M (2005) Direction and distance deficits in path integration after unilateral vestibular loss depends on task complexity. Cogn Brain Res 25:862–872
Péruch P, May M, Wartenberg F (1997) Homing in virtual environments: effects of field of view and path layout. Perception 26:301–311
Potegal M (1982) Vestibular and neostriatal contributions to spatial orientation. In: Potegal M (ed) Spatial abilities development and physiological foundations. Academic Press, New York, pp 361–387
Pratt DR, Barham PT, Locke J, Zyda MJ, Eastman B, Moore T et al (1994) Insertion of an articulated human into a networked virtual environment. In: Proceedings of the 1994 AI, simulation, and planning in high autonomy systems conference. University of Florida, Gainesville
Prinz W (1997) Perception and action planning. Eur J Cogn Psychol 9:129–154
Proffitt DR (2006) Distance perception. Curr Dir Psychol Sci 15:131–135
Razzaque S (2005) Redirected walking. Doctoral dissertation, University of North Carolina, Chapel Hill
Razzaque S, Swapp D, Slater M, Whitton MC, Steed A (2002) Redirected walking in place. In: Muller S, Stuzlinger W (eds) Proceedings of the eurographics workshop on virtual environments. Eurographics Association, pp 123–130
Riecke BE, Cunningham DW, Bülthoff HH (2007) Spatial updating in virtual reality: the sufficiency of visual information. Psychol Res 71:298–313
Riecke B, Feuereissen D, Rieser JJ (2009) Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion. ACM Trans Appl Percept 6, Article 20
Riecke BE, van Veen HAHC, Bülthoff HH (2002) Visual homing is possible without landmarks: a path integration study in virtual reality. Presence Teleoperators Virtual Environ 11:443–473
Rossano MJ, West SO, Robertson TJ, Wayne MC, Chase RB (1999) The acquisition of route and survey knowledge from computer models. J Environ Psychol 19:101–115
Ruddle RA (2001) Navigation: am I really lost or virtually there? In: Harris D (ed) Engineering psychology and cognitive ergonomics, vol 6. Ashgate, Burlington, pp 135–142
Ruddle RA, Lessels S (2009) The benefits of using a walking interface to navigate virtual environments. ACM Trans Comput Human Interact 16:1–18
Ruddle RA, Payne SJ, Jones DM (1997) Navigating buildings in “desk-top” virtual environments: experimental investigations using extended navigational experience. J Exp Psychol Appl 3:143–159
Sandvad J (1999) Auditory perception of reverberant surroundings. J Acoust Soc Am 105:1193
Schenkman BN, Nilsson ME (2010) Human echolocation: blind and sighted persons’ ability to detect sounds recorded in the presence of a reflected object. Perception 39:483–501
Shelton AL, McNamara TP (1997) Multiple views of spatial memory. Psychon Bull Rev 4:102–106
Shin YK, Proctor RW, Capaldi EJ (2010) A review of contemporary ideomotor theory. Psychol Bull 136:943–974
Sholl MJ (1989) The relation between horizontality and rod-and-frame and vestibular navigational performance. J Exp Psychol Learn Memory Cogn 15:110–125
Sholl MJ (1996) From visual information to cognitive maps. In: Portugali J (ed) The construction of cognitive maps. Kluwer, Dordrecht, pp 157–186
Souman JL, Giordano PR, Schwaiger M, Frissen I, Thümmel T, Ulbrich H, De Luca A, Bülthoff HH, Ernst MO (2011) CyberWalk: enabling unconstrained omnidirectional walking through virtual environments. ACM Trans Appl Percept 8, Article 25
Stanton DEB, Wilson PN, Foreman N (2003) Human shortcut performance in a computer-simulated maze: a comparative study. Spatial Cogn Comput 3:315–329
Stoffregen TA, Pittenger JB (1995) Human echolocation as a basic form of perception and action. Ecol Psychol 7:181–216
Stratton G (1896) Some preliminary experiments on vision without inversion of the retinal image. Psychol Rev 3:341–360
Steinicke F, Bruder G, Jerald J, Frenz H, Lappe M (2009) Estimation of detection thresholds for redirected walking techniques. IEEE Trans Vis Comput Graph 16(1):17–27
St. George RJ, Fitzpatrick RC (2011) The sense of self-motion, orientation and balance explored by vestibular stimulation. J Physiol 589:807–813
Sun HJ, Campos JL, Chan GSW (2004) Multisensory integration in the estimation of relative path length. Exp Brain Res 154:246–254
Telford L, Howard IP, Ohmi M (1995) Heading judgments during active and passive self-motion. Exp Brain Res 104:502–510
Templeman JN, Denbrook PS, Sibert LE (1999) Virtual locomotion: walking in place through virtual environments. Presence Teleoperators Virtual Environ 8:598–619
Thorndyke PW, Hayes-Roth B (1982) Differences in spatial knowledge acquired from maps and navigation. Cogn Psychol 14:560–589
Toet A, Jansen SEM, Delleman NJ (2007) Effects of field-of-view restrictions on speed and accuracy of manoeuvring. Percept Motor Skills 105:1245–1256
Turvey MT, Carello C (1986) The ecological approach to perceiving-acting: a pictorial essay. Acta Psychol 63:133–155
Van Erp JBF, Van Veen HAHC, Jansen C, Dobbins T (2005) Waypoint navigation with a vibrotactile waist belt. ACM Trans Appl Percept 2:106–117
Waller D, Bachmann E, Hodgson E, Beall AC (2007) The HIVE: A Huge Immersive Virtual Environment for research in spatial cognition. Behav Res Methods 39:835–843
Waller D, Greenauer N (2007) The role of body-based sensory information in the acquisition of enduring spatial representations. Psychol Res 71:322–332
Waller D, Hunt E, Knapp D (1998) The transfer of spatial knowledge in virtual environment training. Presence Teleoperators Virtual Environ 7:129–143
Waller D, Loomis JM, Haun DBM (2004) Body-based senses enhance knowledge of directions in large-scale environments. Psychon Bull Rev 11:157–163
Waller D, Loomis JM, Steck S (2003) Inertial cues do not enhance knowledge of environmental layout. Psychon Bull Rev 10:987–993
Witmer BG, Bailey JH, Knerr BW, Parsons KC (1996) Virtual spaces and real world places: transfer of route knowledge. Int J Human Comput Stud 45:413–428
Yardley L (1992) Motion sickness and perception: a reappraisal of the sensory conflict approach. Br J Psychol 83:449–471
Yardley L, Higgins M (1998) Spatial updating during rotation: the role of vestibular information and mental activity. J Vestib Res 8:435–442
Yost WA (2001) Auditory localization and scene perception. In: Goldstein EB (ed) Blackwell handbook of perception. Blackwell Publishers, Malden, pp 437–468
Zhang H, Mou W, McNamara TP (2011) Spatial updating according to the intrinsic reference direction of a briefly viewed layout. Cognition 119:419–429
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Waller, D., Hodgson, E. (2013). Sensory Contributions to Spatial Knowledge of Real and Virtual Environments. In: Steinicke, F., Visell, Y., Campos, J., Lécuyer, A. (eds) Human Walking in Virtual Environments. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8432-6_1
Download citation
DOI: https://doi.org/10.1007/978-1-4419-8432-6_1
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-8431-9
Online ISBN: 978-1-4419-8432-6
eBook Packages: EngineeringEngineering (R0)