Skip to main content
SpringerLink
Log in

Cross-Modal Modulation of Spatial Attention by Emotion

  • Chapter
  • First Online:
Book cover Integrating Face and Voice in Person Perception

Abstract

A huge amount of environmental stimulus input constantly enters the brain via the different sensory channels of the organism. Due to its limited capacity, the brain cannot process all the inputs exhaustively, and thus needs to select a subset of stimuli for further processing at the cost of others. Emotional stimuli, for example social signals such as angry faces or happy voices, are privileged in the competition for attentional processing resources: the neural representation of emotional stimuli is stronger and more robust compared to neutral stimuli; emotional stimuli are prioritized in perception, draw attention more quickly, and impede attentional disengagement longer than neutral stimuli. The representations of emotional stimuli are thus intensified at different stages of processing. This generates a vivid conscious percept allowing organisms to prepare and implement adequate responses. By modulating frontoparietal attention systems, emotional stimuli also impact on the perceptual processing of subsequent stimuli appearing at the same location as an emotional stimulus. Such neurocognitive selection mechanisms thus drastically reorganize our representation and perception of the environment by focusing on emotionally and motivationally relevant events and their immediate spatial and temporal periphery.

Until now, the effects of emotional stimuli on attentional processes have mainly been described within a sensory modality, most frequently using pictures of emotional stimuli to modulate visuospatial attention. However, in real-life situations humans typically encounter simultaneous input to several different senses, such as vision, audition, olfaction, and touch. Signals entering these different channels might originate from a common, emotionally relevant source. To receive maximal benefit from multimodal sensory input, the brain must coordinate the input appropriately so that signals from a relevant common source are rapidly processed and integrated across the different input channels to allow for the preparation and implementation of adaptive responses.

We review the current evidence for cross-modal modulation of spatial attention by emotional information. Presenting data from behavioral and electrophysiological investigations in human subjects we illustrate, for example, the effects of emotional voices on visual attention and the effects of emotional images on haptic attention. The data converge to show that emotion modulates attentional processing across sensory modalities by boosting early sensory stages of processing, potentially implemented by a large-scale neural network centered around the amygdala, providing direct and indirect top-down signals to sensory pathways and frontoparietal pathways involved in exogenous and endogenous attentional selection processes.

This rapid cross-modal integration at multiple stages of processing may reflect a fundamental principle of human brain organization: to prioritize the processing of emotionally relevant stimuli, even if they are outside the focus of spatial attention, thus facilitating the multimodal assessment of emotionally relevant stimuli in the environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Amaral, D. G., Behniea, H., & Kelly, J. L. (2003). Topographic organization of projections from the amygdala to the visual cortex in the macaque monkey. Neuroscience, 118, 1099–1120.

    Article  PubMed  CAS  Google Scholar 

  • Andersen, R. A., Snyder, L. H., Bradley, D. C., & Xing, J. (1997). Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annual Review of Neuroscience, 20, 303–330.

    Article  PubMed  CAS  Google Scholar 

  • Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411, 305–309.

    Article  PubMed  CAS  Google Scholar 

  • Bertels, J., Kolinsky, R., & Morais, J. (2010). Emotional valence of spoken words influences the spatial orienting of attention. Acta Psychologica, 134, 264–278.

    Article  PubMed  Google Scholar 

  • Bishop, S. J., Duncan, J., Brett, M., & Lawrence, A. D. (2004). Prefrontal cortical function and anxiety: Controlling attention to threat-related stimuli. Nature Neuroscience, 7, 184–188.

    Article  PubMed  CAS  Google Scholar 

  • Brosch, T., Grandjean, D., Sander, D., & Scherer, K. R. (2008). Behold the voice of wrath: Cross-modal modulation of visual attention by anger prosody. Cognition, 106, 1497–1503.

    Article  PubMed  Google Scholar 

  • Brosch, T., Grandjean, D., Sander, D., & Scherer, K. R. (2009). Cross-modal emotional attention: Emotional voices modulate early stages of visual processing. Journal of Cognitive Neuroscience, 21, 1670–1679.

    Article  PubMed  Google Scholar 

  • Brosch, T., Sander, D., Pourtois, G., & Scherer, K. R. (2008). Beyond fear: Rapid spatial orienting towards positive emotional stimuli. Psychological Science, 19, 362–370.

    Article  PubMed  Google Scholar 

  • Brosch, T., Sander, D., & Scherer, K. R. (2007). That baby caught my eye… Attention capture by infant faces. Emotion, 7, 685–689.

    Article  PubMed  Google Scholar 

  • Brosch, T., Pourtois, G., Sander, D., & Vuilleumier, P. (2011). Additive effects of emotional, endogenous, and exogenous attention: behavioral and electrophysiological evidence. Neuropsychologia, 49(7), 1779–1787.

    Article  PubMed  Google Scholar 

  • Campanella, S., & Belin, P. (2007). Integrating face and voice in person perception. Trends in Cognitive Sciences, 11, 535–543.

    Article  PubMed  Google Scholar 

  • Carrasco, M., Ling, S., & Read, S. (2004). Attention alters appearance. Nature Neuroscience, 7, 308–313.

    Article  PubMed  CAS  Google Scholar 

  • Cerf, M., Thiruvengadam, N., Mormann, F., Kraskov, A., Quiroga, R. Q., Koch, C., et al. (2010). On-line, voluntary control of human temporal lobe neurons. Nature, 467, 1104–1108.

    Article  PubMed  CAS  Google Scholar 

  • Corbetta, M., Patel, G., & Shulman, G. L. (2008). The reorienting system of the human brain: From environment to theory of mind. Neuron, 58, 306–324.

    Article  PubMed  CAS  Google Scholar 

  • Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215.

    Article  PubMed  CAS  Google Scholar 

  • Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222.

    Article  PubMed  CAS  Google Scholar 

  • Dolan, R. J., Morris, J. S., & de Gelder, B. (2001). Crossmodal binding of fear in voice and face. Proceedings of the National Academy of Sciences of the United States of America, 98, 10006–10010.

    Article  PubMed  CAS  Google Scholar 

  • Dowman, R. (2007). Neural mechanisms of detecting and orienting attention toward unattended threatening somatosensory targets. I. Intermodal effects. Psychophysiology, 44, 407–419.

    Article  PubMed  Google Scholar 

  • Dowman, R., & Ben-Avraham, D. (2008). An artificial neural network model of orienting attention toward threatening somatosensory stimuli. Psychophysiology, 45, 229–239.

    Article  PubMed  Google Scholar 

  • Driver, J. (2001). A selective review of selective attention research from the past century. British Journal of Psychology, 92, 53–78.

    Article  CAS  Google Scholar 

  • Driver, J., & Spence, C. (1998). Crossmodal attention. Current Opinion in Neurobiology, 8, 245–253.

    Article  PubMed  CAS  Google Scholar 

  • Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48, 269–297.

    Article  PubMed  CAS  Google Scholar 

  • Eimer, M., Cockburn, D., Smedley, B., & Driver, J. (2001). Cross-modal links in endogenous spatial attention are mediated by common external locations: Evidence from event-related brain potentials. Experimental Brain Research, 139, 398–411.

    Article  CAS  Google Scholar 

  • Eimer, M., & Driver, J. (2001). Crossmodal links in endogenous and exogenous spatial attention: Evidence from event-related brain potential studies. Neuroscience and Biobehavioral Reviews, 25, 497–511.

    Article  PubMed  CAS  Google Scholar 

  • Eimer, M., & Holmes, A. (2007). Event-related brain potential correlates of emotional face processing. Neuropsychologia, 45, 15–31.

    Article  PubMed  Google Scholar 

  • Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L., Saur, R., et al. (2006). Impact of voice on emotional judgment of faces: An event-related fMRI study. Human Brain Mapping, 27, 707–714.

    Article  PubMed  Google Scholar 

  • Ethofer, T., Anders, S., Wiethoff, S., Erb, M., Herbert, C., Saur, R., et al. (2006). Effects of prosodic emotional intensity on activation of associative auditory cortex. Neuroreport, 17, 249–253.

    Article  PubMed  Google Scholar 

  • Grandjean, D., Bänziger, T., & Scherer, K. R. (2006). Intonation as an interface between language and affect. Progress in Brain Research, 156, 235–247.

    Article  PubMed  Google Scholar 

  • Grandjean, D., Sander, D., Lucas, N., Scherer, K. R., & Vuilleumier, P. (2008). Effects of emotional prosody on auditory extinction for voices in patients with spatial neglect. Neuropsychologia, 46, 487–496.

    Article  PubMed  Google Scholar 

  • Grandjean, D., Sander, D., Pourtois, G., Schwartz, S., Seghier, M. L., Scherer, K. R., et al. (2005). The voices of wrath: Brain responses to angry prosody in meaningless speech. Nature Neuroscience, 8, 145–146.

    Article  PubMed  CAS  Google Scholar 

  • Hein, G., & Knight, R. T. (2008). Superior temporal sulcus—It’s my area: Or is it? Journal of Cognitive Neuroscience, 20, 2125–2136.

    Article  PubMed  Google Scholar 

  • Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society B: Biological Sciences, 353, 1257–1270.

    Article  CAS  Google Scholar 

  • Keil, A., Bradley, M. M., Junghofer, M., Russmann, T., Lowenthal, W., & Lang, P. J. (2007). Cross-modal attention capture by affective stimuli: Evidence from event-related potentials. Cognitive Affective and Behavioral Neuroscience, 7, 18–24.

    Article  Google Scholar 

  • Kennett, S., Eimer, M., Spence, C., & Driver, J. (2001). Tactile-visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs. Journal of Cognitive Neuroscience, 13, 462–478.

    Article  PubMed  CAS  Google Scholar 

  • Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2010). Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134, 372–384.

    Article  PubMed  Google Scholar 

  • Kreifelts, B., Ethofer, T., Shiozawa, T., Grodd, W., & Wildgruber, D. (2009). Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus. Neuropsychologia, 47, 3059–3066.

    Article  PubMed  Google Scholar 

  • LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 23, 155–184.

    Article  PubMed  CAS  Google Scholar 

  • Lipp, O. V., & Derakshan, N. (2005). Attentional bias to pictures of fear-relevant animals in a dot probe task. Emotion, 5, 365–369.

    Article  PubMed  Google Scholar 

  • Lloyd, D. M., Bolanowski, S. J., Jr., Howard, L., & McGlone, F. (1999). Mechanisms of attention in touch. Somatosensory and Motor Research, 16, 3–10.

    Article  PubMed  CAS  Google Scholar 

  • MacLeod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal of Abnormal Psychology, 95, 15–20.

    Article  PubMed  CAS  Google Scholar 

  • Marois, R., & Ivanoff, J. (2005). Capacity limits of information processing in the brain. Trends in Cognitive Sciences, 9, 296–305.

    Article  PubMed  Google Scholar 

  • McDonald, J. J., Teder-Salejarvi, W. A., Di Russo, F., & Hillyard, S. A. (2003). Neural substrates of perceptual enhancement by cross-modal spatial attention. Journal of Cognitive Neuroscience, 15, 10–19.

    Article  PubMed  Google Scholar 

  • Miles, E., Poliakoff, E., & Brown, R. J. (2008). Investigating the time course of tactile reflexive attention using a non-spatial discrimination task. Acta Psychologica, 128, 210–215.

    Article  PubMed  Google Scholar 

  • Mohanty, A., Egner, T., Monti, J. M., & Mesulam, M. M. (2009). Search for a threatening target triggers limbic guidance of spatial attention. Journal of Neuroscience, 29, 10563–10572.

    Article  PubMed  CAS  Google Scholar 

  • Ohman, A., Flykt, A., & Esteves, F. (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology: General, 130, 466–478.

    Article  CAS  Google Scholar 

  • Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77, 247–265.

    Article  PubMed  Google Scholar 

  • Peelen, M., Atkinson, A., Andersson, F., & Vuilleumier, P. (2007). Emotional modulation of body-selective visual areas. Social Cognitive and Affective Neuroscience, 2, 274–283.

    Article  PubMed  Google Scholar 

  • Peelen, M., Heslenfeld, D. J., & Theeuwes, J. (2004). Endogenous and exogenous attention shifts are mediated by the same large-scale neural network. Neuroimage, 22, 822–830.

    Article  PubMed  Google Scholar 

  • Penfield, W., & Rasmussen, T. (1950). The cerebral cortex of man. New York, NY: Macmillan.

    Google Scholar 

  • Phelps, E. A. (2006). Emotion and cognition: Insights from studies of the human amygdala. Annual Review of Psychology, 57, 27–53.

    Article  PubMed  Google Scholar 

  • Phelps, E. A., Ling, S., & Carrasco, M. (2006). Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychological Science, 17, 292–299.

    Article  PubMed  Google Scholar 

  • Poliakoff, E., Miles, E., Li, X., & Blanchette, I. (2007). The effect of visual threat on spatial attention to touch. Cognition, 102, 405–414.

    Article  PubMed  Google Scholar 

  • Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3–25.

    Article  PubMed  CAS  Google Scholar 

  • Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980). Attention and the detection of signals. Journal of Experimental Psychology, 109, 160–174.

    PubMed  CAS  Google Scholar 

  • Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14, 619–633.

    Article  PubMed  Google Scholar 

  • Pourtois, G., Schwartz, S., Seghier, M. L., Lazeyras, F., & Vuilleumier, P. (2006). Neural systems for orienting attention to the location of threat signals: An event-related fMRI study. Neuroimage, 31, 920–933.

    Article  PubMed  Google Scholar 

  • Pourtois, G., Thut, G., Grave de Peralta, R., Michel, C., & Vuilleumier, P. (2005). Two electrophysiological stages of spatial orienting towards fearful faces: Early temporo-parietal activation preceding gain control in extrastriate visual cortex. Neuroimage, 26, 149–163.

    Article  PubMed  Google Scholar 

  • Rinck, M., Reinecke, A., Ellwart, T., Heuer, K., & Becker, E. S. (2005). Speeded detection and increased distraction in fear of spiders: Evidence from eye movements. Journal of Abnormal Psychology, 114, 235–248.

    Article  PubMed  Google Scholar 

  • Rossi, A. F., & Paradiso, M. A. (1995). Feature-specific effects of selective visual attention. Vision Research, 35, 621–634.

    Article  PubMed  CAS  Google Scholar 

  • Sander, D., Grafman, J., & Zalla, T. (2003). The human amygdala: An evolved system for relevance detection. Reviews in the Neurosciences, 14, 303–316.

    Article  PubMed  Google Scholar 

  • Sander, D., Grandjean, D., Pourtois, G., Schwartz, S., Seghier, M. L., Scherer, K. R., et al. (2005). Emotion and attention interactions in social cognition: Brain regions involved in processing anger prosody. Neuroimage, 28, 848–858.

    Article  PubMed  Google Scholar 

  • Sander, K., & Scheich, H. (2001). Auditory perception of laughing and crying activates human amygdala regardless of attentional state. Cognitive Brain Research, 12, 181–198.

    Article  PubMed  CAS  Google Scholar 

  • Scherer, K. R. (2001). Appraisal considered as a process of multilevel sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 92–120). New York, NY: Oxford University Press.

    Google Scholar 

  • Schirmer, A., Teh, K. S., Wang, S., Vijayakumar, R., Ching, A., Nithianantham, D., Escoffier, N., Cheok, A.D. (2011). Squeeze me, but don’t tease me: human and mechanical touch enhance visual attention and emotion discrimination. Social Neuroscience, 6(3), 219–230.

    Google Scholar 

  • Smith, S. D., Most, S. B., Newsome, L. A., & Zald, D. H. (2006). An emotion-induced attentional blink elicited by aversively conditioned stimuli. Emotion, 6, 523–527.

    Article  PubMed  Google Scholar 

  • Spence, C. (2010). Crossmodal spatial attention. Annals of the New York Academy of Science, 1191, 182–200.

    Article  Google Scholar 

  • Spence, C., & Driver, J. (1994). Covert spatial orienting in audition: Exogenous and endogenous mechanisms facilitate sound localization. Journal of Experimental Psychology-Human Perception and Performance, 20, 555–574.

    Article  Google Scholar 

  • Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9, 255–266.

    Article  PubMed  CAS  Google Scholar 

  • Theeuwes, J. (1991). Exogenous and endogenous control of attention: The effect of visual onsets and offsets. Perception and Psychophysics, 49, 83–90.

    Article  PubMed  CAS  Google Scholar 

  • Turatto, M., Mazza, V., & Umilta, C. (2005). Crossmodal object-based attention: Auditory objects affect visual processing. Cognition, 96, B55–B64.

    Article  PubMed  Google Scholar 

  • Van Damme, S., Gallace, A., Spence, C., Crombez, G., & Moseley, G. L. (2009). Does the sight of physical threat induce a tactile processing bias? Modality-specific attentional facilitation induced by viewing threatening pictures. Brain Research, 1253, 100–106.

    Article  PubMed  Google Scholar 

  • von Kriegstein, K., Kleinschmidt, A., Sterzer, P., & Giraud, A. L. (2005). Interaction of face and voice areas during speaker recognition. Journal of Cognitive Neuroscience, 17, 367–376.

    Article  Google Scholar 

  • Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9, 585–594.

    Article  PubMed  Google Scholar 

  • Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J. (2001). Effects of attention and emotion on face processing in the human brain: An event-related fMRI study. Neuron, 30, 829–841.

    Article  PubMed  CAS  Google Scholar 

  • Vuilleumier, P., & Brosch, T. (2009). Interactions of emotion and attention. In M. S. Gazzaniga (Ed.), The cognitive neurosciences IV (pp. 925–934). Cambridge, MA: MIT Press.

    Google Scholar 

  • Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45, 174–194.

    Article  PubMed  Google Scholar 

  • Vuilleumier, P., & Schwartz, S. (2001). Beware and be aware: Capture of spatial attention by fear-related stimuli in neglect. Neuroreport, 12, 1119–1122.

    Article  PubMed  CAS  Google Scholar 

  • West, G. L., Anderson, A. A., Ferber, S., & Pratt, J. (2011). Electrophysiological evidence for biased competition in V1 for fear expressions. Journal of Cognitive Neuroscience, 23(11), 3410–3418.

    Article  PubMed  Google Scholar 

  • West, G. L., Anderson, A. A., & Pratt, J. (2009). Motivationally significant stimuli show visual prior entry: Evidence for attentional capture. Journal of Experimental Psychology-Human Perception and Performance, 35, 1032–1042.

    Article  PubMed  Google Scholar 

  • Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998). Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411–418.

    PubMed  CAS  Google Scholar 

  • Wolfe, J. M., & Horowitz, T. S. (2004). What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience, 5, 495–501.

    Article  PubMed  CAS  Google Scholar 

  • Yantis, S. (1992). Multielement visual tracking: Attention and perceptual organization. Cognitive Psychology, 24, 295–340.

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, New York University, 6 Washington Place, New York, NY, 10003, USA

    Tobias Brosch

  2. Department of Psychology, Centre Interfacultaire en Sciences Affectives (NCCR), University of Geneva, 7 rue des Battoirs, 1205, Geneva, Switzerland

    Didier Grandjean

Authors
  1. Tobias Brosch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Didier Grandjean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Didier Grandjean .

Editor information

Editors and Affiliations

  1. Dept. Psychology, Voice Neurocognition Lab., University of Glasgow, Glasgow, Glasgow, G12 8QB, United Kingdom

    Pascal Belin

  2. Brugmann University Hospital, Brussels, B-1020, Belgium

    Salvatore Campanella

  3. University of Tubingen, Tubingen, 72074, Germany

    Thomas Ethofer

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Brosch, T., Grandjean, D. (2013). Cross-Modal Modulation of Spatial Attention by Emotion. In: Belin, P., Campanella, S., Ethofer, T. (eds) Integrating Face and Voice in Person Perception. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3585-3_11

Download citation

Publish with us

Policies and ethics

Search

Navigation