Skip to main content
SpringerLink
Log in
Book cover

Integrating Face and Voice in Person Perception pp 225–251Cite as

Audiovisual Integration of Emotional Information from Voice and Face

  • Chapter
  • First Online:

Abstract

When judging their social counterpart’s emotional state, humans predominantly rely on nonverbal signals. In a natural environment, this nonverbal emotional communication is multimodal (i.e., facial expressions and speech melody, but also gestures, posture, or nonverbal vocalizations). Therefore, the integration of information from different sensory channels into a common percept of the current emotional state, intentions, or attitude of the social counterpart presents an elementary ability required for successful social interaction.

The first part of this chapter deals with current behavioral, neuroanatomical, electrophysiological, and neuroimaging studies on the integration of nonverbal emotional information from voice and face with special emphasis on functional magnetic resonance imaging (fMRI). The correlates of audiovisual integration of emotional information on the different levels of observation (behavioral, electrophysiological, neuroimaging) are discussed with respect to neuroanatomical data and along with methodological issues concerning current concepts of multisensory integration.

In the second part of the chapter, a methodological focus is put on the different analytical approaches (conjunction analyses, interaction analyses, correlation analyses, and connectivity analyses) used to capture and localize integration effects in the human brain as well as on the relationship between integration effects on different observational levels. We argue that none of these methods captures all facets of the integration process but that instead each of these approaches provides complementary information for the assessment of different aspects of multisensory integration of emotional signals. We demonstrate that the employment of multiple analysis techniques is necessary to dissociate effects of audiovisual emotional integration from possible confounds such as basic effects of spatiotemporal voice–face correspondence or effects of audiovisual integration of speech content.

The third and last part of this chapter is dedicated to the alteration of audiovisual emotional integration processes in states of psychiatric disease. While processing of emotional cues in general is altered in many different psychiatric diseases, disturbance of multimodal integration occurs much less frequently. We review the yet relatively small but fast-growing number of studies in patients with schizophrenia as an exemplary psychiatric disorder with respect to alterations in behavior and neural processing of audiovisual nonverbal emotional information.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Adolphs R, Tranel D, Damasio H, Damasio A (1994) Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372(6507):669–672

    PubMed  CAS  Google Scholar 

  • Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. Journal of Cognitive Neuroscience 17(3):377–391

    PubMed  Google Scholar 

  • Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004a) Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience 7(11):1190–1192

    PubMed  CAS  Google Scholar 

  • Beauchamp MS, Lee KE, Argall BD, Martin A (2004b) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41(5):809–823

    PubMed  CAS  Google Scholar 

  • Belin P, Zatorre RJ, Lafaille P, Ahad P, Pike B (2000) Voice-selective areas in human auditory cortex. Nature 403(6767):309–312

    PubMed  CAS  Google Scholar 

  • Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Buckner RL et al (1996) Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17(5):875–887

    PubMed  CAS  Google Scholar 

  • Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. Journal of Neurophysiology 46(2):369–384

    PubMed  CAS  Google Scholar 

  • Calvert GA (2001) Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex 11(12):1110–1123

    PubMed  CAS  Google Scholar 

  • Calvert GA, Brammer MJ, Bullmore ET, Campbell R, Iversen SD, David AS (1999) Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport 10(12):2619–2623

    PubMed  CAS  Google Scholar 

  • Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology 10(11):649–657

    PubMed  CAS  Google Scholar 

  • Calvert GA, Spence C, Stein BE (eds) (2004) The handbook of multisensory processes. MIT Press, Cambridge, MA

    Google Scholar 

  • Calvert GA, Thesen T (2004) Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology 98(1–3):191–205

    PubMed  Google Scholar 

  • Chandrasekaran C, Ghazanfar AA (2009) Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. Journal of Neurophysiology 101(2):773–788

    PubMed  Google Scholar 

  • Chavis DA, Pandya DN (1976) Further observations on corticofrontal connections in the rhesus monkey. Brain Research 117(3):369–386

    PubMed  CAS  Google Scholar 

  • Chen YH, Edgar JC, Holroyd T, Dammers J, Thonnessen H, Roberts TP et al (2010) Neuromagnetic oscillations to emotional faces and prosody. European Journal of Neuroscience 31(10):1818–1827

    PubMed  Google Scholar 

  • Collignon O, Girard S, Gosselin F, Roy S, Saint-Amour D, Lassonde M et al (2008) Audio–visual integration of emotion expression. Brain Research 1242:126–135

    PubMed  CAS  Google Scholar 

  • Collignon O, Girard S, Gosselin F, Saint-Amour D, Lepore F, Lassonde M (2010) Women process multisensory emotion expressions more efficiently than men. Neuropsychologia 48(1):220–225

    PubMed  CAS  Google Scholar 

  • Dahl CD, Logothetis NK, Kayser C (2010) Modulation of visual responses in the superior temporal sulcus by audio–visual congruency. Frontiers in Integrative Neuroscience 4:10

    PubMed  Google Scholar 

  • Damasio AR (1989) Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition. Cognition 33(1–2):25–62

    PubMed  CAS  Google Scholar 

  • de Gelder B, Bocker KB, Tuomainen J, Hensen M, Vroomen J (1999) The combined perception of emotion from voice and face: Early interaction revealed by human electric brain responses. Neuroscience Letters 260(2):133–136

    PubMed  Google Scholar 

  • De Gelder B, Vroomen J (2000) The perception of emotions by ear and by eye. Cognition and Emotion 14(3):289–311

    Google Scholar 

  • de Gelder B, Vroomen J, de Jong SJ, Masthoff ED, Trompenaars FJ, Hodiamont P (2005) Multisensory integration of emotional faces and voices in schizophrenics. Schizophrenia Research 72(2–3):195–203

    PubMed  Google Scholar 

  • de Jong JJ, Hodiamont PP, de Gelder B (2010) Modality-specific attention and multisensory integration of emotions in schizophrenia: Reduced regulatory effects. Schizophrenia Research 122(1–3):136–143

    PubMed  Google Scholar 

  • de Jong JJ, Hodiamont PP, Van den Stock J, de Gelder B (2009) Audiovisual emotion recognition in schizophrenia: Reduced integration of facial and vocal affect. Schizophrenia Research 107(2–3):286–293

    PubMed  Google Scholar 

  • Dolan RJ, Morris JS, de Gelder B (2001) Crossmodal binding of fear in voice and face. Proceedings of the National Academy of Sciences of the United States of America 98(17):10006–10010

    PubMed  CAS  Google Scholar 

  • Driver J, Spence C (2000) Multisensory perception: Beyond modularity and convergence. Current Biology 10(20):R731–R735

    PubMed  CAS  Google Scholar 

  • Epstein R, Harris A, Stanley D, Kanwisher N (1999) The parahippocampal place area: Recognition, navigation, or encoding? Neuron 23(1):115–125

    PubMed  CAS  Google Scholar 

  • Ethofer T, Anders S, Erb M, Droll C, Royen L, Saur R et al (2006a) Impact of voice on emotional judgment of faces: An event-related fMRI study. Human Brain Mapping 27(9):707–714

    PubMed  Google Scholar 

  • Ethofer T, Pourtois G, Wildgruber D (2006b) Investigating audiovisual integration of emotional signals in the human brain. Progress in Brain Research 156:345–361

    PubMed  Google Scholar 

  • Fallon JH, Benevento LA, Loe PR (1978) Frequency-dependent inhibition to tones in neurons of cat insular cortex (AIV). Brain Research 145(1):161–167

    PubMed  CAS  Google Scholar 

  • Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME (2005) The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences of the United States of America 102(27):9673–9678

    PubMed  CAS  Google Scholar 

  • Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. NeuroReport 16(5):419–423

    PubMed  Google Scholar 

  • Fries W (1984) Cortical projections to the superior colliculus in the macaque monkey: A retrograde study using horseradish peroxidase. The Journal of Comparative Neurology 230(1):55–76

    PubMed  CAS  Google Scholar 

  • Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. NeuroImage 6(3):218–229

    PubMed  CAS  Google Scholar 

  • Friston KJ, Harrison L, Penny W (2003) Dynamic causal modelling. NeuroImage 19(4):1273–1302

    PubMed  CAS  Google Scholar 

  • Friston KJ, Holmes AP, Price CJ, Buchel C, Worsley KJ (1999) Multisubject fMRI studies and conjunction analyses. NeuroImage 10(4):385–396

    PubMed  CAS  Google Scholar 

  • Friston KJ, Josephs O, Rees G, Turner R (1998) Nonlinear event-related responses in fMRI. Magnetic Resonance in Medicine 39(1):41–52

    PubMed  CAS  Google Scholar 

  • Friston KJ, Mechelli A, Turner R, Price CJ (2000) Nonlinear responses in fMRI: The Balloon model, Volterra kernels, and other hemodynamics. NeuroImage 12(4):466–477

    PubMed  CAS  Google Scholar 

  • Friston KJ, Penny WD, Glaser DE (2005) Conjunction revisited. NeuroImage 25(3):661–667

    PubMed  Google Scholar 

  • Ghazanfar AA, Chandrasekaran C, Logothetis NK (2008) Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. Journal of Neuroscience 28(17):4457–4469

    PubMed  CAS  Google Scholar 

  • Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience 25(20):5004–5012

    PubMed  CAS  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory–visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience 11(5):473–490

    PubMed  CAS  Google Scholar 

  • Gitelman DR, Penny WD, Ashburner J, Friston KJ (2003) Modeling regional and psychophysiologic interactions in fMRI: The importance of hemodynamic deconvolution. NeuroImage 19(1):200–207

    PubMed  Google Scholar 

  • Goldman-Rakic PS (1995) Architecture of the prefrontal cortex and the central executive. Annals of the New York Academy of Sciences 769:71–83

    PubMed  CAS  Google Scholar 

  • Gordon B (1973) Receptive fields in deep layers of cat superior colliculus. Journal of Neurophysiology 36(2):157–178

    PubMed  CAS  Google Scholar 

  • Grossmann T, Striano T, Friederici AD (2006) Crossmodal integration of emotional information from face and voice in the infant brain. Developmental Science 9(3):309–315

    PubMed  Google Scholar 

  • Hagan CC, Woods W, Johnson S, Calder AJ, Green GG, Young AW (2009) MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus. Proceedings of the National Academy of Sciences of the United States of America 106(47):20010–20015

    PubMed  CAS  Google Scholar 

  • Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. Journal of Neurophysiology 60(5):1615–1637

    PubMed  CAS  Google Scholar 

  • John ER (2002) The neurophysics of consciousness. Brain Research Brain Research Reviews 39(1):1–28

    PubMed  Google Scholar 

  • Jones EG, Powell TP (1970) An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93(4):793–820

    PubMed  CAS  Google Scholar 

  • Kanwisher N, McDermott J, Chun MM (1997) The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17(11):4302–4311

    PubMed  CAS  Google Scholar 

  • Kayser C, Logothetis NK (2009) Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Frontiers in Integrative Neuroscience 3:7

    PubMed  Google Scholar 

  • Kayser C, Logothetis NK, Panzeri S (2010) Visual enhancement of the information representation in auditory cortex. Current Biology 20(1):19–24

    PubMed  CAS  Google Scholar 

  • Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48(2):373–384

    PubMed  CAS  Google Scholar 

  • Kayser C, Petkov CI, Augath M, Logothetis NK (2007) Functional imaging reveals visual modulation of specific fields in auditory cortex. Journal of Neuroscience 27(8):1824–1835

    PubMed  CAS  Google Scholar 

  • Kreifelts B, Ethofer T, Grodd W, Erb M, Wildgruber D (2007) Audiovisual integration of emotional signals in voice and face: An event-related fMRI study. NeuroImage 37(4):1445–1456

    PubMed  Google Scholar 

  • Kreifelts B, Ethofer T, Huberle E, Grodd W, Wildgruber D (2010) Association of trait emotional intelligence and individual fMRI-activation patterns during the perception of social signals from voice and face. Human Brain Mapping 31(7):979–991

    PubMed  Google Scholar 

  • Kreifelts B, Ethofer T, Shiozawa T, Grodd W, Wildgruber D (2009) Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus. Neuropsychologia 47(14):3059–3066

    PubMed  Google Scholar 

  • Lambrecht L, Kreifelts B, Wildgruber D (2012) Age-related decrease in recognition of emotional facial and prosodic expressions. Emotion. Epub ahead of print.

    Google Scholar 

  • Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Experimental Brain Research 166(3–4):289–297

    Google Scholar 

  • Lewis JW, Beauchamp MS, DeYoe EA (2000) A comparison of visual and auditory motion processing in human cerebral cortex. Cerebral Cortex 10(9):873–888

    PubMed  CAS  Google Scholar 

  • Llinas R, Ribary U (2001) Consciousness and the brain. The thalamocortical dialogue in health and disease. Annals of the New York Academy of Science 929:166–175

    CAS  Google Scholar 

  • Loe PR, Benevento LA (1969) Auditory–visual interaction in single units in the orbito-insular cortex of the cat. Electroencephalography and Clinical Neurophysiology 26(4):395–398

    PubMed  CAS  Google Scholar 

  • Macaluso E, Frith C, Driver J (2000) Selective spatial attention in vision and touch: Unimodal and multimodal mechanisms revealed by PET. Journal of Neurophysiology 83(5):3062–3075

    PubMed  CAS  Google Scholar 

  • Massaro DW, Egan PB (1996) Perceiving affect from the voice and the face. Psychonomic Bulletin and Review 3(2):215–221

    Google Scholar 

  • McDonald AJ (1998) Cortical pathways to the mammalian amygdala. Progress in Neurobiology 55(3):257–332

    PubMed  CAS  Google Scholar 

  • Mechelli A, Price CJ, Friston KJ (2001) Nonlinear coupling between evoked rCBF and BOLD signals: A simulation study of hemodynamic responses. NeuroImage 14(4):862–872

    PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221(4608):389–391

    PubMed  CAS  Google Scholar 

  • Mesulam MM (1998) From sensation to cognition. Brain 121(Pt 6):1013–1052

    PubMed  Google Scholar 

  • Mesulam MM, Mufson EJ (1982) Insula of the old world monkey. III: Efferent cortical output and comments on function. The journal of Comparative Neurology 212(1):38–52

    PubMed  CAS  Google Scholar 

  • Miller J (1982) Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology 14(2):247–279

    PubMed  CAS  Google Scholar 

  • Morris JS, Frith CD, Perrett DI, Rowland D, Young AW, Calder AJ et al (1996) A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383(6603):812–815

    PubMed  CAS  Google Scholar 

  • Mufson EJ, Mesulam MM (1984) Thalamic connections of the insula in the rhesus monkey and comments on the paralimbic connectivity of the medial pulvinar nucleus. The Journal of Comparative Neurology 227(1):109–120

    PubMed  CAS  Google Scholar 

  • Müller VI, Habel U, Derntl B, Schneider F, Zilles K, Turetsky BI et al (2011) Incongruence effects in crossmodal emotional integration. NeuroImage 54(3):2257–2266

    PubMed  Google Scholar 

  • Murray EA, Mishkin M (1985) Amygdalectomy impairs crossmodal association in monkeys. Science 228(4699):604–606

    PubMed  CAS  Google Scholar 

  • Nichols T, Brett M, Andersson J, Wager T, Poline JB (2005) Valid conjunction inference with the minimum statistic. NeuroImage 25(3):653–660

    PubMed  Google Scholar 

  • Park JY, Gu BM, Kang DH, Shin YW, Choi CH, Lee JM et al (2010) Integration of cross-modal emotional information in the human brain: An fMRI study. Cortex 46(2):161–169

    PubMed  Google Scholar 

  • Pearson RC, Brodal P, Gatter KC, Powell TP (1982) The organization of the connections between the cortex and the claustrum in the monkey. Brain Research 234(2):435–441

    PubMed  CAS  Google Scholar 

  • Peck CK (1987) Visual–auditory interactions in cat superior colliculus: Their role in the control of gaze. Brain Research 420(1):162–166

    PubMed  CAS  Google Scholar 

  • Phillips ML, Young AW, Scott SK, Calder AJ, Andrew C, Giampietro V et al (1998) Neural responses to facial and vocal expressions of fear and disgust. Proceedings of the Biological Sciences 265(1408):1809–1817

    CAS  Google Scholar 

  • Pitkänen A (2000) Connectivity of the rat amygdaloid complex. Oxford University Press, In The Amygdala. A functional analysis. New York

    Google Scholar 

  • Pourtois G, de Gelder B, Bol A, Crommelinck M (2005) Perception of facial expressions and voices and of their combination in the human brain. Cortex 41(1):49–59

    PubMed  Google Scholar 

  • Pourtois G, de Gelder B, Vroomen J, Rossion B, Crommelinck M (2000) The time-course of intermodal binding between seeing and hearing affective information. NeuroReport 11(6):1329–1333

    PubMed  CAS  Google Scholar 

  • Pourtois G, Debatisse D, Despland PA, de Gelder B (2002) Facial expressions modulate the time course of long latency auditory brain potentials. Brain Research Cognitive Brain Research 14(1):99–105

    PubMed  Google Scholar 

  • Price CJ, Friston KJ (1997) Cognitive conjunction: A new approach to brain activation experiments. NeuroImage 5(4 Pt 1):261–270

    PubMed  CAS  Google Scholar 

  • Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America 98(2):676–682

    PubMed  CAS  Google Scholar 

  • Robins DL, Hunyadi E, Schultz RT (2009) Superior temporal activation in response to dynamic audio–visual emotional cues. Brain and Cognition 69(2):269–278

    PubMed  Google Scholar 

  • Schroger E, Widmann A (1998) Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 35(6):755–759

    PubMed  CAS  Google Scholar 

  • Schutte N, Malouff J, Hall L, Haggerty D, Cooper J, Golden C et al (1998) Development and validation of a measure of emotional intelligence. Personality and Individual Differences 25:167–177

    Google Scholar 

  • Scott SK, Young AW, Calder AJ, Hellawell DJ, Aggleton JP, Johnson M (1997) Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 385(6613):254–257

    PubMed  CAS  Google Scholar 

  • Seltzer B, Pandya DN (1978) Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Research 149(1):1–24

    PubMed  CAS  Google Scholar 

  • Stanford TR, Stein BE (2007) Superadditivity in multisensory integration: Putting the computation in context. NeuroReport 18(8):787–792

    PubMed  Google Scholar 

  • Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience 8:497–506

    Google Scholar 

  • Stein BE, Meredith MA (1993) Merging of senses. MIT Press, Cambridge

    Google Scholar 

  • Stein BE, Wallace MT (1996) Comparisons of cross-modality integration in midbrain and cortex. Progress in Brain Research 112:289–299

    PubMed  CAS  Google Scholar 

  • Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Experimental Brain Research 179(1):85–95

    Google Scholar 

  • Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. NeuroImage 44(3):1210–1223

    PubMed  Google Scholar 

  • Szycik GR, Munte TF, Dillo W, Mohammadi B, Samii A, Emrich HM et al (2009) Audiovisual integration of speech is disturbed in schizophrenia: An fMRI study. Schizophrenia Research 110(1–3):111–118

    PubMed  CAS  Google Scholar 

  • van Atteveldt N, Formisano E, Goebel R, Blomert L (2004) Integration of letters and speech sounds in the human brain. Neuron 43(2):271–282

    PubMed  Google Scholar 

  • von Kriegstein K, Giraud AL (2006) Implicit multisensory associations influence voice recognition. PLoS Biology 4(10):e326

    Google Scholar 

  • Vroomen J, Driver J, de Gelder B (2001) Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective, & Behavioral Neuroscience 1(4):382–387

    CAS  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1992) Integration of multiple sensory modalities in cat cortex. Experimental Brain Research 91(3):484–488

    CAS  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1993) Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. Journal of Neurophysiology 69(6):1797–1809

    PubMed  CAS  Google Scholar 

  • Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. Journal of Neurophysiology 76(2):1246–1266

    PubMed  CAS  Google Scholar 

  • Wang Y, Celebrini S, Trotter Y, Barone P (2008) Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience 9:79

    PubMed  Google Scholar 

  • Werner S, Noppeney U (2010a) Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. Journal of Neuroscience 30(7):2662–2675

    PubMed  CAS  Google Scholar 

  • Werner S, Noppeney U (2010b) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cerebral Cortex 20(8):1829–1842

    PubMed  Google Scholar 

  • White M (1999) Representation of facial expressions of emotion. The American Journal of Psychology 112(3):371–381

    PubMed  CAS  Google Scholar 

  • Williams LE, Light GA, Braff DL, Ramachandran VS (2010) Reduced multisensory integration in patients with schizophrenia on a target detection task. Neuropsychologia 48(10):3128–3136

    PubMed  Google Scholar 

  • Worsley K, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans A (1996) A unified statistical approach for determining significant signals in images of cerebral activation. Human Brain Mapping 4(1):74–90

    PubMed  CAS  Google Scholar 

  • Wright TM, Pelphrey KA, Allison T, McKeown MJ, McCarthy G (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cerebral Cortex 13(10):1034–1043

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of General Psychiatry, University of Tübingen, Osianderstraße 24, 72076, Tübingen, Germany

    Benjamin Kreifelts, Dirk Wildgruber & Thomas Ethofer

Authors
  1. Benjamin Kreifelts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dirk Wildgruber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Ethofer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Kreifelts .

Editor information

Editors and Affiliations

  1. Dept. Psychology, Voice Neurocognition Lab., University of Glasgow, Glasgow, Glasgow, G12 8QB, United Kingdom

    Pascal Belin

  2. Brugmann University Hospital, Brussels, B-1020, Belgium

    Salvatore Campanella

  3. University of Tubingen, Tubingen, 72074, Germany

    Thomas Ethofer

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Kreifelts, B., Wildgruber, D., Ethofer, T. (2013). Audiovisual Integration of Emotional Information from Voice and Face. In: Belin, P., Campanella, S., Ethofer, T. (eds) Integrating Face and Voice in Person Perception. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3585-3_12

Download citation

Publish with us

Policies and ethics

Search

Navigation