Skip to main content
SpringerLink
Log in

Dorsal and Ventral Cortical Pathways for Visuo-haptic Shape Integration Revealed Using fMRI

  • Chapter
  • First Online:
Multisensory Object Perception in the Primate Brain

Abstract

Two sensory streams theories have had an important influence on sensory and sensorimotor research for the past several decades. Here we apply the perspective of two sensory streams to interactions between visual and haptic object shape processes. We specifically focus on the presence and pattern of multisensory integration or neuronal convergence in dorsal action pathways and ventral perception pathways. To investigate integration of visual and haptic processing streams, we assessed potential sites of visuo-haptic integration for a phenomenon called inverse effectiveness, that is, increased multisensory gain with decreasing stimulus salience. Unexpectedly, the opposite pattern, which we called enhanced effectiveness, was found. Nevertheless, finding enhanced effectiveness implies neuronal convergence of visual and haptic inputs in regions considered part of separable dorsal action and ventral perception visuo-haptic processing pathways.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Amedi A, Jacobson G, Hendler T, Malach R, Zohary E (2002) Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb Cortex 12:1202–1212

    Article  PubMed  Google Scholar 

  • Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci 10:687–689

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal indentification and object recognition. Exp Brain Res 166:559–571

    Article  PubMed  CAS  Google Scholar 

  • Arnott SR, Binns MA, Grady CL, Alain C (2004) Assessing the auditory dual-pathway model in humans. Neuroimage 22:401–408

    Article  PubMed  Google Scholar 

  • Baizer JS, Ungerleider LG, Desimone R (1991) Organization of visual inputs to the inferior temporal and posterior parietal cortex in macaques. J Neurosci 11:168–190

    PubMed  CAS  Google Scholar 

  • Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 17:377–391

    Article  PubMed  Google Scholar 

  • Beauchamp MS (2005a) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145–153

    Article  PubMed  CAS  Google Scholar 

  • Beauchamp MS (2005b) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93–113

    Article  PubMed  Google Scholar 

  • Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004a) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190–1192

    Article  PubMed  CAS  Google Scholar 

  • Beauchamp MS, Lee KE, Argall BD, Martin A (2004b) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809–823

    Article  PubMed  CAS  Google Scholar 

  • Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory--visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849–872

    Article  PubMed  CAS  Google Scholar 

  • Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 94:115–147

    Article  PubMed  CAS  Google Scholar 

  • Binkofski F, Dohle C, Posse S, Stephan KM, Hefter H, Seitz RJ, Freund HJ (1998) Human anterior intraparietal area subserves prehension: a combined lesion and functional MRI activation study. Neurology 50:1253–1259

    Article  PubMed  CAS  Google Scholar 

  • Binkofski F, Kunesch E, Classen J, Seitz RJ, Freund HJ (2001) Tactile apraxia: unimodal apractic disorder of tactile object exploration associated with parietal lobe lesions. Brain 124:132–144

    Article  PubMed  CAS  Google Scholar 

  • Bodegard A, Geyer S, Grefkes C, Zilles K, Roland PE (2001) Hierarchical processing of tactile shape in the human brain. Neuron 31:317–328

    Article  PubMed  CAS  Google Scholar 

  • Buneo CA, Jarvis MR, Batista AP, Andersen RA (2002) Direct visuomotor transformations for reaching. Nature 416:632–636

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649–657

    Article  PubMed  CAS  Google Scholar 

  • Clark A (1997) Being there. MIT Press, Cambridge

    Google Scholar 

  • Coats R, Bingham GP, Mon-Williams M (2008) Calibrating grasp size and reach distance: interactions reveal integral organization of reaching-to-grasp movements. Exp Brain Res 189:211–220

    Article  PubMed  Google Scholar 

  • Culham JC, Cavina-Pratesi C, Singhal A (2006) The role of parietal cortex in visuomotor control: what have we learned from neuroimaging? Neuropsychologia 44:2668–2684

    Article  PubMed  Google Scholar 

  • Culham JC, Kanwisher NG (2001) Neuroimaging of cognitive functions in human parietal cortex. Curr Opin Neurobiol 11:157–163

    Article  PubMed  CAS  Google Scholar 

  • Culham JC, Valyear KF (2006) Human parietal cortex in action. Curr Opin Neurobiol 16:205–212

    Article  PubMed  CAS  Google Scholar 

  • de Sa VR, Ballard DH (1998) Perceptual learning from cross-modal feedback. In: Goldstone RL, Schyns PG, Medin DL (eds) Psychology of learning and motivation, vol 36. Adademic Press, San Diego, CA

    Google Scholar 

  • Dijkerman HC, de Haan EH (2007) Somatosensory processes subserving perception and action. Behav Brain Sci 30:189–201; discussion 201–139

    Article  PubMed  Google Scholar 

  • Easton RD, Greene AJ, Srinivas K (1997a) Transfer between vision and haptics: memory for 2-D patterns and 3-D objects. Psychonomic Bulletin and Review 4:403–410

    Article  Google Scholar 

  • Easton RD, Srinivas K, Greene AJ (1997b) Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. J Exp Psychol: Learn, Mem, Cogn 23:153–163

    Article  CAS  Google Scholar 

  • Feinberg TE, Gonzalez Rothi LJ, Heilman KM (1986) Multimodal agnosia after unilateral left hemisphere lesion. Neurology 36:864–867

    Article  PubMed  CAS  Google Scholar 

  • Frey SH, Vinton D, Norlund R, Grafton ST (2005) Cortical topography of human anterior intraparietal cortex active during visually guided grasping. Brain Res Cogn Brain Res 23:397–405

    Article  PubMed  Google Scholar 

  • Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15:20–25

    Article  PubMed  CAS  Google Scholar 

  • Goodale MA, Milner AD, Jakobson LS, Carey DP (1991) A neurological dissociation between perceiving objects and grasping them. Nature 349:154–156

    Article  PubMed  CAS  Google Scholar 

  • Grefkes C, Fink GR (2005) The functional organization of the intraparietal sulcus in humans and monkeys. J Anat 207:3–17

    Article  PubMed  Google Scholar 

  • Grefkes C, Weiss PH, Zilles K, Fink GR (2002) Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron 35:173–184

    Article  PubMed  CAS  Google Scholar 

  • Grill-Spector K, Kourtzi Z, Kanwisher N (2001) The lateral occipital complex and its role in object recognition. Vision Res 41:1409–1422

    Article  PubMed  CAS  Google Scholar 

  • Hickok G, Poeppel D (2004) Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 92:67–99

    Article  PubMed  Google Scholar 

  • Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615–1637

    PubMed  CAS  Google Scholar 

  • Humphrey GK, Goodale MA, Corbetta M, Aglioti S (1995) The McCollough effect reveals orientation discrimination in a case of cortical blindness. Curr Biol 5:545–551

    Article  PubMed  CAS  Google Scholar 

  • Humphrey GK, Goodale MA, Jakobson LS, Servos P (1994) The role of surface information in object recognition: studies of a visual form agnosic and normal subjects. Perception 23:1457–1481

    Article  PubMed  CAS  Google Scholar 

  • James TW, Culham JC, Humphrey GK, Milner AD, Goodale MA (2003) Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain 126:2463–2475

    Article  PubMed  Google Scholar 

  • James TW, Humphrey GK, Gati JS, Menon RS, Goodale MA (2000) The effects of visual object priming on brain activation before and after recognition. Curr Biol 10:1017–1024

    Article  PubMed  CAS  Google Scholar 

  • James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714

    Article  PubMed  Google Scholar 

  • James TW, James KH, Humphrey GK, Goodale MA (2005) Do visual and tactile object representations share the same neural substrate? In: Heller MA, Ballesteros S (eds) Touch and blindness: psychology and neuroscience. Lawrence Erlbaum, Mahwah, NJ

    Google Scholar 

  • James TW, Kim S, Fisher JS (2007) The neural basis of haptic object processing. Can J Exp Psychol 61:219–229

    Article  PubMed  Google Scholar 

  • Kim S, James TW (in press) Enhanced effectiveness in visuo-haptic object-selective brain regions with increasing stimulus salience. Hum Brain Mapp

    Google Scholar 

  • Kitada R, Kito T, Saito DN, Kochiyama T, Matsumura M, Sadato N, Lederman SJ (2006) Multisensory activation of the intraparietal area when classifying grating orientation: a functional magnetic resonance imaging study. J Neurosci 26:7491–7501

    Article  PubMed  CAS  Google Scholar 

  • Kourtzi Z, Tolias AS, Altmann CF, Augath M, Logothetis NK (2003) Integration of local features into global shapes: monkey and human fMRI studies. Neuron 37:333–346

    Article  PubMed  CAS  Google Scholar 

  • Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21:269–274

    Article  PubMed  Google Scholar 

  • Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27:1155–1163

    Article  PubMed  Google Scholar 

  • Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289–297

    Article  PubMed  Google Scholar 

  • Lennie P (1998) Single units and visual cortical organization. Perception 27:889–935

    Article  PubMed  CAS  Google Scholar 

  • Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci U S A 92:8135–8139

    Article  PubMed  CAS  Google Scholar 

  • Maunsell JH, Sclar G, Nealey TA, DePriest DD (1991) Extraretinal representations in area V4 in the macaque monkey. Vis Neurosci 7:561–573

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA (2002) On the neuronal basis for multisensory convergence: a brief overview. Brain Res Cogn Brain Res 14:31–40

    Article  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1986) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662

    PubMed  CAS  Google Scholar 

  • Milner AD, Goodale MA (1995) The visual brain in action. Oxford University Press, Oxford, UK

    Google Scholar 

  • Milner AD, Perrett DI, Johnston RS, Benson RS, Jordan PJ, Heeley TR, Bettucci DW, Mortara D, Mutani F, Terazzi R, Davidson DLW (1991) Perception and action in visual form agnosia. Brain 114:405–428

    Article  PubMed  Google Scholar 

  • Molyneux W (1688) Letter to John Locke. In: de Beer ES (ed) The correspondence of John Locke, vol 3. Clarendon Press, Oxford

    Google Scholar 

  • Morel A, Bullier J (1990) Anatomical segregation of two cortical visual pathways in the macaque monkey. Vis Neurosci 4:555–578

    Article  PubMed  CAS  Google Scholar 

  • Morin P, Rivrain Y, Eustache F, Lambert J, Courtheoux P (1984) Visual and tactile agnosia. Rev Neurol 140:271–277

    PubMed  CAS  Google Scholar 

  • Murata A, Gallese V, Luppino G, Kaseda M, Sakata H (2000) Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol 83:2580–2601

    PubMed  CAS  Google Scholar 

  • Newell FN, Ernst MO, Tjan BS, Bulthoff HH (2001) Viewpoint dependence in visual and haptic object recognition. Psychol Sci 12:37–42

    Article  PubMed  CAS  Google Scholar 

  • Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and haptic perception of natural object shape. Percept Psychophys 66:342–351

    Article  PubMed  Google Scholar 

  • Ohtake H, Fujii T, Yamadori A, Fujimori M, Hayakawa Y, Suzuki K (2001) The influence of misnaming on object recognition: a case of multimodal agnosia. Cortex 37:175–186

    Article  PubMed  CAS  Google Scholar 

  • Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. Prog Brain Res 134:427–445

    Article  PubMed  CAS  Google Scholar 

  • Pause M, Kunesch E, Binkofsku F, Freund HJ (1989) Sensorimotor disturbances in patients with lesions of the parietal corex. Brain 112:1599–1625

    Article  PubMed  Google Scholar 

  • Peiffer AM, Mozolic JL, Hugenschmidt CE, Laurienti PJ (2007) Age-related multisensory enhancement in a simple audiovisual detection task. Neuroreport 18:1077–1081

    Article  PubMed  Google Scholar 

  • Peltier S, Stilla R, Mariola E, LaConte S, Hu X, Sathian K (2007) Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia 45:476–483

    Article  PubMed  Google Scholar 

  • Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A default mode of brain function. Proc Natl Acad Sci U S A 98:676–682

    Article  PubMed  CAS  Google Scholar 

  • Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: corss-modal priming depends on structural descriptions. J Exp Psychol: Learn, Mem, Cogn 25:644–663

    Article  Google Scholar 

  • Reed CL, Klatzky RL, Halgren E (2005) What vs. where in touch: an fMRI study. Neuroimage 25:718–726

    Article  PubMed  Google Scholar 

  • Reed CL, Shoham S, Halgren E (2004) Neural substrates of tactile object recognition: an fMRI study. Hum Brain Mapp 21:236–246

    Article  PubMed  Google Scholar 

  • Roland PE, O'Sullivan B, Kawashima R (1998) Shape and roughness activate different somatosensory areas in the human brain. Proc Natl Acad Sci 95:3295–3300

    Article  PubMed  CAS  Google Scholar 

  • Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8:3877–3881

    Article  PubMed  CAS  Google Scholar 

  • Saur D, Kreher BW, Schnell S, Kummerer D, Kellmeyer P, Vry MS, Umarova R, Musso M, Glauche V, Abel S, Huber W, Rijntjes M, Hennig J, Weiller C (2008) Ventral and dorsal pathways for language. Proc Natl Acad Sci U S A 105:18035–18040

    Article  PubMed  CAS  Google Scholar 

  • Stark CE, Squire LR (2001) When zero is not zero: the problem of ambiguous baseline conditions in fMRI. Proc Natl Acad Sci U S A 98:12760–12766

    Article  PubMed  CAS  Google Scholar 

  • Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9:255–266

    Article  PubMed  CAS  Google Scholar 

  • Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp Brain Res 179:85–95

    Article  PubMed  Google Scholar 

  • Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210–1223

    Article  PubMed  Google Scholar 

  • Stilla R, Sathian K (2008) Selective visuo-haptic processing of shape and texture. Hum Brain Mapp 29:1123–1138

    Article  PubMed  Google Scholar 

  • Taira M, Mine S, Georgopoulos AP, Murata A, Sakata H (1990) Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Exp Brain Res 83:29–36

    Article  PubMed  CAS  Google Scholar 

  • Tal N, Amedi A (2009) Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach. Exp Brain Res 198:165–182

    Article  PubMed  Google Scholar 

  • Tootell RB, Tsao D, Vanduffel W (2003) Neuroimaging weighs in: Humans meet macaques in “primate” visual cortex. J Neurosci 23:3981–3989

    PubMed  CAS  Google Scholar 

  • Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: Ingle DJ, Goodale MA, Mansfield RJ (eds) The analysis of visual behavior. MIT Press, Cambridge, MA, pp 549–586

    Google Scholar 

  • Young MP (1992) Objective analysis of the topological organization of the primate cortical visual system. Nature 358:152–155

    Article  PubMed  CAS  Google Scholar 

  • Zangaladze A, Epstein CM, Grafton ST, Sathian K (1999) Involvement of visual cortex in tactile discrimination of orientation. Nature 401:587–590

    Article  PubMed  CAS  Google Scholar 

  • Zhang M, Weisser VD, Stilla R, Prather SC, Sathian K (2004) Multisensory cortical processing of object shape and its relation to mental imagery. Cogn Affect Behav Neurosci 4:251–259

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the Faculty Research Support Program, administered by the Indiana University Office of the Vice President of Research, and in part by the Indiana METACyt Initiative of Indiana University, funded in part through a major grant from the Lilly Endowment, Inc.

Author information

Authors and Affiliations

  1. Department of Psychological and Brain Sciences, Cognitive Science Program, Indiana University, 1101 E Tenth St., Bloomington, IN, 47405, USA

    Thomas W. James & Sunah Kim

Authors
  1. Thomas W. James
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sunah Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas W. James .

Editor information

Editors and Affiliations

  1. Inst. Medizinische Psychologie (IMP), Universität Frankfurt, Heinrich-Hoffmann-Str. 10, Frankfurt, 60528, Germany

    Jochen Kaiser

  2. Abt. Neurophysiologie, MPI für Hirnforschung, Deutschordenstr. 46, Frankfurt/Main, 60528, Germany

    Marcus Johannes Naumer

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science + Business Media, LLC

About this chapter

Cite this chapter

James, T.W., Kim, S. (2010). Dorsal and Ventral Cortical Pathways for Visuo-haptic Shape Integration Revealed Using fMRI. In: Kaiser, J., Naumer, M. (eds) Multisensory Object Perception in the Primate Brain. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5615-6_13

Download citation

Publish with us

Policies and ethics

Search

Navigation