Multiple Mechanisms of Top-Down Processing in Vision

  • Giorgio Ganis
  • Stephen M. Kosslyn


No animal could survive for long without perception. We must perceive the world, not only to find food, shelter, and mates, but also to avoid predators. Perception will fail if an animal does not register what is actually in the world. However, this simple observation does not imply that all processing during perception is “bottom up”rd—driven purely by the sensory input. Rather, bottom-up processing can be usefully supplemented by using stored information, engaging in processing that is “top down”—driven by stored knowledge, goals, or expectations. In this chapter we explore the nature of top-down processing and its intimate dance with bottom-up processing. We begin by considering basic facts about the primate visual system, and then consider a theory of its functional organization, followed by novel proposals regarding the nature of different sorts of top-down processing.


Visual Area Illusory Contour Ventral Stream Attention Window Inferotemporal Cortex 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andersen RA, Essick GK, Siegel RM (1985) Encoding of spatial location by posterior parietal neurons. Science 230:456–458PubMedCrossRefGoogle Scholar
  2. Barone P, Batardiere A, Knoblauch K, Kennedy H (2000) Laminar distribution of neurons in extrastriate areas projecting to visual areas V1 and V4 correlates with the hierarchical rank and indicates the operation of a distance rule. J Neurosci 20:3263–3281PubMedGoogle Scholar
  3. Beauchamp MS, Petit L, Ellmore TM, Ingeholm J, Haxby JV (2001) A parametric fMRI study of overt and covert shifts of visuospatial attention. Neuroimage 14:310–321PubMedCrossRefGoogle Scholar
  4. Brefczynski JA, DeYoe EA (1999) A physiological correlate of the’ spotlight’ of visual attention. Nat Neurosci 2:370–374PubMedCrossRefGoogle Scholar
  5. Budd JM (1998) Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity. Proc Biol Sci 265:1037–1044PubMedCrossRefGoogle Scholar
  6. Buffalo EA, Fries P, Landman R, Liang H, Desimone R (2005) Latency of attentional modulation in ventral visual cortex. Paper presented at the Society for Neuroscience, Washington, DCGoogle Scholar
  7. Cave KR, Kosslyn SM (1989) Varieties of size-specific visual selection. J Exp Psychol Gen 118:148–164PubMedCrossRefGoogle Scholar
  8. Chan D, Fox NC, Scahill RI, Crum WR, Whitwell JL, Leschziner G, et al. (2001) Patterns of temporal lobe atrophy in semantic dementia and Alzheimer’s disease. Ann Neurol 49:433–442PubMedCrossRefGoogle Scholar
  9. Clavagnier S, Falchier A, Kennedy H (2004) Long-distance feedback projections to area V1: implications for multisensory integration, spatial awareness, and visual consciousness. Cognit Affect Behav Neurosci 4:117–126CrossRefGoogle Scholar
  10. Corbetta M (1993) Positron emission tomography as a tool to study human vision and attention. Proc Natl Acad Sci U S A 90:10901–10903PubMedCrossRefGoogle Scholar
  11. Corbetta M, Shulman GL (1998) Human cortical mechanisms of visual attention during orienting and search. Philos Trans R Soc Lond B Biol Sci 353:1353–1362PubMedCrossRefGoogle Scholar
  12. Dale AM, Liu AK, Fischl BR, Buckner RL, Belliveau JW, Lewine JD (2000) Dynamic statistical parametric mapping: combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron 26:55–67PubMedCrossRefGoogle Scholar
  13. Damasio AR (1985) The frontal lobes. In: Heilman KM, Valenstein E (eds) Clinical neuropsychology. Oxford University Press, New YorkGoogle Scholar
  14. Desimone R, Ungerleider LG (1989) Neural mechanisms of visual processing in monkeys. In: Boller F, Grafman J (eds) Handbook of neuropsychology. Elsevier, Amsterdam, pp 267–299Google Scholar
  15. Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1–47PubMedCrossRefGoogle Scholar
  16. Fox PT, Mintun MA, Raichle ME, Miezin FM, Allman JM, Van Essen DC (1986) Mapping human visual cortex with positron emission tomography. Nature (Lond) 323:806–809PubMedCrossRefGoogle Scholar
  17. Fujita I, Tanaka K, Ito M, Cheng K (1992) Columns for visual features of objects in monkey inferotemporal cortex. Nature (Lond) 360:343–346PubMedCrossRefGoogle Scholar
  18. Fukushima K (1988) Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Networks 1:119–130CrossRefGoogle Scholar
  19. Fuster JM, Bauer RH, Jervey JP (1985) Functional interactions between inferotemporal and prefrontal cortex in a cognitive task. Brain Res 330:299–307PubMedCrossRefGoogle Scholar
  20. Ganis G, Schendan HE, Kosslyn SM (2007) Neuroimaging evidence for object model verification theory: role of prefrontal control in visual object categorization. Neuroimage 34:384–398PubMedCrossRefGoogle Scholar
  21. Girard P, Hupe J, Bullier J (2001) Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. J Neurophysiol 85:1328–1331PubMedGoogle Scholar
  22. Gregory RL (1970) The intelligent eye. Weidenfeld and Nicholson, LondonGoogle Scholar
  23. Gross CG, Mishkin M (1977) The neural basis of stimulus equivalence across retinal translation. In: Harnard S, Doty R, Jaynes, Goldstein JL, Krauthamer (eds) Lateralization in the visual system. Academic Press, New York, pp 109–122Google Scholar
  24. Grossberg S, Mingolla E (1985) Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading. Psychol Rev 92:173–211PubMedCrossRefGoogle Scholar
  25. Halgren E, Mendola J, Chong CD, Dale AM (2003) Cortical activation to illusory shapes as measured with magnetoencephalography. Neuroimage 18:1001–1009PubMedCrossRefGoogle Scholar
  26. Haxby JV, Grady CL, Horwitz B, Ungerleider LG, Mishkin M, Carson RE (1991) Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proc Natl Acad Sci U S A 88:1621–1625PubMedCrossRefGoogle Scholar
  27. Heeger DJ (1999) Linking visual perception with human brain activity. Curr Opin Neurobiol 9:474–479PubMedCrossRefGoogle Scholar
  28. Hilgetag CC, O’Neill MA, Young MP (1996) Indeterminate organization of the visual system. Science 271:776–777PubMedCrossRefGoogle Scholar
  29. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A 79:2554–2588PubMedCrossRefGoogle Scholar
  30. Hopfinger JB, Buonocore MH, Mangun GR (2000) The neural mechanisms of top-down attentional control. Nat Neurosci 3:284–291PubMedCrossRefGoogle Scholar
  31. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160:106–154PubMedGoogle Scholar
  32. Hubel DH, Wiesel TN (1965) Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J Neurophysiol 28:229–289PubMedGoogle Scholar
  33. Hubel DH, Wiesel TN (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol 195:215–243PubMedGoogle Scholar
  34. Hubel DH, Wiesel TN (1974) Uniformity of monkey striate cortex: a parallel relationship between field size, scatter, and magnification factor. J Comp Neurol 158:295–305PubMedCrossRefGoogle Scholar
  35. Huxlin KR, Saunders RC, Marchionini D, Pham HA, Merigan WH (2000) Perceptual deficits after lesions of inferotemporal cortex in macaques. Cereb Cortex 10:671–683PubMedCrossRefGoogle Scholar
  36. Kastner S, Pinsk MA, De Weerd P, Desimone R, Ungerleider LG (1999) Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron 22:751–761PubMedCrossRefGoogle Scholar
  37. Koechlin E, Basso G, Pietrini P, Panzer S, Grafman J (1999) The role of the anterior prefrontal cortex in human cognition. Nature (Lond) 399:148–151PubMedCrossRefGoogle Scholar
  38. Kosslyn SM (1994) Image and brain. MIT Press, CambridgeGoogle Scholar
  39. Kosslyn SM, Thompson WL, Alpert NM (1995) Identifying objects at different levels of hierarchy: a positron emission tomography study. Hum Brain Mapping 3:107–132CrossRefGoogle Scholar
  40. Kosslyn SM, Thompson WL, Alpert NM (1997) Neural systems shared by visual imagery and visual perception: a positron emission tomography study. Neuroimage 6:320–334PubMedCrossRefGoogle Scholar
  41. Kosslyn SM, Thompson WL, Ganis G (2006) The case for mental imagery. Oxford University Press, New YorkGoogle Scholar
  42. Kosslyn SM, Thompson WL, Gitelman DR, Alpert NM (1998) Neural systems that encode categorical vs. coordinate spatial relations: PET investigations. Psychobiology 26:333–347Google Scholar
  43. LaBerge D, Buchsbaum MS (1990) Positron emission tomographic measurements of pulvinar activity during an attention task. J Neurosci 10:613–619PubMedGoogle Scholar
  44. Lamme VA, Roelfsema PR (2000) The distinct modes of vision offered by feed-forward and recurrent processing. Trends Neurosci 23:571–579PubMedCrossRefGoogle Scholar
  45. Lee TS, Mumford D (2003) Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am A Opt Image Sci Vis 20:1434–1448PubMedCrossRefGoogle Scholar
  46. Lee TS, Nguyen M (2001) Dynamics of subjective contour formation in the early visual cortex. Proc Natl Acad Sci U S A 98:1907–1911PubMedCrossRefGoogle Scholar
  47. Li Z (1998) A neural model of contour integration in the primary visual cortex. Neural Comput 10:903–940PubMedCrossRefGoogle Scholar
  48. Liu AK, Belliveau JW, Dale AM (1998) Spatiotemporal imaging of human brain activity using functional MRI constrained magnetoencephalography data: Monte Carlo simulations. Proc Natl Acad Sci U S A 95:8945–8950PubMedCrossRefGoogle Scholar
  49. Luck SJ, Chelazzi L, Hillyard SA, Desimone R (1997) Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. J Neurophysiol 77:24–42PubMedGoogle Scholar
  50. Luria AR (1980) Higher cortical functions in man. Basic Books, New YorkGoogle Scholar
  51. McAuliffe SP, Knowlton BJ (2000) Long-term retinotopic priming in object identification. Percept Psychophys 62:953–959PubMedGoogle Scholar
  52. McDermott KB, Roediger H III (1994) Effects of imagery on perceptual implicit memory tests. J Exp Psychol Learn Mem Cognit 20:1379–1390CrossRefGoogle Scholar
  53. McMains SA, Somers DC (2004) Multiple spotlights of attentional selection in human visual cortex. Neuron 42:677–686PubMedCrossRefGoogle Scholar
  54. Mechelli A, Price CJ, Friston KJ, Ishai A (2004) Where bottom-up meets top-down: neuronal interactions during perception and imagery. Cereb Cortex 14:1256–1265PubMedCrossRefGoogle Scholar
  55. Mesulam MM (1981) A cortical network for directed attention and unilateral neglect. Ann Neurol 10:309–325PubMedCrossRefGoogle Scholar
  56. Mesulam MM (1990) Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Ann Neurol 28:597–613PubMedCrossRefGoogle Scholar
  57. Miller EK, Cohen JD (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24:167–202PubMedCrossRefGoogle Scholar
  58. Mishkin M, Ungerleider LG, Macko KA (1983) Object vision and spatial vision: Two cortical pathways. Trends Neurosci 6:414–417CrossRefGoogle Scholar
  59. Miyashita Y, Chang HS (1988) Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature (Lond) 331:68–70PubMedCrossRefGoogle Scholar
  60. Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex. Nature (Lond) 421:370–373PubMedCrossRefGoogle Scholar
  61. Mumford D (1992) On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybern 66:241–251PubMedCrossRefGoogle Scholar
  62. Neisser U (1967) Cognitive psychology. Appleton-Century-Crofts, New YorkGoogle Scholar
  63. Neisser U (1976) Cognition and reality. Freeman, San FranciscoGoogle Scholar
  64. O’Regan JK, Noe A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24:939–973; discussion 973–1031PubMedCrossRefGoogle Scholar
  65. Payne BR, Lomber SG, Villa AE, Bullier J (1996) Reversible deactivation of cerebral network components. Trends Neurosci 19:535–542PubMedCrossRefGoogle Scholar
  66. Petrides M (2005) Lateral prefrontal cortex: architectonic and functional organization. Philos Trans R Soc Lond B Biol Sci 360:781–795PubMedCrossRefGoogle Scholar
  67. Posner MI, Petersen SE (1990) The attention system of the human brain. Annu Rev Neurosci 13:25–42PubMedCrossRefGoogle Scholar
  68. Posner MI, Snyder CR, Davidson BJ (1980) Attention and the detection of signals. J Exp Psychol 109:160–174PubMedGoogle Scholar
  69. Rao SC, Rainer G, Miller EK (1997) Integration of what and where in the primate prefrontal cortex. Science 276:821–824PubMedCrossRefGoogle Scholar
  70. Ress D, Backus BT, Heeger DJ (2000). Activity in primary visual cortex predicts performance in a visual detection task. Nat Neurosci 3:940–945PubMedCrossRefGoogle Scholar
  71. Reynolds JH, Chelazzi L, Desimone R (1999) Competitive mechanisms subserve attention in macaque areas V2 and V4. J Neurosci 19:1736–1753PubMedGoogle Scholar
  72. Riesenhuber M, Poggio T (1999) Hierarchical models of object recognition in cortex. Nat Neurosci 2:1019–1025PubMedCrossRefGoogle Scholar
  73. Rockland KS, Pandya DN (1979) Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey. Brain Res 179:3–20PubMedCrossRefGoogle Scholar
  74. Rueckl JG, Cave KR, Kosslyn SM (1989) Why are “what” and “where” processed by separate cortical visual systems? A computational investigation. J Cognit Neurosci 1:171–186CrossRefGoogle Scholar
  75. Salin PA, Bullier J (1995) Cortico-cortical connections in the visual system: structure and function. Physiol Rev 75:107–154PubMedGoogle Scholar
  76. Sandell JH, Schiller PH (1982) Effect of cooling area 18 on striate cortex cells in the squirrel monkey. J Neurophysiol 48:38–48PubMedGoogle Scholar
  77. Schacter DL (1996) Searching for memory. Harper Collins, New YorkGoogle Scholar
  78. Seghier ML, Vuilleumier P (2006) Functional neuroimaging findings on the human perception of illusory contours. Neurosci Biobehav Rev 30:595–612PubMedCrossRefGoogle Scholar
  79. Sereno MI, Dale AM, Reppas JB, Kwong KK, Belliveau JW, Brady TJ (1995) Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268:889–893PubMedCrossRefGoogle Scholar
  80. Sereno MI, Pitzalis S, Martinez A (2001) Mapping of contralateral space in retinotopic coordinates by a parietal cortical area in humans. Science 294:1350–1354PubMedCrossRefGoogle Scholar
  81. Sereno MI, Tootell RB (2005) From monkeys to humans: what do we now know about brain homologies? Curr Opin Neurobiol 15:135–144PubMedCrossRefGoogle Scholar
  82. Serre T, Oliva A, Poggio T (2007) A feedforward architecture accounts for rapid categorization. Proc Natl Acad Sci USA 104:6424–6429PubMedCrossRefGoogle Scholar
  83. Squire LR (1987) Memory and brain. Oxford University Press, OxfordGoogle Scholar
  84. Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19:109–139PubMedCrossRefGoogle Scholar
  85. Tanaka K, Saito H, Fukada Y, Moriya M (1991) Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J Neurophysiol 66:170–189PubMedGoogle Scholar
  86. Tomita H, Ohbayashi M, Nakahara K, Hasegawa I, Miyashita Y (1999) Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature Lond) 401: 699–703CrossRefGoogle Scholar
  87. Tootell RB, Hadjikhani NK, Vanduffel W, Liu AK, Mendola JD, Sereno MI (1998) Functional analysis of primary visual cortex (V1) in humans. Proc Natl Acad Sci U S A 95:811–817PubMedCrossRefGoogle Scholar
  88. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cognit Psychol 12:97–136PubMedCrossRefGoogle Scholar
  89. Ullman S (1989) Aligning pictorial descriptions: an approach to object recognition. Cognition 32:193–254PubMedCrossRefGoogle Scholar
  90. Ullman S (1995) Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex. Cereb Cortex 5:1–11PubMedCrossRefGoogle Scholar
  91. Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: Ingle DJ, Goodale MA, Mansfield RJW (eds) Analysis of visual behavior. MIT Press, Cambridge, pp 549–586Google Scholar
  92. Van Essen DC, Lewis JW, Drury HA, Hadjikhani N, Tootell RB, Bakircioglu M (2001) Mapping visual cortex in monkeys and humans using surface-based atlases. Vision Res 41:1359–1378PubMedCrossRefGoogle Scholar
  93. Van Rullen R, Delorme A, Thorpe SJ (2001) Feed-forward contour integration in primary visual cortex based on asynchronous spike propagation. Neurocomputing 38:1003–1009CrossRefGoogle Scholar
  94. Vezoli J, Falchier A, Jouve B, Knoblauch K, Young M, Kennedy H (2004) Quantitative analysis of connectivity in the visual cortex: extracting function from structure. Neuroscientist 10:476–482PubMedCrossRefGoogle Scholar
  95. von der Heydt R, Peterhans E, Baumgartner G (1984) Illusory contours and cortical neuron responses. Science 224:1260–1262PubMedCrossRefGoogle Scholar
  96. Wallis G, Rolls ET (1997) Invariant face and object recognition in the visual system. Prog Neurobiol 51:167–194PubMedCrossRefGoogle Scholar
  97. Wilson FA, Scalaidhe SP, Goldman-Rakic PS (1993) Dissociation of object and spatial processing domains in primate prefrontal cortex. Science 260:1955–1958PubMedCrossRefGoogle Scholar

Copyright information

© Springer 2007

Authors and Affiliations

  • Giorgio Ganis
    • 1
    • 2
    • 3
  • Stephen M. Kosslyn
    • 3
    • 4
  1. 1.Department of RadiologyHarvard Medical SchoolBostonUSA
  2. 2.Massachusetts General HospitalMartinos CenterCharlestownUSA
  3. 3.Department of PsychologyHarvard UniversityCambridgeUSA
  4. 4.Department of NeurologyMassachusetts General HospitalBostonUSA

Personalised recommendations