Skip to main content
SpringerLink
Log in

The Functional Neuroanatomy of Face Processing: Insights from Neuroimaging and Implications for Deep Learning

  • Chapter
  • First Online:
Deep Learning for Biometrics

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Face perception is critical for normal social functioning, and is mediated by a cortical network of regions in the ventral visual stream. Comparative analysis between present deep neural network architectures for biometrics and neural architectures in the human brain is necessary for developing artificial systems with human abilities. Neuroimaging research has advanced our understanding regarding the functional architecture of the human ventral face network. Here, we describe recent neuroimaging findings in three domains: (1) the macro- and microscopic anatomical features of the ventral face network in the human brain, (2) the characteristics of white matter connections, and (3) the basic computations performed by population receptive fields within face-selective regions composing this network. Then, we consider how empirical findings can inform the development of accurate computational deep neural networks for face recognition as well as shed light on computational benefits of specific neural implementational features.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    See Appendix for abbreviations and definitions.

References

  1. T. Allison, H. Ginter, G. McCarthy, A.C. Nobre, A. Puce et al., Face recognition in human extrastriate cortex. J. Neurophysiol. 71, 821–825 (1994)

    Article  Google Scholar 

  2. T. Allison, G. McCarthy, A. Nobre, A. Puce, A. Belger, Human extrastriate visual cortex and the perception of faces, words, numbers, and colors. Cereb. Cortex 4, 544–554 (1994)

    Article  Google Scholar 

  3. T. Allison, A. Puce, D.D. Spencer, G. McCarthy, Electrophysiological studies of human face perception. I: potentials generated in occipitotemporal cortex by face and non-face stimuli. Cereb. Cortex 9, 415–430 (1999)

    Article  Google Scholar 

  4. T.J. Andrews, M.P. Ewbank, Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage 23, 905–913 (2004)

    Article  Google Scholar 

  5. S. Anzellotti, S.L. Fairhall, A. Caramazza, Decoding representations of face identity that are tolerant to rotation. Cereb. Cortex 24, 1988–1995 (2014)

    Article  Google Scholar 

  6. G. Avidan, M. Behrmann, Functional MRI reveals compromised neural integrity of the face processing network in congenital prosopagnosia. Curr. Biol. 19, 1146–1150 (2009)

    Article  Google Scholar 

  7. G. Avidan, I. Levy, T. Hendler, E. Zohary, R. Malach, Spatial vs. object specific attention in high-order visual areas. Neuroimage 19, 308–318 (2003)

    Article  Google Scholar 

  8. G. Avidan, U. Hasson, R. Malach, M. Behrmann, Detailed exploration of face-related processing in congenital prosopagnosia: 2. Functional neuroimaging findings. J. Cogn. Neurosci. 17, 1150–1167 (2005)

    Article  Google Scholar 

  9. V. Axelrod, G. Yovel, Hierarchical processing of face viewpoint in human visual cortex. J. Neurosci. 32, 2442–2452 (2012)

    Article  Google Scholar 

  10. V. Axelrod, G. Yovel, The challenge of localizing the anterior temporal face area: a possible solution. Neuroimage 2013(81), 371–380 (2013)

    Article  Google Scholar 

  11. M. Behrmann, G. Avidan, Congenital prosopagnosia: face-blind from birth. Trends Cogn. Sci. 9, 180–187 (2005)

    Article  Google Scholar 

  12. M. Behrmann, D.C. Plaut, Distributed circuits, not circumscribed centers, mediate visual recognition. Trends Cogn. Sci. 17, 210–219 (2013)

    Article  Google Scholar 

  13. M.G. Berman, J. Park, R. Gonzalez, T.A. Polk, A. Gehrke et al., Evaluating functional localizers: the case of the FFA. Neuroimage 50, 56–71 (2010)

    Article  Google Scholar 

  14. V. Bruce, A. Young, Understanding face recognition. Br. J. Psychol. 77(Pt 3), 305–327 (1986)

    Article  Google Scholar 

  15. L. Bugatus, K.S. Weiner, K. Grill-Spector, Task differentially modulates the spatial extent of category-selective regions across anatomical locations. Neuroimage (2017) (in press)

    Google Scholar 

  16. C.F. Cadieu, H. Hong, D.L. Yamins, N. Pinto, D. Ardila et al., Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput. Biol. 10, e1003963 (2014)

    Article  Google Scholar 

  17. A.J. Calder, A.W. Young, Understanding the recognition of facial identity and facial expression. Nat. Rev. Neurosci. 6, 641–651 (2005)

    Article  Google Scholar 

  18. A.J. Calder, J.D. Beaver, J.S. Winston, R.J. Dolan, R. Jenkins et al., Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr. Biol. 17, 20–25 (2007)

    Article  Google Scholar 

  19. T. Carlson, H. Hogendoorn, H. Fonteijn, F.A. Verstraten, Spatial coding and invariance in object-selective cortex. Cortex 47, 14–22 (2011)

    Article  Google Scholar 

  20. J. Caspers, K. Zilles, S.B. Eickhoff, A. Schleicher, H. Mohlberg, K. Amunts, Cytoarchitectonical analysis and probabilistic mapping of two extrastriate areas of the human posterior fusiform gyrus. Brain Struct. Funct. 218, 511–526 (2013)

    Article  Google Scholar 

  21. J.A. Collins, I.R. Olson, Beyond the FFA: the role of the ventral anterior temporal lobes in face processing. Neuropsychologia 61, 65–79 (2014)

    Article  Google Scholar 

  22. A.C. Connolly, J.S. Guntupalli, J. Gors, M. Hanke, Y.O. Halchenko et al., The representation of biological classes in the human brain. J. Neurosci. 32, 2608–2618 (2012)

    Article  Google Scholar 

  23. T. Cukur, A.G. Huth, S. Nishimoto, J.L. Gallant, Functional subdomains within human FFA. J. Neurosci. 33, 16748–16766 (2013)

    Article  Google Scholar 

  24. N. Davidenko, D.A. Remus, K. Grill-Spector, Face-likeness and image variability drive responses in human face-selective ventral regions. Hum. Brain Mapp. 33, 2234–2249 (2012)

    Article  Google Scholar 

  25. I. Davidesco, M. Harel, M. Ramot, U. Kramer, S. Kipervasser et al., Spatial and object-based attention modulates broadband high-frequency responses across the human visual cortical hierarchy. J. Neurosci. 33, 1228–1240 (2013)

    Article  Google Scholar 

  26. I. Davidesco, E. Zion-Golumbic, S. Bickel, M. Harel, D.M. Groppe et al., Exemplar selectivity reflects perceptual similarities in the human fusiform cortex. Cereb. Cortex 24, 1879–1893 (2014)

    Article  Google Scholar 

  27. B. de Haas, D.S. Schwarzkopf, I. Alvarez, R.P. Lawson, L. Henriksson et al., Perception and processing of faces in the human brain is tuned to typical feature locations. J. Neurosci. 36, 9289–9302 (2016)

    Article  Google Scholar 

  28. B. Duchaine, K. Nakayama, The Cambridge face memory test: results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia 44, 576–585 (2006)

    Article  Google Scholar 

  29. B. Duchaine, G. Yovel, A revised neural framework for face processing. Annu. Rev. Vis. Sci 1, 393–416 (2015)

    Article  Google Scholar 

  30. B.C. Duchaine, K. Nakayama, Developmental prosopagnosia: a window to content-specific face processing. Curr. Opin. Neurobiol. 16, 166–173 (2006)

    Article  Google Scholar 

  31. S.O. Dumoulin, B.A. Wandell, Population receptive field estimates in human visual cortex. Neuroimage 39, 647–660 (2008)

    Article  Google Scholar 

  32. T. Egner, J.M. Monti, C. Summerfield, Expectation and surprise determine neural population responses in the ventral visual stream. J. Neurosci. 30, 16601–16608 (2010)

    Article  Google Scholar 

  33. M. Eickenberg, A. Gramfort, G. Varoquaux, B. Thirion, Seeing it all: convolutional network layers map the function of the human visual system. Neuroimage (2016)

    Google Scholar 

  34. M.P. Ewbank, T.J. Andrews, Differential sensitivity for viewpoint between familiar and unfamiliar faces in human visual cortex. Neuroimage 40, 1857–1870 (2008)

    Article  Google Scholar 

  35. F. Fang, S. He, Cortical responses to invisible objects in the human dorsal and ventral pathways. Nat. Neurosci. 8, 1380–1385 (2005)

    Article  Google Scholar 

  36. R. Farivar, O. Blanke, A. Chaudhuri, Dorsal-ventral integration in the recognition of motion-defined unfamiliar faces. J. Neurosci. 29, 5336–5342 (2009)

    Article  Google Scholar 

  37. D.J. Felleman, D.C. Van Essen, Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991)

    Article  Google Scholar 

  38. B. Fischl, M.I. Sereno, R.B. Tootell, A.M. Dale, High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272–284 (1999)

    Article  Google Scholar 

  39. W. Freiwald, B. Duchaine, G. Yovel, Face processing systems: from neurons to real-world social perception. Annu. Rev. Neurosci. 39, 325–346 (2016)

    Article  Google Scholar 

  40. M.A. Frost, R. Goebel, Measuring structural-functional correspondence: spatial variability of specialised brain regions after macro-anatomical alignment. Neuroimage 59, 1369–1381 (2012)

    Article  Google Scholar 

  41. K. Fukushima, Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw. 1, 119–130 (1982)

    Article  Google Scholar 

  42. I. Gauthier, P. Skudlarski, J.C. Gore, A.W. Anderson, Expertise for cars and birds recruits brain areas involved in face recognition. Nat. Neurosci. 3, 191–197 (2000)

    Article  Google Scholar 

  43. S. Gilaie-Dotan, R. Malach, Sub-exemplar shape tuning in human face-related areas. Cereb. Cortex 17, 325–338 (2007)

    Article  Google Scholar 

  44. S. Gilaie-Dotan, H. Gelbard-Sagiv, R. Malach, Perceptual shape sensitivity to upright and inverted faces is reflected in neuronal adaptation. Neuroimage 50, 383–395 (2010)

    Article  Google Scholar 

  45. J. Gomez, F. Pestilli, N. Witthoft, G. Golarai, A. Liberman et al., Functionally defined white matter reveals segregated pathways in human ventral temporal cortex associated with category-specific processing. Neuron 85, 216–227 (2015)

    Article  Google Scholar 

  46. C. Gratton, K.K. Sreenivasan, M.A. Silver, M. D’Esposito, Attention selectively modifies the representation of individual faces in the human brain. J. Neurosci. 33, 6979–6989 (2013)

    Article  Google Scholar 

  47. K. Grill-Spector, R. Malach, fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol. (Amst) 107, 293–321 (2001)

    Article  Google Scholar 

  48. K. Grill-Spector, K.S. Weiner, The functional architecture of the ventral temporal cortex and its role in categorization. Nat. Rev. Neurosci. 15, 536–548 (2014)

    Article  Google Scholar 

  49. K. Grill-Spector, T. Kushnir, S. Edelman, G. Avidan, Y. Itzchak, R. Malach, Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 24, 187–203 (1999)

    Article  Google Scholar 

  50. K. Grill-Spector, N. Knouf, N. Kanwisher, The fusiform face area subserves face perception, not generic within-category identification. Nat. Neurosci. 7, 555–562 (2004)

    Article  Google Scholar 

  51. K. Grill-Spector, R. Henson, A. Martin, Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23 (2006)

    Article  Google Scholar 

  52. C.G. Gross, J. Sergent, Face recognition. Curr. Opin. Neurobiol. 2, 156–161 (1992)

    Article  Google Scholar 

  53. C.G. Gross, D.B. Bender, C.E. Rocha-Miranda, Visual receptive fields of neurons in inferotemporal cortex of the monkey. Science 166, 1303–1306 (1969)

    Article  Google Scholar 

  54. M. Gschwind, G. Pourtois, S. Schwartz, D. Van De Ville, P. Vuilleumier, White-matter connectivity between face-responsive regions in the human brain. Cereb. Cortex 22, 1564–1576 (2012)

    Article  Google Scholar 

  55. U. Guclu, M.A. van Gerven, Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015)

    Article  Google Scholar 

  56. R.J. Harris, G.E. Rice, A.W. Young, T.J. Andrews, Distinct but overlapping patterns of response to words and faces in the fusiform gyrus. Cereb. Cortex 26, 3161–3168 (2016)

    Article  Google Scholar 

  57. U. Hasson, I. Levy, M. Behrmann, T. Hendler, R. Malach, Eccentricity bias as an organizing principle for human high-order object areas. Neuron 34, 479–490 (2002)

    Article  Google Scholar 

  58. J.V. Haxby, E.A. Hoffman, M.I. Gobbini, The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233 (2000)

    Article  Google Scholar 

  59. J.V. Haxby, M.I. Gobbini, M.L. Furey, A. Ishai, J.L. Schouten, P. Pietrini, Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001)

    Article  Google Scholar 

  60. C.C. Hemond, N.G. Kanwisher, H.P. Op de Beeck, A preference for contralateral stimuli in human object- and face-selective cortex. PLoS One 2, e574 (2007)

    Article  Google Scholar 

  61. L. Henriksson, M. Mur, N. Kriegeskorte, Faciotopy-A face-feature map with face-like topology in the human occipital face area. Cortex 72, 156–167 (2015)

    Article  Google Scholar 

  62. G. Holmes, Disturbances of vision by cerebral lesions. Br. J. Ophthalmol. 2, 353–384 (1918)

    Article  Google Scholar 

  63. D.H. Hubel, T.N. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154 (1962)

    Article  Google Scholar 

  64. J.B. Hutchinson, M.R. Uncapher, K.S. Weiner, D.W. Bressler, M.A. Silver et al., Functional heterogeneity in posterior parietal cortex across attention and episodic memory retrieval. Cereb. Cortex 24, 49–66 (2012)

    Article  Google Scholar 

  65. A. Ishai, L.G. Ungerleider, A. Martin, J.V. Haxby, The representation of objects in the human occipital and temporal cortex. J. Cogn. Neurosci. 12(Suppl 2), 35–51 (2000)

    Article  Google Scholar 

  66. C. Jacques, N. Witthoft, K.S. Weiner, B.L. Foster, V. Rangarajan et al., Corresponding ECoG and fMRI category-selective signals in human ventral temporal cortex. Neuropsychologia 83, 14–28 (2016)

    Article  Google Scholar 

  67. X. Jiang, E. Rosen, T. Zeffiro, J. Vanmeter, V. Blanz, M. Riesenhuber, Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques. Neuron 50, 159–172 (2006)

    Article  Google Scholar 

  68. J. Jonas, S. Frismand, J.P. Vignal, S. Colnat-Coulbois, L. Koessler et al., Right hemispheric dominance of visual phenomena evoked by intracerebral stimulation of the human visual cortex. Hum. Brain Mapp. 35, 3360–3371 (2014)

    Article  Google Scholar 

  69. J. Jonas, B. Rossion, J. Krieg, L. Koessler, S. Colnat-Coulbois et al., Intracerebral electrical stimulation of a face-selective area in the right inferior occipital cortex impairs individual face discrimination. Neuroimage 99, 487–497 (2014)

    Article  Google Scholar 

  70. J. Jonas, C. Jacques, J. Liu-Shuang, H. Brissart, S. Colnat-Coulbois et al., A face-selective ventral occipito-temporal map of the human brain with intracerebral potentials. Proc. Natl. Acad. Sci. USA 113, E4088–E4097 (2016)

    Article  Google Scholar 

  71. N. Kanwisher, Domain specificity in face perception. Nat. Neurosci. 3, 759–763 (2000)

    Article  Google Scholar 

  72. N. Kanwisher, J. McDermott, M.M. Chun, The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302–4311 (1997)

    Article  Google Scholar 

  73. N. Kanwisher, F. Tong, K. Nakayama, The effect of face inversion on the human fusiform face area. Cognition 68, B1–B11 (1998)

    Article  Google Scholar 

  74. K.N. Kay, J.D. Yeatman, Bottom-up and top-down computations in high-level visual cortex. Elife. 2017 Feb 22;6. pii: e22341 (2017)

    Google Scholar 

  75. K.N. Kay, T. Naselaris, R.J. Prenger, J.L. Gallant, Identifying natural images from human brain activity. Nature 452, 352–355 (2008)

    Article  Google Scholar 

  76. K.N. Kay, J. Winawer, A. Mezer, B.A. Wandell, Compressive spatial summation in human visual cortex. J. Neurophysiol. 110, 481–494 (2013)

    Article  Google Scholar 

  77. K.N. Kay, K.S. Weiner, K. Grill-Spector, Attention reduces spatial uncertainty in human ventral temporal cortex. Curr. Biol. 25, 595–600 (2015)

    Article  Google Scholar 

  78. S.M. Khaligh-Razavi, N. Kriegeskorte, Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol. 10, e1003915 (2014)

    Article  Google Scholar 

  79. T.C. Kietzmann, J.D. Swisher, P. Konig, F. Tong, Prevalence of selectivity for mirror-symmetric views of faces in the ventral and dorsal visual pathways. J. Neurosci. 32, 11763–11772 (2012)

    Article  Google Scholar 

  80. M. Kim, M. Ducros, T. Carlson, I. Ronen, S. He et al., Anatomical correlates of the functional organization in the human occipitotemporal cortex. Magn. Reson. Imaging 24, 583–590 (2006)

    Article  Google Scholar 

  81. B.P. Klein, B.M. Harvey, S.O. Dumoulin, Attraction of position preference by spatial attention throughout human visual cortex. Neuron 84, 227–237 (2014)

    Article  Google Scholar 

  82. J. Konorski, Integrative Activity of the Brain. An Interdisciplinary Approach (The University of Chicago Press, Chicago, 1967)

    Google Scholar 

  83. N. Kriegeskorte, Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu. Rev. Vis. Sci. 1, 417–446 (2015)

    Article  Google Scholar 

  84. N. Kriegeskorte, E. Formisano, B. Sorger, R. Goebel, Individual faces elicit distinct response patterns in human anterior temporal cortex. Proc. Natl. Acad. Sci. USA 104, 20600–20605 (2007)

    Article  Google Scholar 

  85. A. Krizhevsky, I. Sutskever, G. Hinton, Imagenet classification with deep convolutional neural networks. Presented at Neural Information Processing Systems (NIPS) (2012)

    Google Scholar 

  86. J. Kubilius, S. Bracci, H.P. Op de Beeck, Deep neural networks as a computational model for human shape sensitivity. PLoS Comput. Biol. 12, e1004896 (2016)

    Article  Google Scholar 

  87. Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard et al., Neural Information Processing (1989)

    Google Scholar 

  88. Y. Lee, B. Duchaine, H.R. Wilson, K. Nakayama, Three cases of developmental prosopagnosia from one family: detailed neuropsychological and psychophysical investigation of face processing. Cortex 46, 949–964 (2010)

    Article  Google Scholar 

  89. I. Levy, U. Hasson, G. Avidan, T. Hendler, R. Malach, Center-periphery organization of human object areas. Nat. Neurosci. 4, 533–539 (2001)

    Article  Google Scholar 

  90. G. Loffler, G. Yourganov, F. Wilkinson, H.R. Wilson, fMRI evidence for the neural representation of faces. Nat. Neurosci. 8, 1386–1390 (2005)

    Article  Google Scholar 

  91. G.R. Loftus, E.M. Harley, Why is it easier to identify someone close than far away? Psychon. Bull. Rev. 12, 43–65 (2005)

    Article  Google Scholar 

  92. S. Lorenz, K.S. Weiner, J. Caspers, H. Mohlberg, A. Schleicher et al., Two new cytoarchitectonic areas on the human mid-fusiform gyrus. Cereb. Cortex (2015)

    Google Scholar 

  93. A. Martin, C.L. Wiggs, L.G. Ungerleider, J.V. Haxby, Neural correlates of category-specific knowledge. Nature 379, 649–652 (1996)

    Article  Google Scholar 

  94. G. McCarthy, A. Puce, J.C. Gore, T. Allison, Face-specific processing in the human fusiform gyrus. J. Cogn. Neurosci. 9, 605–610 (1997)

    Article  Google Scholar 

  95. G. McCarthy, A. Puce, A. Belger, T. Allison, Electrophysiological studies of human face perception. II: response properties of face-specific potentials generated in occipitotemporal cortex. Cereb. Cortex 9, 431–444 (1999)

    Article  Google Scholar 

  96. E. McKone, Holistic processing for faces operates over a wide range of sizes but is strongest at identification rather than conversational distances. Vis. Res. 49, 268–283 (2009)

    Article  Google Scholar 

  97. K. Moutoussis, S. Zeki, The relationship between cortical activation and perception investigated with invisible stimuli. Proc. Natl. Acad. Sci. USA 99, 9527–9532 (2002)

    Article  Google Scholar 

  98. M. Mur, D.A. Ruff, J. Bodurka, P. De Weerd, P.A. Bandettini, N. Kriegeskorte, Categorical, yet graded-single-image activation profiles of human category-selective cortical regions. J. Neurosci. 32, 8649–8662 (2012)

    Article  Google Scholar 

  99. S. Nasr, N. Liu, K.J. Devaney, X. Yue, R. Rajimehr et al., Scene-selective cortical regions in human and nonhuman primates. J. Neurosci. 31, 13771–13785 (2011)

    Article  Google Scholar 

  100. V. Natu, M.A. Barnett, T. Hartley, J. Gomez, A. Stigliani, K. Grill-Spector, Development of neural sensitivity to face identity correlates with perceptual discriminability. J. Neurosci. 36, 10893–10907 (2016)

    Article  Google Scholar 

  101. V.S. Natu, A.J. O’Toole, Spatiotemporal changes in neural response patterns to faces varying in visual familiarity. Neuroimage 108, 151–159 (2015)

    Article  Google Scholar 

  102. V.S. Natu, F. Jiang, A. Narvekar, S. Keshvari, V. Blanz, A.J. O’Toole, Dissociable neural patterns of facial identity across changes in viewpoint. J. Cogn. Neurosci. 22, 1570–1582 (2010)

    Article  Google Scholar 

  103. A. Nestor, D.C. Plaut, M. Behrmann, Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis. Proc. Natl. Acad. Sci. USA 108, 9998–10003 (2011)

    Article  Google Scholar 

  104. K.M. O’Craven, P.E. Downing, N. Kanwisher, fMRI evidence for objects as the units of attentional selection. Nature 401, 584–587 (1999)

    Article  Google Scholar 

  105. J. Parvizi, C. Jacques, B.L. Foster, N. Withoft, V. Rangarajan et al., Electrical stimulation of human fusiform face-selective regions distorts face perception. J. Neurosci. 32, 14915–14920 (2012)

    Article  Google Scholar 

  106. M.V. Peelen, P.E. Downing, Within-subject reproducibility of category-specific visual activation with functional MRI. Hum. Brain Mapp. 25, 402–408 (2005)

    Article  Google Scholar 

  107. K.A. Pelphrey, N.J. Sasson, J.S. Reznick, G. Paul, B.D. Goldman, J. Piven, Visual scanning of faces in autism. J Autism Dev. Disord. 32, 249–261 (2002)

    Article  Google Scholar 

  108. M.A. Pinsk, M. Arcaro, K.S. Weiner, J.F. Kalkus, S.J. Inati et al., Neural representations of faces and body parts in macaque and human cortex: a comparative FMRI study. J. Neurophysiol. 101, 2581–2600 (2009)

    Article  Google Scholar 

  109. D. Pitcher, V. Walsh, G. Yovel, B. Duchaine, TMS evidence for the involvement of the right occipital face area in early face processing. Curr. Biol. 17, 1568–1573 (2007)

    Article  Google Scholar 

  110. D. Pitcher, D.D. Dilks, R.R. Saxe, C. Triantafyllou, N. Kanwisher, Differential selectivity for dynamic versus static information in face-selective cortical regions. Neuroimage 56, 2356–2363 (2011)

    Article  Google Scholar 

  111. D. Pitcher, V. Walsh, B. Duchaine, The role of the occipital face area in the cortical face perception network. Exp. Brain Res. 209, 481–493 (2011)

    Article  Google Scholar 

  112. D. Pitcher, T. Goldhaber, B. Duchaine, V. Walsh, N. Kanwisher, Two critical and functionally distinct stages of face and body perception. J. Neurosci. 32, 15877–15885 (2012)

    Article  Google Scholar 

  113. E. Privman, Y. Nir, U. Kramer, S. Kipervasser, F. Andelman et al., Enhanced category tuning revealed by intracranial electroencephalograms in high-order human visual areas. J. Neurosci. 27, 6234–6242 (2007)

    Article  Google Scholar 

  114. A. Puce, T. Allison, J.C. Gore, G. McCarthy, Face-sensitive regions in human extrastriate cortex studied by functional MRI. J. Neurophysiol. 74, 1192–1199 (1995)

    Article  Google Scholar 

  115. A. Puce, T. Allison, S. Bentin, J.C. Gore, G. McCarthy, Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188–2199 (1998)

    Article  Google Scholar 

  116. A. Puce, T. Allison, G. McCarthy, Electrophysiological studies of human face perception. III: effects of top-down processing on face-specific potentials. Cereb. Cortex 9, 445–458 (1999)

    Article  Google Scholar 

  117. J.A. Pyles, T.D. Verstynen, W. Schneider, M.J. Tarr, Explicating the face perception network with white matter connectivity. PLoS One 8, e61611 (2013)

    Article  Google Scholar 

  118. R. Rajimehr, J.C. Young, R.B. Tootell, An anterior temporal face patch in human cortex, predicted by macaque maps. Proc. Natl. Acad. Sci. USA 106, 1995–2000 (2009)

    Article  Google Scholar 

  119. V. Rangarajan, D. Hermes, B.L. Foster, K.S. Weiner, C. Jacques et al., Electrical stimulation of the left and right human fusiform gyrus causes different effects in conscious face perception. J. Neurosci. 34, 12828–12836 (2014)

    Article  Google Scholar 

  120. J.H. Reynolds, D.J. Heeger, The normalization model of attention. Neuron 61, 168–185 (2009)

    Article  Google Scholar 

  121. M. Riesenhuber, T. Poggio, Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999)

    Article  Google Scholar 

  122. M. Rosenke, K.S. Weiner, M.A. Barnett, K. Zilles, K. Amunts, R. Goebel, K. Grill-Spector, A cross-validated cytoarchitectonic atlas of the human ventral visual stream. NeuroImage pii: S1053–8119(17), 30151–30159 (2017)

    Google Scholar 

  123. B. Rossion, Constraining the cortical face network by neuroimaging studies of acquired prosopagnosia. Neuroimage 40, 423–426 (2008)

    Article  Google Scholar 

  124. B. Rossion, A. Boremanse, Robust sensitivity to facial identity in the right human occipito-temporal cortex as revealed by steady-state visual-evoked potentials. J. Vis. 11, 1–18 (2011)

    Article  Google Scholar 

  125. B. Rossion, R. Caldara, M. Seghier, A.M. Schuller, F. Lazeyras, E. Mayer, A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. Brain 126, 2381–2395 (2003)

    Article  Google Scholar 

  126. P. Rotshtein, R.N. Henson, A. Treves, J. Driver, R.J. Dolan, Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain. Nat. Neurosci. 8, 107–113 (2005)

    Article  Google Scholar 

  127. D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Cognitive Modeling (1988)

    Google Scholar 

  128. Z.M. Saygin, D.E. Osher, K. Koldewyn, G. Reynolds, J.D. Gabrieli, R.R. Saxe, Anatomical connectivity patterns predict face selectivity in the fusiform gyrus. Nat. Neurosci. 15, 321–327 (2012)

    Article  Google Scholar 

  129. C. Schiltz, B. Sorger, R. Caldara, F. Ahmed, E. Mayer et al., Impaired face discrimination in acquired prosopagnosia is associated with abnormal response to individual faces in the right middle fusiform gyrus. Cereb. Cortex 16, 574–586 (2006)

    Article  Google Scholar 

  130. C. Schiltz, L. Dricot, R. Goebel, B. Rossion, Holistic perception of individual faces in the right middle fusiform gyrus as evidenced by the composite face illusion. J. Vis. 10(25), 1–16 (2010)

    Article  Google Scholar 

  131. F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: a unified embedding for face recognition and clustering. in IEEE Conference on Computer Vision and Pattern Recognition (CVPR, 2015) (2015), pp. 815–823

    Google Scholar 

  132. R.F. Schwarzlose, J.D. Swisher, S. Dang, N. Kanwisher, The distribution of category and location information across object-selective regions in human visual cortex. Proc. Natl. Acad. Sci. USA 105, 4447–4452 (2008)

    Article  Google Scholar 

  133. M.I. Sereno, A.M. Dale, J.B. Reppas, K.K. Kwong, J.W. Belliveau et al., Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268, 889–893 (1995)

    Article  Google Scholar 

  134. J. Sergent, J.L. Signoret, Functional and anatomical decomposition of face processing: evidence from prosopagnosia and PET study of normal subjects. Philos. Trans. R. Soc. Lond. B Biol. Sci. 335, 55–61; discussion 61-2

    Google Scholar 

  135. J. Sergent, S. Ohta, B. MacDonald, Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain 115(Pt 1), 15–36 (1992)

    Article  Google Scholar 

  136. T. Serre, A. Oliva, T. Poggio, A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. USA 104, 6424–6429 (2007)

    Article  Google Scholar 

  137. M.A. Silver, S. Kastner, Topographic maps in human frontal and parietal cortex. Trends Cogn. Sci. 13, 488–495 (2009)

    Article  Google Scholar 

  138. M.A. Silver, D. Ress, D.J. Heeger, Topographic maps of visual spatial attention in human parietal cortex. J. Neurophysiol. 94, 1358–1371 (2005)

    Article  Google Scholar 

  139. H.P. Snippe, J.J. Koenderink, Information in channel-coded systems: correlated receivers. Biol. Cybern. 67, 183–190 (1992)

    Article  MATH  Google Scholar 

  140. B. Sorger, R. Goebel, C. Schiltz, B. Rossion, Understanding the functional neuroanatomy of acquired prosopagnosia. Neuroimage 35, 836–852 (2007)

    Article  Google Scholar 

  141. T.C. Sprague, J.T. Serences, Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices. Nat. Neurosci. 16, 1879–1887 (2013)

    Article  Google Scholar 

  142. J.K. Steeves, J.C. Culham, B.C. Duchaine, C.C. Pratesi, K.F. Valyear et al., The fusiform face area is not sufficient for face recognition: evidence from a patient with dense prosopagnosia and no occipital face area. Neuropsychologia 44, 594–609 (2006)

    Article  Google Scholar 

  143. C. Summerfield, E.H. Trittschuh, J.M. Monti, M.M. Mesulam, T. Egner, Neural repetition suppression reflects fulfilled perceptual expectations. Nat. Neurosci. (2008)

    Google Scholar 

  144. J.D. Swisher, M.A. Halko, L.B. Merabet, S.A. McMains, D.C. Somers, Visual topography of human intraparietal sulcus. J. Neurosci. 27, 5326–5337 (2007)

    Article  Google Scholar 

  145. Y. Taigman, M. Yang, M. Ranzato, L. Wolf, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR, 2014), pp. 1701–1708

    Google Scholar 

  146. H. Takemura, A. Rokem, J. Winawer, J.D. Yeatman, B.A. Wandell, F. Pestilli, A major human white matter pathway between dorsal and ventral visual cortex. Cereb. Cortex 26, 2205–2214 (2016)

    Article  Google Scholar 

  147. T. Tallinen, J.Y. Chung, J.S. Biggins, L. Mahadevan, Gyrification from constrained cortical expansion. Proc. Natl. Acad. Sci. USA 111, 12667–12672 (2014)

    Article  Google Scholar 

  148. J.W. Tanaka, M.J. Farah, Parts and wholes in face recognition. Q. J. Exp. Psychol. A Hum. Exp. Psychol. 46, 225–245 (1993)

    Article  Google Scholar 

  149. I. Tavor, M. Yablonski, A. Mezer, S. Rom, Y. Assaf, G. Yovel, Separate parts of occipito-temporal white matter fibers are associated with recognition of faces and places. Neuroimage (2013)

    Google Scholar 

  150. F. Tong, K. Nakayama, J.T. Vaughan, N. Kanwisher, Binocular rivalry and visual awareness in human extrastriate cortex. Neuron 21, 753–759 (1998)

    Article  Google Scholar 

  151. F. Tong, K. Nakayama, M. Moscovitch, O. Weinrib, N. Kanwisher, Response properties of the human fusiform face area. Cogn. Neuropsychol. 17, 257–280 (2000)

    Article  Google Scholar 

  152. D. Tsao, M. Livingstone, Mechanisms of face perception. Annu. Rev. Neurosci. 31, 411–437 (2008)

    Article  Google Scholar 

  153. D.Y. Tsao, S. Moeller, W.A. Freiwald, Comparing face patch systems in macaques and humans. Proc. Natl. Acad. Sci. USA 105, 19514–19519 (2008)

    Article  Google Scholar 

  154. T. Valentine, Face-space models of face recognition, in Computational, Geometric, and Process Perspectives on Facial Cognition: Contexts and Challenges, ed. by M.J. Wenger, J.T. Townsend (Lawrence Erlbaum Associates Inc, Hillsdale, 2001)

    Google Scholar 

  155. G. Van Belle, P. De Graef, K. Verfaillie, T. Busigny, B. Rossion, Whole not hole: expert face recognition requires holistic perception. Neuropsychologia 48, 2620–2629 (2010)

    Article  Google Scholar 

  156. G. Van Belle, T. Busigny, P. Lefevre, S. Joubert, O. Felician et al., Impairment of holistic face perception following right occipito-temporal damage in prosopagnosia: converging evidence from gaze-contingency. Neuropsychologia 49, 3145–3150 (2011)

    Article  Google Scholar 

  157. D.C. Van Essen, J.L. Gallant, Neural mechanisms of form and motion processing in the primate visual system. Neuron 13, 1–10 (1994)

    Article  Google Scholar 

  158. D.C. Van Essen, C.H. Anderson, D.J. Felleman, Information processing in the primate visual system: an integrated systems perspective. Science 255, 419–423 (1992)

    Article  Google Scholar 

  159. P. Vuilleumier, R.N. Henson, J. Driver, R.J. Dolan, Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming. Nat. Neurosci. 5, 491–499 (2002)

    Article  Google Scholar 

  160. P. Vuilleumier, J.L. Armony, J. Driver, R.J. Dolan, Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nat. Neurosci. 6, 624–631 (2003)

    Article  Google Scholar 

  161. B.A. Wandell, J. Winawer, Imaging retinotopic maps in the human brain. Vis. Res. 51, 718–737 (2011)

    Article  Google Scholar 

  162. B.A. Wandell, J. Winawer, Computational neuroimaging and population receptive fields. Trends Cogn. Sci. 19, 349–357 (2015)

    Article  Google Scholar 

  163. B.A. Wandell, S.O. Dumoulin, A.A. Brewer, Visual field maps in human cortex. Neuron 56, 366–383 (2007)

    Article  Google Scholar 

  164. K. Weibert, T.J. Andrews, Activity in the right fusiform face area predicts the behavioural advantage for the perception of familiar faces. Neuropsychologia 75, 588–596 (2015)

    Article  Google Scholar 

  165. K.S. Weiner, K. Grill-Spector, Sparsely-distributed organization of face and limb activations in human ventral temporal cortex. Neuroimage 52, 1559–1573 (2010)

    Article  Google Scholar 

  166. K.S. Weiner, K. Grill-Spector, The improbable simplicity of the fusiform face area. Trends Cogn. Sci. 16(5), 251–4 (2012)

    Article  Google Scholar 

  167. K.S. Weiner, K. Grill-Spector, The evolution of face processing networks. Trends Cogn. Sci. 19, 240–241 (2015)

    Article  Google Scholar 

  168. K.S. Weiner, K. Zilles, The anatomical and functional specialization of the fusiform gyrus. Neuropsychologia 83, 48–62 (2016)

    Article  Google Scholar 

  169. K.S. Weiner, R. Sayres, J. Vinberg, K. Grill-Spector, fMRI-adaptation and category selectivity in human ventral temporal cortex: regional differences across time scales. J. Neurophysiol. 103, 3349–3365 (2010)

    Article  Google Scholar 

  170. K.S. Weiner, G. Golarai, J. Caspers, M.R. Chuapoco, H. Mohlberg et al., The mid-fusiform sulcus: a landmark identifying both cytoarchitectonic and functional divisions of human ventral temporal cortex. Neuroimage 84, 453–465 (2014)

    Article  Google Scholar 

  171. K.S. Weiner, J. Jonas, J. Gomez, L. Maillard, H. Brissart et al., The face-processing network is resilient to focal resection of human visual cortex. J. Neurosci. 36, 8425–8440 (2016)

    Article  Google Scholar 

  172. K.S. Weiner, J.D. Yeatman, B.A. Wandell, The posterior arcuate fasciculus and the vertical occipital fasciculus. Cortex. 31 Mar 2016. pii: (16)30050-8 (2016). doi:10.1016/S0010-9452

  173. K.S. Weiner, B.A. Barnett, S. Lorenz, J. Caspers, A. Stigliani et al., The cytoarchitecture of domain-specific regions in human high-level visual cortex. Cereb. Cortex (2017)

    Google Scholar 

  174. Y. Weiss, S. Edelman, M. Fahle, Models of perceptual learning in vernier hyperacuity. Neural Comput. 5, 695–718 (1993)

    Article  Google Scholar 

  175. J.S. Winston, R.N. Henson, M.R. Fine-Goulden, R.J. Dolan, fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. J. Neurophysiol. 92, 1830–1839 (2004)

    Article  Google Scholar 

  176. N. Witthoft, S. Poltoratski, M. Nguyen, G. Golarai, A. Liberman et al., Developmental prosopagnosia is associated with reduced spatial integration in the ventral visual cortex, in bioRxiv (2016)

    Google Scholar 

  177. D.L. Yamins, J.J. DiCarlo, Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356–365 (2016)

    Article  Google Scholar 

  178. D.L. Yamins, H. Hong, C.F. Cadieu, E.A. Solomon, D. Seibert, J.J. DiCarlo, Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. USA 111, 8619–8624 (2014)

    Article  Google Scholar 

  179. J.D. Yeatman, K.S. Weiner, F. Pestilli, A. Rokem, A. Mezer, B.A. Wandell, The vertical occipital fasciculus: a century of controversy resolved by in vivo measurements. Proc. Natl. Acad. Sci. USA 111, E5214–E5223 (2014)

    Article  Google Scholar 

  180. D.J. Yi, T.A. Kelley, R. Marois, M.M. Chun, Attentional modulation of repetition attenuation is anatomically dissociable for scenes and faces. Brain Res. 1080, 53–62 (2006)

    Article  Google Scholar 

  181. G. Yovel, N. Kanwisher, Face perception: domain specific, not process specific. Neuron 44, 889–898 (2004)

    Google Scholar 

  182. G. Yovel, N. Kanwisher, The neural basis of the behavioral face-inversion effect. Curr. Biol. 15, 2256–2262 (2005)

    Article  Google Scholar 

  183. X. Yue, B.S. Cassidy, K.J. Devaney, D.J. Holt, R.B. Tootell, Lower-level stimulus features strongly influence responses in the fusiform face area. Cereb. Cortex 21, 35–47 (2011)

    Article  Google Scholar 

  184. S. Zeki, S. Shipp, The functional logic of cortical connections. Nature 335, 311–317 (1988)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Jesse Gomez for comments and edits of an earlier draft. This research has been funded by the following grants: NSF GRFP DGE-114747, 1RO1EY02231801-A1, and 1R01EY02391501-A1. Portions of this chapter were previously included within a paper published in the Annual Reviews of Vision Science, 2017 (http://www.annualreviews.org/doi/10.1146/annurev-vision-102016-061214).

Author information

Authors and Affiliations

  1. Department of Psychology, Stanford University, Stanford, CA, USA

    Kalanit Grill-Spector & Kevin S. Weiner

  2. Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA

    Kalanit Grill-Spector

  3. Department of Radiology, University of Minnesota, Minneapolis, MN, USA

    Kendrick Kay

Authors
  1. Kalanit Grill-Spector
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kendrick Kay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kevin S. Weiner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kalanit Grill-Spector .

Editor information

Editors and Affiliations

  1. University of California, Riverside, California, USA

    Bir Bhanu

  2. Hong Kong Polytechnic University, Hong Kong, China

    Ajay Kumar

Appendix: Abbreviations and Definitions

Appendix: Abbreviations and Definitions

Collateral sulcus (CoS) : a primary sulcus in human ventral temporal cortex; the medial boundary of the fusiform gyrus

Cytoarchitecture : cellular organization across the six-layered cortical ribbon; a property used to parcellate brain areas from one another

Developmental prosopagnosia (DP) : an impairment in recognizing faces despite normal vision, intelligence, and socio-cognitive abilities and no history of brain damage

Fusiform face area (FFA) : once considered a homogenous face-selective area, it contains (at least) two cytoarchitectonically and functionally distinct components

Fusiform gyrus (FG) : a hominoid-specific macroanatomical structure in ventral temporal cortex that contains (at least) four cytoarchitectonic areas and multiple functional regions

FG1–4 : Labels for four cytoarchitectonic areas in the fusiform gyrus and neighboring sulci

Inferior occipital gyrus (IOG) : a gyrus that is posterior-lateral to the fusiform gyrus; considered the first processing stage of the ventral face network

Mid-fusiform sulcus (MFS) : a shallow, longitudinal sulcus bisecting the fusiform gyrus; a landmark identifying cytoarchitectonic and functional boundaries

mFus-faces/FFA-2: a face-selective region overlapping the anterior-lateral tip of the mid-fusiform sulcus, located within cytoarchitectonic area FG4, and 1–1.5 cm anterior to pFus-faces/FFA-1

Occipito-temporal sulcus (OTS) : a primary sulcus in human ventral temporal cortex; the lateral boundary of the fusiform gyrus

Parahippocampal place area (PPA) : a place-selective region in the collateral sulcus and parahippocampal cortex

pFus-faces/FFA-1: a face-selective region typically overlapping the posterior-lateral tip of the mid-fusiform sulcus, located within cytoarchitectonic area FG2, and 1–1.5 cm posterior to mFus-faces/FFA-2

Population receptive field (pRF): in fMRI, the region of visual space that stimulates a voxel

Receptive field (RF): the region of the visual field which elicits a response from a neuron

Ventral Temporal Cortex (VTC): a cortical expanse spanning the inferior aspect of the temporal lobe containing high-level visual regions involved in “what” perception and recognition.

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Grill-Spector, K., Kay, K., Weiner, K.S. (2017). The Functional Neuroanatomy of Face Processing: Insights from Neuroimaging and Implications for Deep Learning. In: Bhanu, B., Kumar, A. (eds) Deep Learning for Biometrics. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-61657-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61657-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61656-8

  • Online ISBN: 978-3-319-61657-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation