Brain Imaging and Behavior

, Volume 12, Issue 2, pp 357–368 | Cite as

Audio-visual speech perception in adult readers with dyslexia: an fMRI study

  • Jascha Rüsseler
  • Zheng Ye
  • Ivonne Gerth
  • Gregor R. Szycik
  • Thomas F. Münte
Original Research


Developmental dyslexia is a specific deficit in reading and spelling that often persists into adulthood. In the present study, we used slow event-related fMRI and independent component analysis to identify brain networks involved in perception of audio-visual speech in a group of adult readers with dyslexia (RD) and a group of fluent readers (FR). Participants saw a video of a female speaker saying a disyllabic word. In the congruent condition, audio and video input were identical whereas in the incongruent condition, the two inputs differed. Participants had to respond to occasionally occurring animal names. The independent components analysis (ICA) identified several components that were differently modulated in FR and RD. Two of these components including fusiform gyrus and occipital gyrus showed less activation in RD compared to FR possibly indicating a deficit to extract face information that is needed to integrate auditory and visual information in natural speech perception. A further component centered on the superior temporal sulcus (STS) also exhibited less activation in RD compared to FR. This finding is corroborated in the univariate analysis that shows less activation in STS for RD compared to FR. These findings suggest a general impairment in recruitment of audiovisual processing areas in dyslexia during the perception of natural speech.


Developmental dyslexia Audio-visual processing Event-related fMRI Independent component analysis 



Supported by BMBF grants 01GJ1303A to TFM and 01AB074401 to JR.

Compliance with ethical standards

This study was funded by BMBF (grants 01GJ1303A to TFM and 01AB074401 to JR).

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.


  1. Baart, M., de Boer-Schellekens, L., & Vroomen, J. (2012). Lipread-induced phonetic recalibration in dyslexia. Acta Psychologica, 140, 91–95.CrossRefPubMedGoogle Scholar
  2. Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX lexical database [CD-ROM]. Philadelphia: University of Pennsylvania, Linguistic Data Consortium.Google Scholar
  3. Baumgart, F., Kaulisch, T., Tempelmann, C., Gachler-Markefski, B., Tegeler, C., Schindler, F., Stiller, D., & Scheich, H. (1998). Electrodynamic headphones and woofers for application in magnetic resonance imaging scanners. Medical Physics, 25, 2068–2070.CrossRefPubMedGoogle Scholar
  4. Beauchamp, M. S. (2005). See me, hear me, touch me: multisensory integration in lateral occipito-temporal cortex. Current Opinion in Neurobiology, 15, 145–153.CrossRefPubMedGoogle Scholar
  5. Bell, A. J., & Sejnowski, T. J. (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159.CrossRefPubMedGoogle Scholar
  6. Birch, H. G., & Belmont, L. (1964). Auditory-visual integration in normal and retarded readers. The American Journal of Orthopsychiatry, 34, 852–861.CrossRefPubMedGoogle Scholar
  7. Blau, V., van Atteveldt, N., Ekkebus, M., Goebel, R., & Blomert, L. (2009). Reduced neural integration of letters and speech sounds links phonological and reading deficits in adult dyslexia. Current Biology, 19, 503–508.CrossRefPubMedGoogle Scholar
  8. Blau, V., Reithler, J., van Atteveldt, N., Seitz, J., Gerretsen, P., Goebel, R., & Blomert, L. (2010). Deviant processing of letters and speech sounds as proximate cause of reading failure: A functional magnetic resonance imaging study of dyslexic children. Brain, 133, 868–879.CrossRefPubMedGoogle Scholar
  9. Brefczynski-Lewis, J., Lowitzsch, S., Parsons, M., Lemieux, S., & Puce, A. (2009). Audiovisual non-verbal dynamic faces elicit converging fMRI and ERP responses. Brain Topography, 21, 193–206.CrossRefPubMedPubMedCentralGoogle Scholar
  10. Brunswick, N., McCrory, E., Price, C. J., Frith, C. D., & Frith, U. (1999). Explicit and implicit processing of words and pseudowords by adult developmental dyslexics: A search for Wernicke's Wortschatz? Brain, 122, 1901–1917.CrossRefPubMedGoogle Scholar
  11. Calhoun, V. D., Adali, T., Pearlson, G. D., & Pekar, J. J. (2001). A method for making group inferences from functional MRI data using independent component analysis. Human Brain Mapping, 14, 140–151.CrossRefPubMedGoogle Scholar
  12. Calhoun, V. D., Maciejewski, P. K., Pearlson, G. D., & Kiehl, K. A. (2007). Temporal lobe and “default” hemodynamic brain modes discriminate between schizophrenia and bipolar disorder. Human Brain Mapping, 29, 1265–1275.CrossRefGoogle Scholar
  13. Calhoun, V. D., Kiehl, K. A., & Pearlson, G. D. (2008). Modulation of temporally coherent brain networks estimated using ICA at rest and during cognitive tasks. Human Brain Mapping, 29, 828–838.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Calhoun, V. D., Liu, J., & Adali, T. (2009). A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. NeuroImage, 45(1 Suppl), S163–S172.CrossRefPubMedGoogle Scholar
  15. Calvert, G. A., Campbell, R., & Brammer, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649–657.CrossRefPubMedGoogle Scholar
  16. Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S., & Dehaene, S. (2002). Language-specific tuning of visual cortex functional properties of the visual word form area. Brain, 125, 1054–1069.CrossRefPubMedGoogle Scholar
  17. De Gelder, B., & Vroomen, J. (1998). Impaired speech perception in poor readers: evidence from hearing and speech reading. Brain and Language, 64, 269–281.CrossRefPubMedGoogle Scholar
  18. Dehaene, S., Pegado, F., Braga, L. W., Ventura, P., Nunes Filho, G., Jobert, A., Dehaene-Lambertz, G., Kolinsky, R., Morais, J., & Cohen, L. (2010). How learning to read changes the cortical networks for vision and language. Science, 330, 1359–1364.CrossRefPubMedGoogle Scholar
  19. Froyen, D., Van Atteveldt, N., Bonte, M., & Blomert, L. (2008). Cross-modal enhancement of the MMN to speech-sounds indicates early and automatic integration of letters and speech-sounds. Neuroscience Letters, 430, 23–28.CrossRefPubMedGoogle Scholar
  20. Froyen, D. J., Bonte, M. L., van Atteveldt, N., & Blomert, L. (2009). The long road to automation: neurocognitive development of letter-speech sound processing. Journal of Cognitive Neuroscience, 21, 567–580.CrossRefPubMedGoogle Scholar
  21. Froyen, D., Willems, G., & Blomert, L. (2011). Evidence for a specific cross-modal association deficit in dyslexia: an electrophysiological study of letter-speech sound processing. Developmental Science, 14, 635–648.CrossRefPubMedGoogle Scholar
  22. Goswami, U. (2000). Phonological representations, reading development and dyslexia: towards a cross-linguistic theoretical framework. Dyslexia, 6, 133–151.CrossRefPubMedGoogle Scholar
  23. Habib, M., & Giraud, M. (2013). Dyslexia. Handbook of Clinical Neurology, 111, 229–235.CrossRefPubMedGoogle Scholar
  24. Hahn, N., Foxe, J. J., & Molholm, S. (2014). Impairments of multisensory integration and cross-sensory learning as pathways to dyslexia. Neuroscience & Biobehavioral Reviews, 47, 384–392.Google Scholar
  25. Hayes, E. A., Tiippana, K., Nicol, T. G., Sams, M., & Kraus, N. (2003). Integration of heard and seen speech: a factor in learning disabilities in children. Neuroscience Letters, 351, 46–50.CrossRefPubMedGoogle Scholar
  26. Kast, M., Bezzola, L., Jäncke, L., & Meyer, M. (2011). Multi- and unisensory decoding of words and nonwords result in differential brain responses in dyslexic and nondyslexic adults. Brain and Language, 119, 136–148.CrossRefPubMedGoogle Scholar
  27. Kere, J. (2014). The molecular genetics and neurobiology of developmental dyslexia as model of a complex phenotype. Biochemical and Biophysical Research Communications, 452, 236–243.CrossRefPubMedGoogle Scholar
  28. Kersting, M., & Althoff, K. (2004). RT. Rechtschreibungstest. Göttingen: Hogrefe.Google Scholar
  29. Kim, D. I., Manoach, D. S., Mathalon, D. H., Turner, J. A., Mannell, M., Brown, G. G., Ford, J. M., Gollub, R. L., White, T., Wible, C., Belger, A., Bockholt, H. J., Clark, V. P., Lauriello, J., O’Leary, D., Mueller, B. A., Lim, K. O., Andreasen, N., Potkin, S. G., & Calhoun, V. D. (2009a). Dysregulation of working memory and default-mode networks in schizoprehnia using independent component analysis, an fBIRN and MCIC study. Human Brain Mapping, 30, 3795–3811.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Kim, D. I., Mathalon, D. H., Ford, J. M., Mannell, M., Turner, J. A., Brown, G. G., Belger, A., Gollub, R., Lauriello, J., Wible, C., O’Leary, D., Lim, K., Toga, A., Potkin, S. G., Birn, F., & Calhoun, V. D. (2009b). Auditory oddball deficits in schizophrenia: an independent component analysis of the fMRI multisite function BIRN study. Schizophrenia Bulletin, 35, 67–81.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Kochan, N.A., Valenzuela, M., Slavin, M.J., McCraw, S., Sachdev, P.S., & Breakspear, M. (2011). Impact of load-related neural processes on feature binding in visuospatial working memory. PLoS One, e23960.Google Scholar
  32. Kronschnabel, J., Brem, S., Maurer, U., & Brandeis, D. (2014). The level of audiovisual print-speech integration deficits in dyslexia. Neuropsychologia, 62, 245–261.CrossRefPubMedGoogle Scholar
  33. Landi, N., Frost, S. J., Mencl, W. E., Sandak, R., & Pugh, K. R. (2013). Neurobiological bases of reading comprehension: Insights from neuroimaging studies of word-level and text-level processing in skilled and impaired readers. Reading & Writing Quarterly, 29, 145–167.CrossRefGoogle Scholar
  34. Langer, N., Benjamin, C., Minas, J., & Gaab, N. (2015). The neural correlates of reading fluency deficits in children. Cerebral Cortex, 25, 1441–1453.CrossRefPubMedGoogle Scholar
  35. Lee, H., & Noppeney, U. (2011). Physical and perceptual factors shape the neural mechanisms that integrate audiovisual signals in speech comprehension. The Journal of Neuroscience, 31, 11338–11350.CrossRefPubMedGoogle Scholar
  36. Li, Y. O., Adali, T., & Calhoun, V. D. (2007). Estimating the number of independent components for functional magnetic resonance imaging data. Human Brain Mapping, 28, 1251–1266.CrossRefPubMedGoogle Scholar
  37. Linder, M., & Grissemann, H. (2000). Zürcher Lesetest (ZLT). Bern: Huber.Google Scholar
  38. Mak, L.E., Minuzzi, L., MacQueen, G., Hall, G., Kennedy, S., Milev, R. (2016). The default mode network in healthy individuals: A systematic review and meta-analysis. Brain Connect. [Epub ahead of print]Google Scholar
  39. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.CrossRefPubMedGoogle Scholar
  40. Megnin-Viggars, O., & Goswami, U. (2013). Audiovisual perception of noise vocoded speech in dyslexic and non-dyslexic adults: The role of low-frequency visual modulations. Brain and Language, 124, 165–173.CrossRefPubMedGoogle Scholar
  41. Mittag, M., Thesleff, P., Laasonen, M., & Kujala, T. (2013). The neurophysiological basis of the integration of written and heard syllables in dyslexic adults. Clinical Neurophysiology, 124, 315–326.CrossRefPubMedGoogle Scholar
  42. Nath, A. R., & Beauchamp, M. S. (2011). Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech. The Journal of Neuroscience, 31, 1704–1714.CrossRefPubMedPubMedCentralGoogle Scholar
  43. Nath, A. R., & Beauchamp, M. S. (2012). A neural basis of interindividual differences in the McGurk effect, a multisensory speech illusion. NeuroImage, 59, 781–787.CrossRefPubMedGoogle Scholar
  44. Nestor, A., Behrmann, M., & Plaut, D. C. (2013). The neural basis of visual word form processing: A multivariate investigation. Cerebral Cortex, 23, 1673–1684.CrossRefPubMedGoogle Scholar
  45. Pekkola, J., Laasonen, M., Ojanen, V., Autti, T., Jaaskelainen, I. P., Kujala, T., & Sams, M. (2005). Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3 T. NeuroImage, 29, 797–807.CrossRefPubMedGoogle Scholar
  46. Peterson, R. L., & Pennington, B. F. (2012). Developmental dyslexia. Lancet, 379, 1997–2007.CrossRefPubMedPubMedCentralGoogle Scholar
  47. Raichle, M. E. (2015). The brain's default mode network. Annual Review of Neuroscience (Palo Alto, CA), 38, 433–447.CrossRefGoogle Scholar
  48. Ramirez, J., & Mann, V. (2005). Using auditory-visual speech to probe the basis of noise-impaired consonant-vowel perception in dyslexia and auditory neuropathy. The Journal of the Acoustical Society of America, 118, 1122–1133.CrossRefPubMedGoogle Scholar
  49. Ramus, F. (2003). Developmental dyslexia: specific phonological deficit or general sensorimotor dysfunction? Current Opinion in Neurobiology, 13, 212–218.CrossRefPubMedGoogle Scholar
  50. Raskind, W. H., Peter, B., Richards, T., Eckert, M. M., & Berninger, V. W. (2013). The genetics of reading disabilities: from phrnotypres to candidate genes. Frontiers in Psychology, 3, 601.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Richlan, F., Kronbichler, M., & Wimmer, H. (2009). Functional abnormalities in the dyslexic brain: a quantitative meta-analysis of neuroimaging studies. Human Brain Mapping, 30, 3299–3308.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Richlan, F., Kronbichler, M., & Wimmer, H. (2011). Meta-analyzing brain dysfunctions in dyslexic children and adults. NeuroImage, 56, 1735–1742.CrossRefPubMedGoogle Scholar
  53. Rüsseler, J., Gerth, I., Heldmann, M., & Münte, T. F. (2015). Audiovisual perception of natural speech is impaired in adult dyslexics: an ERP study. Neuroscience, 287, 55–65.CrossRefPubMedGoogle Scholar
  54. Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y. (2003). Auditory-visual speech perception examined by fMRI and PET. Neuroscience Research, 47, 277–287.CrossRefPubMedGoogle Scholar
  55. Shaywitz, S. E., & Shaywitz, B. A. (2008). Paying attention to reading: the neurobiology of reading and dyslexia. Development and Psychopathology, 20, 1329–1349.CrossRefPubMedGoogle Scholar
  56. Shaywitz, S. E., Shaywitz, B. A., Fletcher, J. M., & Escobar, M. D. (1990). Prevalence of reading disability in boys and girls. Results of the Connecticut longitudinal study. Journal of the American Medical Association, 264, 998–1002.CrossRefPubMedGoogle Scholar
  57. Sigurdardottir, H. M., Ívarsson, E., Kristinsdóttir, K., & Kristjánsson, Á. (2015). Impaired recognition of faces and objects in dyslexia: Evidence for ventral stream dysfunction? Neuropsychology, 29, 739–750.CrossRefPubMedGoogle Scholar
  58. Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007). Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17, 2387–2399.CrossRefPubMedPubMedCentralGoogle Scholar
  59. Snowling, M. J. (1980). The development of grapheme-phoneme correspondence in normal and dyslexic readers. Journal of Experimental Child Psychology, 29, 294–305.CrossRefPubMedGoogle Scholar
  60. Snowling, M. J. (2001). From language to reading and dyslexia. Dyslexia, 7, 37–46.CrossRefPubMedGoogle Scholar
  61. Stevenson, R. A., van der Klock, R. M., Pisoni, D. B., & James, T. W. (2011). Discrete neural substrates underlie complementary audiovisual speech integration processes. NeuroImage, 55, 1339–1345.CrossRefPubMedGoogle Scholar
  62. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America , 26, 212–215.CrossRefGoogle Scholar
  63. Szycik, G. R., Jansma, H., & Münte, T. F. (2008). Audiovisual integration during speech comprehension: an fMRI study comparing ROI-based and whole brain analyses. Human Brain Mapping, 30, 1990–1999.CrossRefGoogle Scholar
  64. Szycik, G. R., Münte, T. F., Dillo, W., Mohammadi, B., Samii, A., Emrich, H. M., & Dietrich, D. E. (2009). Audiovisual integration of speech is disturbed in schizophrenia: An fMRI study. Schizophrenia Research, 110, 111–118.CrossRefPubMedGoogle Scholar
  65. Szycik, G. R., Stadler, J., Tempelmann, C., & Münte, T. F. (2012). Examining the McGurk illusion using high-field 7 Tesla functional MRI. Frontiers in Human Neuroscience, 6, 95.CrossRefPubMedPubMedCentralGoogle Scholar
  66. Szycik, G. R., Ye, Z., Mohammadi, B., Dillo, W., Te Wildt, B. T., Samii, A., Frieling, H., Bleich, S., & Münte, T. F. (2013). Maladaptive connectivity of Broca's area in schizophrenia during audiovisual speech perception: an fMRI study. Neuroscience, 253, 274–282.CrossRefPubMedGoogle Scholar
  67. Tiippana, K. (2014). What is the McGurk effect? Frontiers in Psychology, 5, 725.CrossRefPubMedPubMedCentralGoogle Scholar
  68. Tree, J. J. (2008). Two types of phonological dyslexia - a contemporary review. Cortex, 44, 698–706.CrossRefPubMedGoogle Scholar
  69. Vellutino, F. R., Fletcher, J. M., Snowling, M. J., & Scanlon, D. M. (2004). Specific reading disability (dyslexia): what have we learned in the past four decades? Journal of Child Psychology and Psychiatry, 45, 2–40.CrossRefPubMedGoogle Scholar
  70. Widmann, A., Schröger, E., Tervaniemi, M., Pakarinen, S., & Kujala, T. (2012). Mapping symbols to sounds: electrophysiological correlates of the impaired reading process in dyslexia. Frontiers in Psychology, 2(3), 60.Google Scholar
  71. Wright, T. M., Pelphrey, K. A., Allison, T., McKeown, M. J., & McCarthy, G. (2003). Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cerebral Cortex, 13, 1034–1043.CrossRefPubMedGoogle Scholar
  72. Ye, Z., Doñamayor, N., & Münte, T. F. (2014). Brain network of semantic integration in sentence reading: insights from independent component analysis and graph theoretical analysis. Human Brain Mapping, 35, 367–376.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Jascha Rüsseler
    • 1
  • Zheng Ye
    • 2
  • Ivonne Gerth
    • 3
  • Gregor R. Szycik
    • 4
  • Thomas F. Münte
    • 5
    • 6
  1. 1.Department of PsychologyOtto-Friedrich University BambergBambergGermany
  2. 2.Institute of PsychologyChinese Academy of SciencesBeijingChina
  3. 3.Klinikum Magdeburg, NeurologieMagdeburgGermany
  4. 4.Department of PsychiatryMedical School HannoverHannoverGermany
  5. 5.Department of NeurologyUniversity of LübeckLübeckGermany
  6. 6.Institute of Psychology IIUniversity of LübeckLübeckGermany

Personalised recommendations