Advertisement

Interactive effects of linguistic abstraction and stimulus statistics in the online modulation of neural speech encoding

  • Joseph C. Y. Lau
  • Patrick C. M. WongEmail author
  • Bharath Chandrasekaran
Perceptual/Cognitive Constraints on the Structure of Speech Communication: In Honor of Randy Diehl
  • 44 Downloads

Abstract

Speech processing is highly modulated by context. Prior studies examining frequency-following responses (FFRs), an electrophysiological ‘neurophonic’ potential that faithfully reflects phase-locked activity from neural ensembles within the auditory network, have demonstrated that stimulus context modulates the integrity of speech encoding. The extent to which context-dependent encoding reflects general auditory properties or interactivities between statistical and higher-level linguistic processes remains unexplored. Our study examined whether speech encoding, as reflected by FFRs, is modulated by abstract phonological relationships between a stimulus and surrounding contexts. FFRs were elicited to a Mandarin rising-tone syllable (/ji-TR/, ‘second’) randomly presented with other syllables in three contexts from 17 native listeners. In a contrastive context, /ji-TR/ occurred with meaning-contrastive high-level-tone syllables (/ji-H/, ‘one’). In an allotone context, TR occurred with dipping-tone syllables /ji-D/, a non-meaning-contrastive variant of /ji-TR/. In a repetitive context, the same /ji-TR/ occurred with other speech tokens of /ji-TR/. Consistent with prior work, neural tracking of /ji-TR/ pitch contour was more faithful in the repetitive condition wherein /ji-TR/ occurred more predictably (p = 1) than in the contrastive condition (p = 0.34). Crucially, in the allotone context, neural tracking of /ji-TR/ was more accurate relative to the contrastive context, despite both having an identical transitional probability (p = 0.34). Mechanistically, the non-meaning-contrastive relationship may have augmented the probability to /ji-TR/ occurrence in the allotone context. Results indicate online interactions between bottom-up and top-down mechanisms, which facilitate speech perception. Such interactivities may predictively fine-tune incoming speech encoding using linguistic and statistical information from prior context.

Keywords

Neural speech encoding Frequency-following response (FFR) Lexical tone Context-dependent plasticity Linguistic abstraction Allotones 

Notes

Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders Grant 1R01-DC-013315 (to B. Chandrasekaran), Research Grants Council of Hong Kong General Research Fund 14117514 (to P.C.M. Wong), Global Parent Child Resource Centre Limited (to P.C.M. Wong), Lui Che Woo Institute of Innovative Medicine (to P. C. M. Wong), and Dr. Stanley Ho Medical Development Foundation (to P.C.M. Wong). We also wish to thank Christopher Chan, Kirin Cheung, Tianfan Liu, Grace Pan, Binghui Shen, Xiaohui Sun, and Yi Wu for their assistance with data collection.

References

  1. Anderson, L., & Malmierca, M. (2013). The effect of auditory cortex deactivation on stimulus-specific adaptation in the inferior colliculus of the rat. European Journal of Neuroscience, 37(1), 52–62.PubMedGoogle Scholar
  2. Bidelman, G. M. (2015). Multichannel recordings of the human brainstem frequency-following response: scalp topography, source generators, and distinctions from the transient ABR. Hearing Research, 323, 68–80.PubMedGoogle Scholar
  3. Bidelman, G. M. (2018). Subcortical sources dominate the neuroelectric auditory frequency-following response to speech. NeuroImage, 175, 56–69.PubMedGoogle Scholar
  4. Bidelman, G. M., Gandour, J. T., & Krishnan, A. (2011). Musicians and tone-language speakers share enhanced brainstem encoding but not perceptual benefits for musical pitch. Brain and Cognition, 77(1), 1–10.PubMedPubMedCentralGoogle Scholar
  5. Boersma, P., & Weenink, D. (2014). Praat: doing phonetics by computer [computer program]. (Version 5. 3. 73, retrieved 21 April 2014 from http://www.praat.org/).
  6. Chandrasekaran, B., Hornickel, J., Skoe, E., Nicol, T., & Kraus, N. (2009). Context-dependent encoding in the human auditory brainstem relates to hearing speech in noise: implications for developmental dyslexia. Neuron, 64(3), 311– 319.PubMedPubMedCentralGoogle Scholar
  7. Chandrasekaran, B., & Kraus, N. (2010a). The scalp-recorded brainstem response to speech: neural origins and plasticity. Psychophysiology, 47(2), 236–246.PubMedGoogle Scholar
  8. Chandrasekaran, B., Sampath, P. D., & Wong, P. C. (2010b). Individual variability in cue-weighting and lexical tone learning. The Journal of the Acoustical Society of America, 128(1), 456– 465.PubMedPubMedCentralGoogle Scholar
  9. Chandrasekaran, B., Skoe, E., & Kraus, N. (2014). An integrative model of subcortical auditory plasticity. Brain Topography, 27(4), 539–552.PubMedGoogle Scholar
  10. Chen, Y., Shen, R., & Schiller, N. O. (2011). Representation of allophonic tone sandhi variants. In Proceedings of psycholinguistics representation of tone. Satellite Workshop to ICPhS, Hong Kong (pp. 38–41).Google Scholar
  11. Chien, Y. F., Sereno, J. A., & Zhang, J. (2016). Priming the representation of Mandarin tone 3 sandhi words. Language, Cognition and Neuroscience, 31(2), 179–189.Google Scholar
  12. Coffey, E. B., Herholz, S. C., Chepesiuk, A. M., Baillet, S., & Zatorre, R. J. (2016). Cortical contributions to the auditory frequency-following response revealed by MEG. Nature Communications, 7.Google Scholar
  13. Coffey, E. B., Musacchia, G., & Zatorre, R. J. (2017). Cortical correlates of the auditory frequency-following and onset responses: EEG and fMRI evidence. Journal of Neuroscience, 37(4), 830–838.PubMedGoogle Scholar
  14. Connolly, J. F., & Phillips, N. A. (1994). Event-related potential components reflect phonological and semantic processing of the terminal word of spoken sentences. Journal of Cognitive Neuroscience, 6(3), 256–266.PubMedGoogle Scholar
  15. Denham, S. L., & Winkler, I. (2017). Predictive coding in auditory perception: challenges and unresolved questions. European Journal of Neuroscience.Google Scholar
  16. Diaz, M. T., & Swaab, T. Y. (2007). Electrophysiological differentiation of phonological and semantic integration in word and sentence contexts. Brain Research, 1146, 85–100.PubMedGoogle Scholar
  17. Diehl, R. L. (1987). Auditory constraints on speech perception. In M. Schouten (Ed.) The psychophysics of speech perception (pp. 39:210–219). Martimus-Nihboff: Dordrecht.Google Scholar
  18. Diehl, R. L., Lotto, A. J., & Holt, L. L. (2004). Speech perception. Annual Review of Psychology, 55, 149–179.PubMedGoogle Scholar
  19. Escera, C., Yago, E., Corral, M. J., Corbera, S., & Nuñez, M. I. (2003). Attention capture by auditory significant stimuli: semantic analysis follows attention switching. European Journal of Neuroscience, 18(8), 2408–2412.PubMedGoogle Scholar
  20. Eulitz, C., & Lahiri, A. (2004). Neurobiological evidence for abstract phonological representations in the mental lexicon during speech recognition. Journal of Cognitive Neuroscience, 16(4), 577–583.PubMedGoogle Scholar
  21. Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2), 78–84.PubMedGoogle Scholar
  22. Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society of London, Series B, 360(1456), 815–836.Google Scholar
  23. Ganong, W. F. (1980). Phonetic categorization in auditory word perception. Journal of Experimental Psychology: Human Perception and Performance, 6(1), 110.PubMedGoogle Scholar
  24. Gaskell, M. G., & Marslen-Wilson, W. D. (2002). Representation and competition in the perception of spoken words. Cognitive Psychology, 45(2), 220–266.PubMedGoogle Scholar
  25. Giard, M. H., Perrin, F., Pernier, J., & Bouchet, P. (1990). Brain generators implicated in the processing of auditory stimulus deviance: a topographic event-related potential study. Psychophysiology, 27(6), 627–640.PubMedGoogle Scholar
  26. Hagoort, P., & Brown, C. M. (2000). ERP Effects of listening to speech: semantic ERP effects. Neuropsychologia, 38(11), 1518–1530.PubMedGoogle Scholar
  27. Hickok, G., & Peoppel, D. (2007). The cortical organization of speech processing. Nature Reviews Neuroscience, 8, 393–402.PubMedGoogle Scholar
  28. Hickok, G. (2012). The cortical organization of speech processing: feedback control and predictive coding the context of a dual-stream model. Journal of Communication Disorders, 45(6), 393–402.PubMedPubMedCentralGoogle Scholar
  29. Holt, L. L., Lotto, A. J., & Kluender, K. R. (1998). Incorporating principles of general learning in theories of language acquisition. Chicago Linguistic Society, 34, 253–268.Google Scholar
  30. Jääskeläinen, I. P., Ahveninen, J., Bonmassar, G., Dale, A. M., Ilmoniemi, R. J., Levänen, S., & et al. (2004). Human posterior auditory cortex gates novel sounds to consciousness. Proceedings of the National Academy of Sciences of the United States of America, 101(17), 6809–6814.PubMedPubMedCentralGoogle Scholar
  31. Kraus, N., Anderson, S., & White-Schwoch, T. (2017). The frequency-following response: a window into human communication. In The frequency-following response (pp. 1–15): Springer.Google Scholar
  32. Kraus, N., & White-Schwoch, T. (2015). Unraveling the biology of auditory learning: a cognitive–sensorimotor–reward framework. Trends in Cognitive Sciences, 19(11), 642–654.PubMedPubMedCentralGoogle Scholar
  33. Krishnan, A., Xu, Y., Gandour, J. T., & Cariani, P. A. (2004). Human frequency-following response: representation of pitch contours in Chinese tones. Hearing Research, 189(1), 1–12.PubMedGoogle Scholar
  34. Krishnan, A., & Gandour, J. T. (2009). The role of the auditory brainstem in processing linguistically-relevant pitch patterns. Brain and Language, 110(3), 135–148.PubMedPubMedCentralGoogle Scholar
  35. Lau, E. F., Phillips, C., & Poeppel, D. (2008). A cortical network for semantics: (de) constructing the n400. Nature Reviews Neuroscience, 9(12), 920–933.PubMedGoogle Scholar
  36. Lau, J. C., Wong, P. C., & Chandrasekaran, B. (2017). Context-dependent plasticity in the subcortical encoding of linguistic pitch patterns. Journal of Neurophysiology, 117(2), 594–603.PubMedGoogle Scholar
  37. Li, X., & Chen, Y. (2015). Representation and processing of lexical tone and tonal variants: evidence from the mismatch negativity. PLOS One, 10(12), e0143097.PubMedPubMedCentralGoogle Scholar
  38. Liu, F., Maggu, A. R., Lau, J. C., & Wong, P. C. (2014). Brainstem encoding of speech and musical stimuli incongenital amusia: evidence from Cantonese speakers. Frontiers in Human Neuroscience, 8, 1029.PubMedGoogle Scholar
  39. López-Caballero, F., Zarnowiec, K., & Escera, C. (2016). Differential deviant probability effects on two hierarchical levels of the auditory novelty system. Biological Psychology, 120, 1–9.PubMedGoogle Scholar
  40. Lotto, A. J. (2000). Language acquisition as complex category formation. Phonetica, 57(2-4), 189–196.PubMedGoogle Scholar
  41. Luce, P. A., & Pisoni, D. B. (1998). Recognizing spoken words: the neighborhood activation model. Ear and Hearing, 19(1), 1.PubMedPubMedCentralGoogle Scholar
  42. Lupyan, G., & Clark, A. (2015). Words and the world predictive coding and the language–perception–cognition interface. Current Directions in Psychological Science, 24(4), 279–284.Google Scholar
  43. Malmierca, M. S., Cristaudo, S., Pérez-González, D., & Covey, E. (2009). Stimulus-specific adaptation in the inferior colliculus of the anesthetized rat. Journal of Neuroscience, 29(17), 5483–5493.PubMedGoogle Scholar
  44. Malmierca, M. S., Anderson, L. A., & Antunes, F. M. (2015). The cortical modulation of stimulus-specific adaptation in the auditory midbrain and thalamus: a potential neuronal correlate for predictive coding. Frontiers in Systems Neuroscience, 9, 19.PubMedPubMedCentralGoogle Scholar
  45. Marslen-Wilson, W. D., & Welsh, A. (1978). Processing interactions and lexical access during word recognition in continuous speech. Cognitive psychology, 10(1), 29–63.Google Scholar
  46. McClelland, J. L., & Elman, J. L. (1986). The TRACE model of speech perception. Cognitive Psychology, 18 (1), 1–86.PubMedGoogle Scholar
  47. Moulines, E., & Charpentier, F. (1990). Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones. Speech Communication, 9(5-6), 453–467.Google Scholar
  48. Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology, 118(12), 2544–2590.PubMedGoogle Scholar
  49. Natan, R. G., Briguglio, J. J., Mwilambwe-Tshilobo, L., Jones, S. I., Aizenberg, M., Goldberg, E. M., & Geffen, M. N. (2015). Complementary control of sensory adaptation by two types of cortical interneurons. Elife, 4, e09868.PubMedPubMedCentralGoogle Scholar
  50. Nourski, K. V., Banks, M. I., Steinschneider, M., Rhone, A. E., Kawasaki, H., Mueller, R. N., & Howard III, M. A. (2017). Electrocorticographic delineation of human auditory cortical fields based on effects of propofol anesthesia. Neuroimage, 152, 78–93.PubMedPubMedCentralGoogle Scholar
  51. Nusbaum, H., & Magnuson, J. (1997). Talker normalization: phonetic constancy as a cognitive process. In K.J.J. Mullenni (Ed.) Talker variability in speech processing (pp. 109–132). New York: Academic Press.Google Scholar
  52. Parbery-Clark, A., Strait, D., & Kraus, N. (2011). Context-dependent encoding in the auditory brainstem subserves enhanced speech-in-noise perception in musicians. Neuropsychologia, 49(12), 3338–3345.PubMedPubMedCentralGoogle Scholar
  53. Pérez-González, D., Malmierca, M. S., & Covey, E. (2005). Novelty detector neurons in the mammalian auditory midbrain. European Journal of Neuroscience, 22(11), 2879–2885.PubMedGoogle Scholar
  54. Politzer-Ahles, S., & Zhang, J. (2012). The role of phonological alternation in speech production: evidence from Mandarin tone sandhi. In Proceedings of meetings on acoustics 164ASA (vol. 18, No. 1, p. 060001). ASA.Google Scholar
  55. Politzer-Ahles, S., Schluter, K., Wu, K., & Almeida, D. (2016). Asymmetries in the perception of Mandarin, tones: evidence from mismatch negativity. Journal of Experimental Psychology: Human Perception and Performance, 42(10), 1547.PubMedGoogle Scholar
  56. Rauschecker, J. P., & Scott, S. K. (2009). Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nature Neuroscience, 12(6), 718–724.PubMedPubMedCentralGoogle Scholar
  57. Rubin, J., Ulanovsky, N., Nelken, I., & Tishby, N. (2016). The representation of prediction error in auditory cortex. PLoS Computational Biology, 12(8), e1005058.PubMedPubMedCentralGoogle Scholar
  58. Russo, N., Nicol, T., Musacchia, G., & Kraus, N. (2004). Brainstem responses to speech syllables. Clinical Neurophysiology, 115(9), 2021–2030.PubMedPubMedCentralGoogle Scholar
  59. Shen, X. S., & Lin, M. (1991). A perceptual study of Mandarin tones 2 and 3. Language and Speech, 34(2), 145–156.Google Scholar
  60. Skoe, E., & Kraus, N. (2010). Auditory brainstem response to complex sounds: a tutorial. Ear and Hearing, 31(3), 302.PubMedPubMedCentralGoogle Scholar
  61. Skoe, E., Chandrasekaran, B., Spitzer, E. R., Wong, P. C., & Kraus, N. (2014). Human brainstem plasticity: the interaction of stimulus probability and auditory learning. Neurobiology of Learning and Memory, 109, 82–93.PubMedGoogle Scholar
  62. Slabu, L., Grimm, S., & Escera, C. (2012). Novelty detection in the human auditory brainstem. Journal of Neuroscience, 32(4), 1447–1452.PubMedGoogle Scholar
  63. Slowiaczek, L. M., Nusbaum, H. C., & Pisoni, D. B. (1987). Phonological priming in auditory word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(1), 64.PubMedGoogle Scholar
  64. Song, J. H., Skoe, E., Wong, P. C., & Kraus, N. (2008). Plasticity in the adult human auditory brainstem following short-term linguistic training. Journal of Cognitive Neuroscience, 20(10), 1892–1902.PubMedPubMedCentralGoogle Scholar
  65. Strait, D. L., Hornickel, J., & Kraus, N. (2011). Subcortical processing of speech regularities underlies reading and music aptitude in children. Behavioral and Brain Functions, 7(1), 1.Google Scholar
  66. Strait, D. L., Parbery-Clark, A., Hittner, E., & Kraus, N. (2012). Musical training during early childhood enhances the neural encoding of speech in noise. Brain and Language, 123(3), 191–201.PubMedPubMedCentralGoogle Scholar
  67. Suga, N. (2008). Role of corticofugal feedback in hearing. Journal of Comparative Physiology A, 194(2), 169–183.Google Scholar
  68. Whalen, D. H., & Xu, Y. (1992). Information for Mandarin tones in the amplitude contour and in brief segments. Phonetica, 49(1), 25–47.PubMedGoogle Scholar
  69. Winer, J. A., Larue, D. T., Diehl, J. J., & Hefti, B. J. (1998). Auditory cortical projections to the cat inferior colliculus. Journal of Comparative Neurology, 400(2), 147–174.PubMedGoogle Scholar
  70. Wong, P. C., & Diehl, R. L. (2003). Perceptual normalization for inter-and intratalker variation in Cantonese level tones. Journal of Speech, Language, and Hearing Research, 46(2), 413–421.PubMedGoogle Scholar
  71. Wong, P. C., Skoe, E., Russo, N. M., Dees, T., & Kraus, N. (2007). Musical experience shapes human brainstem encoding of linguistic pitch patterns. Nature Neuroscience, 10(4), 420–422.PubMedPubMedCentralGoogle Scholar
  72. Xie, Z., Reetzke, R., & Chandrasekaran, B. (2017). Stability and plasticity in neural encoding of linguistically relevant pitch patterns. Journal of Neurophysiology, 117(3), 1409–1424.Google Scholar
  73. Xie, Z., Reetzke, R., & Chandrasekaran, B. (2018). Taking attention away from the auditory modality: context-dependent effects on early sensory representation of speech. Neuroscience, 384, 64–75.PubMedGoogle Scholar
  74. Yip, M. (2002) Tone. Cambridge: Cambridge University Press.Google Scholar
  75. Zhang, C., Xia, Q., & Peng, G. (2015). Mandarin third tone sandhi requires more effortful phonological encoding in speech production: evidence from an ERP study. Journal of Neurolinguistics, 33, 149–162.Google Scholar
  76. Zhou, X., & Marslen-Wilson, W. (1997). The abstractness of phonological representation in the Chinese mental lexicon. In Cognitive Processing of Chinese and other Asian languages (pp. 3–26).Google Scholar

Copyright information

© The Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Joseph C. Y. Lau
    • 1
    • 2
  • Patrick C. M. Wong
    • 1
    • 2
    Email author
  • Bharath Chandrasekaran
    • 3
  1. 1.Department of Linguistics and Modern LanguagesThe Chinese University of Hong KongHong Kong SARChina
  2. 2.Brain and Mind InstituteThe Chinese University of Hong KongHong Kong SARChina
  3. 3.Department of Communication Science and Disorders, School of Health and Rehabilitation SciencesUniversity of PittsburghPittsburghUSA

Personalised recommendations