Abstract
The brain is the most complex biological system that exists. Timbre, in its very nature, is a multidimensional concept with several levels of abstraction thus rendering the investigation of its processing in the brain extremely challenging. Timbre processing can be discussed in relation to levels of abstraction. Low- to mid-level representations can be associated with the neural representation of acoustic structure while high-level abstractions correspond to the neural representation of sound source properties. Furthermore, neural correlates of timbre can be broadly classified based on three stimulus categories, that is, those pertaining to music, speech, and environmental sounds. This chapter summarizes studies that have attempted to uncover neural correlates of varying levels of timbre abstractions. Finally, developments in methodological approaches are described, including the shift from univariate to multivariate statistical models, the employment of more naturalistic stimuli, and brain measurement paradigms from hitherto controlled auditory paradigms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allen EJ, Oxenham AJ (2014) Symmetric interactions and interference between pitch and timbre. J Acoust Soc Am 55455:1371–1379. https://doi.org/10.1121/1.4863269
Allen EJ, Burton PC, Olman CA, Oxenham AJ (2017) Representations of pitch and timbre variation in human auditory cortex. J Neurosci 37:1284–1293. https://doi.org/10.1523/JNEUROSCI.2336-16.2016
Alluri V, Toiviainen P (2010) Exploring perceptual and acoustical correlates of polyphonic timbre. Music Percept 27:223–242. https://doi.org/10.1525/mp.2010.27.3.223
Alluri V, Toiviainen P, Jääskeläinen IP et al (2012) Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm. NeuroImage 59:3677–3689. https://doi.org/10.1016/j.neuroimage.2011.11.019
Alluri V, Toiviainen P, Lund TE et al (2013) From Vivaldi to Beatles and back: predicting lateralized brain responses to music. NeuroImage 83:627–636. https://doi.org/10.1016/j.neuroimage.2013.06.064
Aucouturier JJ (2006) Dix expériences sur la modélisation du timbre polyphonique (Ten experiments on the modeling of polyphonic timbre). Dissertation, Université Paris 6
Auzou P, Eustache F, Etevenon P et al (1995) Topographic EEG activations during timbre and pitch discrimination tasks using musical sounds. Neuropsychologia 33:25–37. https://doi.org/10.1016/0028-3932(94)00100-4
Baumann S, Griffiths TD, Sun L et al (2011) Orthogonal representation of sound dimensions in the primate midbrain. Nat Neurosci 14:423–425. https://doi.org/10.1038/nn.2771
Baumann S, Joly O, Rees A et al (2015) The topography of frequency and time representation in primate auditory cortices. elife 4:e03256. https://doi.org/10.7554/eLife.03256
Barbedo JGA, Lopes A (2007) Automatic genre classification of musical signals. EURASIP J Adv Signal Process, 2007(1): 157–168
Beauchamp MS, Lee K, Argall B, Martin A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809–823. https://doi.org/10.1016/S0896-6273(04)00070-4
Belin P, Zatorre RJ, Lafaille P et al (2000) Voice-selective areas in human auditory cortex. Nature 403:309–312. https://doi.org/10.1038/35002078
Bizley JK, Cohen YE (2013) The what, where and how of auditory-object perception. Nat Publ Gr 14:693–707. https://doi.org/10.1038/nrn3565
Bizley JK, Walker KMM, Silverman BW et al (2009) Interdependent encoding of pitch, timbre, and spatial location in auditory cortex. J Neurosci 29:2064–2075. https://doi.org/10.1523/JNEUROSCI.4755-08.2009
Bregman AS (1990) Auditory scene analysis: the perceptual organization of sound. Cambridge (MA), MIT Press
Burunat I, Toiviainen P, Alluri V et al (2016) The reliability of continuous brain responses during naturalistic listening to music. NeuroImage 124:224–231. https://doi.org/10.1016/j.neuroimage.2015.09.005
Caclin A, Brattico E, Tervaniemi M et al (2006) Separate neural processing of timbre dimensions in auditory sensory memory. J Cogn Neurosci 18:1959–1972. https://doi.org/10.1162/jocn.2006.18.12.1959
Caclin A, McAdams S, Smith BK, Giard M-H (2008) Interactive processing of timbre dimensions: an exploration with event-related potentials. J Cogn Neurosci 20:49–64. https://doi.org/10.1162/jocn.2008.20.1.49
Casey MA (2017) Music of the 7Ts: predicting and decoding multivoxel fMRI responses with acoustic, schematic, and categorical music features. Front Psychol 8:1179. https://doi.org/10.3389/fpsyg.2017.01179
Casey M, Thompson J, Kang O, et al (2011) Population codes representing musical timbre for high-level fMRI categorization of music genres. In: Langs G, Rish I, Grosse-Wentrup M, Murphy B (eds) Machine learning and interpretation in neuroimaging. International workshop, MLINI 2011, held at NIPS 2011, Sierra Nevada, December 2011. Springer, Heidelberg, p 34–41
Crummer GC, Walton JP, Wayman JW et al (1994) Neural processing of musical timbre by musicians, nonmusicians, and musicians possessing absolute pitch. J Acoust Soc Am 95:2720–2727. https://doi.org/10.1121/1.409840
Davis T, LaRocque KF, Mumford JA et al (2014) What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis. NeuroImage 97:271–283. https://doi.org/10.1016/j.neuroimage.2014.04.037
Deike S, Gaschler-Markefski B, Brechmann A, Scheich H (2004) Auditory stream segregation relying on timbre involves left auditory cortex. Neuroreport 15:1511–1514. https://doi.org/10.1097/01.wnr.0000132919.12990.34
Doehrmann O, Naumer MJ, Volz S et al (2008) Probing category selectivity for environmental sounds in the human auditory brain. Neuropsychologia 46:2776–2786. https://doi.org/10.1016/j.neuropsychologia.2008.05.011
Engelien A, Tüscher O, Hermans W, Isenberg N, Eidelberg D, Frith C et al (2006) Functional neuroanatomy of non-verbal semantic sound processing in humans. J Neural Transm 113:599–608
Ferrer R (2011) Timbral environments: an ecological approach to the cognition of timbre. Empir Musicol Rev 6:64–74
Formisano E, Kim DS, Di SF (2003) Mirror-symmetric tonotopic maps in human primary auditory cortex. Neuron 40:859–869
Formisano E, De Martino F, Bonte M, Goebel R (2008) “Who” is saying “what”? Brain-based decoding of human voice and speech. Science 322:970–973
Fox MD, Zhang D, Snyder AZ, Raichle ME (2009) The global signal and observed anticorrelated resting state brain networks. J Neurophys 101(6):3270–3283
Giordano BL, Rocchesso D, McAdams S (2010) Integration of acoustical information in the perception of impacted sound sources: the role of information accuracy and exploitability. J Exp Psychol Hum Percept Perform 36:462–476. https://doi.org/10.1037/a0018388
Giordano BL, McAdams S, Zatorre RJ et al (2013) Abstract encoding of auditory objects in cortical activity patterns. Cereb Cortex 23:2025–2037. https://doi.org/10.1093/cercor/bhs162
Giordano BL, Pernet C, Charest I et al (2014) Automatic domain-general processing of sound source identity in the left posterior middle frontal gyrus. Cortex 58:170–185. https://doi.org/10.1016/j.cortex.2014.06.005
Gjerdingen RO, Perrott D (2008) Scanning the dial: the rapid recognition of music genres. J New Music Res 37:93–100. https://doi.org/10.1080/09298210802479268
Goldstone RL (1998) Perceptual learning. Annu Rev Psychol 49:585–612. https://doi.org/10.1146/annurev.psych.49.1.585
Halpern AR, Zatorre RJ, Bouffard M, Johnson JA (2004) Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 42:1281–1292. https://doi.org/10.1016/j.neuropsychologia.2003.12.017
Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R (2004) Intersubject synchronization of cortical activity during natural vision. Science 303:1634–1640
Hoefle S, Engel A, Basilio R et al (2018) Identifying musical pieces from fMRI data using encoding and decoding models. Sci Rep 8:2266. https://doi.org/10.1038/s41598-018-20732-3
Humphries C, Liebenthal E, Binder JR (2010) Tonotopic organization of human auditory cortex. NeuroImage 50:1202–1211. https://doi.org/10.1016/j.neuroimage.2010.01.046
Kriegeskorte N, Goebel R, Bandettini P (2006) Information-based functional brain mapping. Proc Natl Acad Sci U S A 103:3863–3868. https://doi.org/10.1073/pnas.0600244103
Kumar S, Stephan KE, Warren JD et al (2007) Hierarchical processing of auditory objects in humans. PLoS Comput Biol 3:0977–0985. https://doi.org/10.1371/journal.pcbi.0030100
Langner G (2009) A map of periodicity orthogonal to frequency representation in the cat auditory cortex. Front Integr Neurosci 3:27. https://doi.org/10.3389/neuro.07.027.2009
Leaver AM, Rauschecker JP (2010) Cortical representation of natural complex sounds: effects of acoustic features and auditory object category. J Neurosci 30:7604–7612. https://doi.org/10.1523/JNEUROSCI.0296-10.2010
Lemaitre G, Pyles JA, Halpern AR et al (2017) Who’s that knocking at my door? Neural bases of sound source identification. Cereb Cortex:1–14. https://doi.org/10.1093/cercor/bhw397
Liu D, Lu L, Zhang HJ (2003) Automatic mood detection from acoustic music data. In: Hoos HH, Bainbridge D (eds) Proceedings of the 4th international conference on music information retrieval, October 2013. Johns Hopkins University, Baltimore, p 81–87
Lewis JW (2005) Distinct cortical pathways for processing tool versus animal sounds. J Neurosci 25:5148–5158. https://doi.org/10.1523/JNEUROSCI.0419-05.2005
Lewis JW, Phinney RE, Brefczynski-Lewis JA, DeYoe EA (2006) Lefties get it “right” when hearing tool sounds. J Cogn Neurosci 18:1314–1330. https://doi.org/10.1162/jocn.2006.18.8.1314
McAdams S, Winsberg S, Donnadieu S et al (1995) Perceptual scaling of synthesized musical timbres: common dimensions, specificities, and latent subject classes. Psychol Res 58:177–192. https://doi.org/10.1007/BF00419633
McAvoy M, Larson-Prior L, Nolan TS et al (2008) Resting states affect spontaneous bold oscillations in sensory and paralimbic cortex. J Neurpophysiol 100(2):922–931
McDermott JH, Simoncelli EP (2011) Sound texture perception via statistics of the auditory periphery: evidence from sound synthesis. Neuron 71:926–940. https://doi.org/10.1016/j.neuron.2011.06.032
McDermott JH, Schemitsch M, Simoncelli EP (2013) Summary statistics in auditory perception. Nat Neurosci 16(4):493–498
Menon V, Levitin DJ, Smith BK et al (2002) Neural correlates of timbre change in harmonic sounds. NeuroImage 17:1742–1754. https://doi.org/10.1006/nimg.2002.1295
Miller LM, Escabí MA, Read HL, Schreiner CE (2002) Spectrotemporal receptive fields in the lemniscal auditory thalamus and cortex. J Neurophysiol 87:516–527. https://doi.org/10.1152/jn.00395.2001
Mlynarski W, McDermott JH (2018) Learning midlevel auditory codes from natural sound statistics. Neural Comput 30:631–669
Norman-Haignere S, Kanwisher NG, McDermott JH (2015) Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron 88:1281–1296. https://doi.org/10.1016/j.neuron.2015.11.035
Obleser J, Scott SK, Eulitz C (2006) Now you hear it, now you don’t: Tran- sient traces of consonants and their nonspeech analogues in the human brain. Cereb Cortex 16:1069–1076
Pallesen KJ, Brattico E, Bailey CJ, Korvenoja A, Gjedde A (2009) Cognitive and emotional modulation of brain default operation. J. Cogn. Neurosci. 21(6), 901 1065–1080. https://doi.org/10.1162/jocn.2009.21086
Pantev C, Engelien A, Candia V, Elbert T (2001) Representational cortex in musicians. Plastic alterations in response to musical practice. Ann N Y Acad Sci 930:300–314. https://doi.org/10.1111/j.1749-6632.2001.tb05740.x
Petersen S, Posner M (2012) The attention system of the human brain: 20 years after. Annu Rev Neurosci 21:73–89. https://doi.org/10.1146/annurev-neuro-062111-150525.The
Platel H, Price C, Baron JC, et al (1997) The structural components of music perception. A functional anatomical study. Brain 120 (Pt 2:229–243. doi: https://doi.org/10.1093/brain/120.2.229
Rauschecker JP (1998) Cortical processing of complex sounds. Curr Opin Neurobiol 8(4):516–521. https://doi.org/10.1016/S0959-4388(98)80040-8
Rauschecker JP (2012) Ventral and dorsal streams in the evolution of speech and language. Front Evol Neurosci 4:7. https://doi.org/10.3389/fnevo.2012.00007
Rauschecker JP, Scott SK (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat Neurosci 12:718–724. https://doi.org/10.1038/nn.2331
Ress D, Chandrasekaran B (2013) Tonotopic organization in the depth of human inferior colliculus. Front Hum Neurosci 7:586. https://doi.org/10.3389/fnhum.2013.00586
Rodriguez FA, Read HL, Escabi MA (2010) Spectral and temporal modulation tradeoff in the inferior colliculus. J Neurophysiol 103:887–903. https://doi.org/10.1152/jn.00813.2009
Samson S (2003) Neuropsychological studies of musical timbre. Ann N Y Acad Sci 999:144–151
Samson S, Zatorre RJ, Ramsay JO (2002) Deficits of musical timbre perception after unilateral temporal-lobe lesion revealed with multidimensional scaling. Brain 125:511–523. https://doi.org/10.1093/brain/awf051
Samson F, Zeffiro TA, Toussaint A, Belin P (2011) Stimulus complexity and categorical effects in human auditory cortex: an activation likelihood estimation meta-analysis. Front Psychol 1:241. https://doi.org/10.3389/fpsyg.2010.00241
Sandell GJ (1998) Macrotimbre: Contribution of attack and steady state, Proc. 16th Int. Congress on Acoustics and 135th Meeting of the Acoustical Society of America, Vol. 3, Seattle (Acoustical Society of America, Woodbury, NY), pp 1881–1882
Santoro R, Moerel M, De Martino F et al (2014) Encoding of natural sounds at multiple spectral and temporal resolutions in the human auditory cortex. PLoS Comput Biol 10(1):e1003412. https://doi.org/10.1371/journal.pcbi.1003412
Staeren N, Renvall H, De Martino F et al (2009) Sound categories are represented as distributed patterns in the human auditory cortex. Curr Biol 19:498–502. https://doi.org/10.1016/j.cub.2009.01.066
Stelzer J, Chen Y, Turner R (2013) Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control. NeuroImage 65:69–82. https://doi.org/10.1016/j.neuroimage.2012.09.063
Theunissen FE, Elie JE (2014) Neural processing of natural sounds. Nat Rev Neurosci 15:355–366
Toiviainen P, Tervaniemi M, Louhivuori J et al (1998) Timbre similarity: convergence of neural, behavioral, and computational approaches. Music Percept 16:223–241. https://doi.org/10.2307/40285788
Toiviainen P, Alluri V, Brattico E et al (2014) Capturing the musical brain with lasso: dynamic decoding of musical features from fMRI data. NeuroImage 88:170–180
Town SM, Bizley JK (2013) Neural and behavioral investigations into timbre perception. Front Syst Neurosci 7:1–14. https://doi.org/10.3389/fnsys.2013.00088
Uddin LQ, Clare Kelly A, Biswal BB, Xavier Castellanos F, Milham MP (2009) Functional connectivity of default mode network components: Correlation, anticorrelation, and causality. Hum. Brain Mapp., 30: 625–637. https://doi.org/10.1002/hbm.20531
Varoquaux G, Thirion B (2014) How machine learning is shaping cognitive neuroimaging. Gigascience 3:28
Warren JD, Jennings AR, Griffiths TD (2005) Analysis of the spectral envelope of sounds by the human brain. NeuroImage 24:1052–1057. https://doi.org/10.1016/j.neuroimage.2004.10.031
Wessinger CM, Vanmeter J, Tian B et al (2001) Hierarchical organization of the Human Auditory Cortex Revealed by functional magnetic resonance imaging. J Cogn Neurosci 13(1):1–7
Zatorre RJ, Belin P (2001) Spectral and temporal processing in human auditory cortex. Cereb Cortex 11:946–953. https://doi.org/10.1093/cercor/11.10.94
Acknowledgements
The authors would like to thank Prof. Stephen McAdams and Kai Siedenburg for their valuable inputs. V. A. extends her sincere gratitude to Prof. Elvira Brattico for opening the door to neuroscience and Prof. Petri Toiviainen for being the wonderful mentor that he is and providing never-ending support both professionally and personally.
Compliance with Ethics Requirements
Vinoo Alluri declares that she has no conflict of interest.
Sudarsana Reddy Kadiri declares that he has no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Alluri, V., Kadiri, S.R. (2019). Neural Correlates of Timbre Processing. In: Siedenburg, K., Saitis, C., McAdams, S., Popper, A., Fay, R. (eds) Timbre: Acoustics, Perception, and Cognition. Springer Handbook of Auditory Research, vol 69. Springer, Cham. https://doi.org/10.1007/978-3-030-14832-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-14832-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14831-7
Online ISBN: 978-3-030-14832-4
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)