Abstract
We present a recurrent multimodal model of binding written words to mental objects and investigate the capability of the network in reading misspelt but categorically related words. Our model consists of three mutually interconnected association modules which store mental objects, represent their written names and bind these together to form mental concepts. A feedback gain controlling top-down influence is incorporated into the model architecture and it is shown that correct settings for this during map formation and simulated reading experiments is necessary for correct interpretation and semantic binding of the written words.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bright, P., Moss, H.E., Tyler, L.K.: Unitary vs multiple semantics: PET studies of word and picture processing. Cerebral Cortex 89, 417–432 (2004)
Chou, S., Papliński, A.P., Gustafsson, L.: Speaker-dependent bimodal integration of Chinese phonemes and letters using multimodal self-organizing networks. In: Proc. Int. Joint Conf. Neural Networks, Orlando, Florida (August 2007)
Davis, C.J., Bowers, J.S.: Contrasting five different theories of letter position coding: Evidence from orthographic similarity effects. J. Exp. Psych.: Human Perception and Performance 32(3), 535–557 (2006)
Dehaene, S.: Reading in the Brain. Viking (2009), http://pagesperso-orange.fr/readinginthebrain/figures.htm
Glezer, L.S., Jiang, X., Riesenhuber, M.: Evidence for highly selective neuronal tuning to whole words in the “Visual Word Form Area”. Neuron 62(2), 199–204 (2009)
Gliozzi, V., Mayor, J., Hu, J.F., Plunkett, K.: Labels as features (not names) for infant categorisation: A neuro-computational approach. Cog. Sci. 33(3), 709–738 (2009)
Graboi, D., Lisman, J.: Recognition by top-down and bottom-up processing in cortex: The control of selective attention. J. Neurophysiol. 90, 798–810 (2003)
Grainger, J.: Cracking the orthographic code: An introduction. Language and Cogn. Processes 23(1), 1–35 (2007)
Jantvik, T., Gustafsson, L., Papliński, A.P.: A self-organized artificial neural network architecture for sensory integration with applications to letter–phoneme integration. Neural Computation, 1–39 (2011), doi:10.1162/NECO_a_00149
Kohonen, T.: Self-Organising Maps, 3rd edn. Springer, Berlin (2001)
Li, P., Zhao, X., MacWhinney, B.: Dynamic self-organization and early lexical development in children. Neuron 31, 581–612 (2007)
Mayor, J., Plunkett, K.: A neurocomputational account of taxonomic responding and fast mapping in early word learning. Psychol. Rev. 117(1), 1–31 (2010)
McCandliss, B.D., Cohen, L., Dehaene, S.: The visual word form area: expertise for reading in the fusiform gyrus. TRENDS Cog. Sci. 7(7), 293–299 (2003)
Miikkulainen, R.: Dyslexic and category-specific aphasic impairments in a self-organizing feature map model of the lexicon. Brain and Language, 334–366 (1997), http://nn.cs.utexas.edu/miikkulainen:bl97
Miikkulainen, R., Kiran, S.: Modeling the Bilingual Lexicon of an Individual Subject. In: Príncipe, J.C., Miikkulainen, R. (eds.) WSOM 2009. LNCS, vol. 5629, pp. 191–199. Springer, Heidelberg (2009)
Monner, D., Reggia, J.A.: An unsupervised learning method for representing simple sentences. In: Proc. Int. Joint Conf. Neural Net., Atlanta, USA, pp. 2133–2140 (June 2009)
Noppeney, U., Josephs, O., Hocking, J., Price, C., Friston, K.: The effect of prior visual information on recognition of speech and sounds. Cerebral Cortex 18, 598–609 (2008)
Papliński, A.P., Gustafsson, L.: Feedback in Multimodal Self-organizing Networks Enhances Perception of Corrupted Stimuli. In: Sattar, A., Kang, B.-h. (eds.) AI 2006. LNCS (LNAI), vol. 4304, pp. 19–28. Springer, Heidelberg (2006)
Papliński, A.P., Gustafsson, L., Mount, W.M.: A model of binding concepts to spoken names. Aust. Journal of Intelligent Information Processing Systems 11(2), 1–5 (2010)
Whitney, C.: Comparison of the SERIOL and SOLAR theories of letter-position encoding. Brain and Language 107, 170–178 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Papliński, A.P., Gustafsson, L., Mount, W.M. (2011). A Recurrent Multimodal Network for Binding Written Words and Sensory-Based Semantics into Concepts. In: Lu, BL., Zhang, L., Kwok, J. (eds) Neural Information Processing. ICONIP 2011. Lecture Notes in Computer Science, vol 7062. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24955-6_50
Download citation
DOI: https://doi.org/10.1007/978-3-642-24955-6_50
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24954-9
Online ISBN: 978-3-642-24955-6
eBook Packages: Computer ScienceComputer Science (R0)