Abstract
We present experiments with a multimodal system based on a novel variant of the Self-Organizing Map (SOM) called the Associative Self-Organizing Map (A-SOM). The A-SOM is similar to the SOM and develops a representation of its input space, but also learns to associate its activity with additional inputs, e.g. the activities of one or several external SOMs. This enables the modelling of expectations in one sensory modality due to the activity elicited in another modality. The paper presents the A-SOM algorithm generalized to an arbitrary number of (possibly delayed) associated activities together with simulation results with a multimodal sensory system and its extension to a system that also includes an action network. The simulation results were very encouraging and confirmed: The ability of the A-SOM to learn to associate the representations of its input space with the representations of the input spaces developed in two connected SOMs; The ability of the extended system to elicit proper activity in the action network; The simulations demonstrated good generalization ability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balkenius, C., MorĂ©n, J., Johansson, B., Johnsson, M.: Ikaros: Building cognitive models for robots. In: HĂŒlse, M., Hild, M. (eds.) Workshop on current software frameworks in cognitive robotics integrating different computational paradigms (in conjunction with IROS 2008), Nice, France, pp. 47â54 (2008)
Bartolomeo, P.: The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence. Cortex 38, 357â378 (2002)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 3, 698â713 (1992)
Hesslow, G.: Conscious thought as simulation of behaviour and perception. Trends Cogn. Sci. 6, 242â247 (2002)
Johnsson, M., Balkenius, C.: Associating SOM representations of haptic submodalities. In: Ramamoorthy, S., Hayes, G.M. (eds.) Towards Autonomous Robotic Systems 2008, pp. 124â129 (2008)
Johnsson, M., Balkenius, C., Hesslow, G.: Neural network architecture for crossmodal activation and perceptual sequences. In: Papers from the AAAI Fall Symposium Biologically Inspired Cognitive Architectures 2009, pp. 85â86 (2009)
Kayser, C., Petkov, C.I., Augath, M., Logothetis, N.K.: Functional imaging reveals visual modification of specific fields in auditory cortex. J. Neurosci. 27, 1824â1835 (2007)
Kohonen, T.: Self-Organization and Associative Memory. Springer, Heidelberg (1988)
Kosslyn, S.M., Ganis, G., Thompson, W.L.: Neural foundations of imagery. Nature Rev. Neurosci. 2, 635â642 (2001)
McGurk, H., MacDonald, J.: Hearing lips and seeing voices. Nature 264, 746â748 (1976)
Nguyen, L.D., Woon, K.Y., Tan, A.H.: A self-organizing neural model for multimedia information fusion. In: International Conference on Information Fusion 2008, pp. 1738â1744 (2008)
Tan, A.H.: Adaptive resonance associative map. Neural Networks 8, 437â446 (1995)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Johnsson, M., Balkenius, C., Hesslow, G. (2011). Multimodal System Based on Self-organizing Maps. In: Madani, K., Correia, A.D., Rosa, A., Filipe, J. (eds) Computational Intelligence. IJCCI 2009. Studies in Computational Intelligence, vol 343. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20206-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-20206-3_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20205-6
Online ISBN: 978-3-642-20206-3
eBook Packages: EngineeringEngineering (R0)