An Integration Principle for Multimodal Sensor Data Based on Temporal Coherence of Self-Organized Patterns
The world around us offers continuously huge amounts of information, from which living organisms can elicit the knowledge and understanding they need for survival or well-being. A fundamental cognitive feature, that makes this possible is the ability of a brain to integrate the inputs it receives from different sensory modalities into a coherent description of its surrounding environment. By analogy, artificial autonomous systems are designed to record continuously large amounts of data with various sensors. A major design problem by the last is the lack of reference of how the information from the different sensor streams can be integrated into a consistent description. This paper focuses on the development of a sinergistic integration principle, supported by the synchronization of the multimodal information streams on temporal coherence principle. The processing of the individual information streams is done by a self organizing neural algorithm, known as Neural gas algorithm. The integration itself uses a supervised learning method to allow the various information streams to interchange their knowledge as emerged experts. Two complementary data streams, recorded by exploration of autonomous robot of unprepared environments are used to simultaneously illustrate and motivate in a concrete sense the developed integration approach.
KeywordsProcessing Stream Autonomous Robot Dynamic Trajectory Surrounding World Temporal Synchronization
Unable to display preview. Download preview PDF.
- Barakova, E.I. & Zimmer, U.R. Dynamical situation and trajectory discrimination by means of clustering and accumulation of raw range measurements, Proc. of the Intl. Conf. on Advances in Intelligent Systems, Canberra, Australia, February 2000.Google Scholar
- Barakova, E.I. & Zimmer, U.R. “Global spatial modelling based on dynamics identification according to discriminated static sensations” Underwater Technology 2000, Tokyo, Japan, May 23-26 2000.Google Scholar
- Grossberg, S. The complementary brain: A Unifying View of Brain Specialization and Modularity.Google Scholar
- von der Malsburg, C. The correlation theory of brain function., in Models of Neural networks II, edited by E. Domany, J.L. van Hemmen, and K. Schulten (Springer, Berlin, 1994) Chapter 2, pp. 95, 119.Google Scholar
- de Sa, V.R. and Dana H. Ballard, Category Learning through Multi-Modality Sensing, In Neural Computation 10(5) 1998.Google Scholar
- Triesch, J. (2000). Democratic Integration: A theory of adaptive sensory integration. Technical Report NRL TR 00.1, Nat’l. Resource Lab’y. for the Study of Brain and Behavior,U. Rochester.Google Scholar
- Zeki, S. Functional specialization in the visual cortex of the rhesus monkey, Nature (Lond.)274,423–8.Google Scholar