Abstract
In this paper we present a new version of our previous work on a maze learning animat. Its sensory/motor capabilities have been extended and modified so that they are more biologically plausible than before. The animat's learning architecture is based around a hybrid RBF Neural Network/Evolutionary Strategy implementation of an Adaptive Heuristic Critic. We conduct experiments in which the animat either acquires persistent but undetectable internal errors in its sensory equipment, or operates in an environment where undetectable factors influence motor actions. We also observe the effects of random sensory errors on the usefulness of the information which the animat acquires. Through interactions with its environment the animat learns a subjective “cognitive map” which is a fusion of the features in its surroundings, the path to a goal state, and the errors/environmental influences which it cannot directly detect. We find that despite the subjective nature of the map it remains useful under quite high levels of error/distortion in our experiments.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Barto A. G., Bradtke S. J., Singh S. P., 1991, ‘Real-Time Learning and Control using Asynchronous Dynamic Programming', Dept. of Computer Science, University of Massachusetts, USA, Tech. Report 91-57
Barto A. G., Sutton R. S., Watkins C. J. C. H., 1989, ‘Learning and Sequential Decision Making', COINS Technical Report 89-95
Booker L. B., Goldberg D. E., Holland J. H., 1989, ‘Classifier Systems and Genetic Algorithms', Artificial Intelligence 40, pp.235–282
Cliff D., Harvey I., Husbands P., 1993, ‘Explorations in Evolutionary Robotics', Journal of Adaptive Behaviour, 2(1), pp.71–104
ECAL I, 1991, ‘Towards a Practice of Autonomous Systems', Proceedings of the First & Second European Conference on Artificial Life, Eds. Varela F. J., Bourgine P., MIT Press
ECAL II, 1993, Proceedings of the Second European Conference on Artificial Life, MIT Press
Gallistel C. R., 1990, ‘The Organization of Learning', MIT Press
Grefenstette J. J., 1991, ‘Lamarckian Learning in Multi-agent Environments', Proceedings of the Fourth International Conference on Genetic Algorithms, Morgan-Kaufmann, pp.303–310
Lin L., PhD thesis, 1993, ‘Reinforcement Learning for Robots using Neural Networks', School of Computer Science, Carnegie Mellon University Pittsburgh, USA
Pipe A. G. 1, Carse B., 1994, 'A Comparison between Two Architectures for Searching and Learning in Maze Problems', Selected papers from AISB Workshop in Evolutionary Computation, Springer-Verlag Lecture Notes in Computer Science #865, pp.238–249
Pipe A. G. 2, Fogarty T. C., Winfield A., 1994, ‘A Hybrid Architecture for Learning Continuous Environmental Models in Maze Problems', From Animals to Animats 3, Proceedings of third International Conference on Simulation of Adaptive Behaviour, Eds. Cliff D., Husbands P., Meyer J-A., Wilson S. W., MIT Press, pp. 198–205
Pipe A. G. 3, Fogarty T. C., Winfield A., 1994, ‘Hybrid Adaptive Heuristic Critic Architectures for Learning in Mazes with Continuous Search Spaces', Parallel Problem Solving from Nature (PPSNIII), Proceedings of the third International Conference on Evolutionary Computation, Springer-Verlag Lecture Notes in Computer Science #866, pp.482–491
Roitblat H. L., 1994, ‘Mechanism and Process in Animal Behaviour: Models of Animals, Animals as Models', From Animals to Animats 3, Proceedings of third International Conference on Simulation of Adaptive Behaviour, Eds. Cliff D., Husbands P., Meyer J-A., Wilson S. W., MIT Press, pp. 12–21
Roberts G., 1993, ‘Dynamic Planning for Classifier Systems', Proceedings of the 5th International Conference on Genetic Algorithms, pp.231–237
SAB92, From Animals to Animats 2, Proceedings of the Seconds International Conference on Simulation of Adaptive Behaviour, Eds. Meyer J-A., Roitblat H. L., Wilson S. W., MIT Press
SAB94, From Animals to Animats 3, Proceedings of third International Conference on Simulation of Adaptive Behaviour, Eds. Cliff D., Husbands P., Meyer J-A., Wilson S. W., MIT Press
Sutton R. S., 1984, PhD thesis ‘Temporal Credit Assignment in Reinforcement Learning', University of Massachusetts, Dept. of computer and Information Science
Sutton R. S., 1991, ‘Reinforcement Learning Architectures for Animats', From Animals to Animats, pp288–296, Editors Meyer, J., Wilson, S., MIT Press
Tolman E. C., Ritchie B. F., Kalish D., 1946, ‘Studies in Spatial Learning I. Orientation and the Short-Cut', Journal of Experimental Psychology #36, pp. 13–24
Watkins C. J. C. H., 1989, PhD thesis ‘Learning from Delayed Rewards', King's College, Cambridge.
Werbos, P. J., 1992, ‘Approximate Dynamic Programming for Real-Time Control and Neural Modelling', Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Van Nostrand Reinhold, Ed. White D. A., Sofge D. A.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pipe, A.G., Carse, B., Fogarty, T.C., Winfield, A. (1995). Learning subjective “cognitive maps” in the presence of sensory-motor errors. In: Morán, F., Moreno, A., Merelo, J.J., Chacón, P. (eds) Advances in Artificial Life. ECAL 1995. Lecture Notes in Computer Science, vol 929. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59496-5_318
Download citation
DOI: https://doi.org/10.1007/3-540-59496-5_318
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-59496-3
Online ISBN: 978-3-540-49286-3
eBook Packages: Springer Book Archive