Abstract
To date there have been few implementation of Holland’s Learning Classifier System (LCS) on real robots. The paper introduces a Temporal Classifier System (TCS), an LCS derived from Wilson’s ZCS. Traditional LCS have the ability to generalise over the state action-space of a reinforcement learning problem using evolutionary techniques. In TCS this generalisation ability can also be used to determine the state divisions in the state space considered by the LCS. TCS also implements components from Semi-Mark- Decision Process (SMDP) theory to weight the influence of time on the reward functions of the LCS. A simple light-seeking task on a real robot platform using TCS is presented which demonstrates desirable adaptive characteristics for the use of LCS on real robots.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ahluwalia, M & Bull, L (1999) A Genetic Programming Based Classifier System. In W. Banzhaf, J. Daida, A. E. Eihen, M. H. Garzon, V. Honavar, M. Jakiela & R. E. Smith (eds) Proceedings of the Genetic and Evolution Computation Conference-GECCO-99. Morgan Kaufmann, pp11–18.
Asada, M, Noda, S, Tawaratsumida, S Hosoda, K (1996) Purposive Behavior Acquisition for a Real Robot by a Vision Based Reinforcement Learning. Machine Learning Vol23 pp279–303.
Barry, A (2000) Specifying Action Persistence within XCS. In D. Whitley, D. Goldberg, E. Cantu-Paz, L. Spector, I. Parmee & H-G Beyer (eds) Proceedings of the Genetic and Evolutionary Computation Conference. Morgan Kaufmann, pp50–57
Booker, L (1990) Instinct as an inductive bias for learning behavioral Sequences In Meyer, J, A and Wilson (eds) From animals to animats-The first International Conference on Simulation of Adaptive Behavior, MIT Press
Bull, L., Hurst, J. & Tomlinson, A. (2000) Self-Adaptive Mutation in Classifier System Controllers. In J-A. Meyer, A. Berthoz, D. Floreano, H. Roitblatt & S. W. Wilson (eds) From Animals to Animats 6—The Sixth International Conference on the Simulation of Adaptive Behaviour, MIT Press.
Bull L & Hurst. J (2002) ZCS Redux EvolutionaryComputation In Press
Bradtke, S. J & Duff, M. O (1995) Reinforcement Learning Models for continuous-time Markov decision problems, Advances in Neural information Processing Systems 7 MIT Press pp393–400
Cobb, H & Grefenstette, J (1991)Learning the Persistance of Actions in Reactive Control Rules Proceedings of the 8th International Machine Learning Workshop. Morgan Kaufmann pp293–297
Cliff, D & Ross, S (1995) Adding Temporary Memory to ZCS Adaptive Behavior 3(2) 101–150
Dorigo, M, Colombetti, M (1997) Robot Shaping, MIT Press
Donnart, J. Y. Meyer, J. A. (1996b). Learning Reactive and Planning Rules in a Motivationally Autonomous Animat. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics. 26(3), 381–395.
Donnart, J. Y et Meyer, J. A. (1996a)Spatial exploration, map learning, and self-positioning with MonaLysa. In Maes, P., Mataric, M., Meyer, J. A., Pollack, J. et Wilson, S. (Eds.). From animals to animats 4. Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior. The MIT Press.
Doya, K (2000) Reinforcement learning in continuous time and space Neural Computation 12(1) 219–245
Holland, J. H. (1975) Adaptation in Natural and Artificial Systems. University of Michigan Press.
Hurst, J, Bull, L (2001) A Self-Adaptive Classifier System. In P-L Lanzi, W. Stolzmann & S. Wilson (eds) Proceedings of the Third International Workshop on Learning Classifier Systems.
Katagami, D, Yamada, S, (2002) Interactive Evolutionary Computation for Real Robot from a view point of observation. In Proceedings of the The 7th International Conference on Intelligent Autonomous Systems (IAS7)
Lanzi, P. L (1999) Extending the representation of Classifier Conditions from Messy Coding to S-Expressions. In W. Banzhaf, J. Daida, A. E. Eihen, M. H. Garzon, V. Honavar, M. Jakiela & R. E. Smith (eds) Proceedings of the Genetic and Evolution Computation Conference—GECCO-99. Morgan Kaufmann, pp345–352.
Nolfi, S, Floreano, D (2000) Evolutionary robotics, MIT Press
Parr R (1998) Hierarchical Control and Learning for Markov Decision Processes, Ph.D. Thesis, University of California, Berkeley.
Santamaria, J Suton, R and Ram, A (1998) Experiments with reinforcement learning in problems with continuous state and action spaces. Adaptive Behavior Vol 6 No 2 pp163–217
Stolzmann, Butz. M (2000) Latent learning and action planning in robots with anticipatory classifier systems In P-L. Lanzi, W. Stolzmann & W. Wilson (eds) Learning Classifier Systems: From Foundations to Applications. Springer.
Sutton, R & Barto, A (1998) Reinforcement Learning An Introduction, MIT Press
Sutton, R, Precup, D and Singh, S (1999) Between MDPs and semi-MDPs: A Framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112 pp181–211
Uchibi, Eiji, Asada, M and Hosoda, K (1997) Vision Based State Space Construction for Learning Mobile Robots in Multi-Agent Environments. Proceedings of Sixth European Workshop on Learning Robots (EWLR-6) pp33–41
Valenzuela-Rendon, M (1991) The Fuzzy Classifier System: a Classifier System for Continuously Varying Variables. Im L. Booker & R. Belew (eds) Proceedings of the Fourth International Conference on genetic Algorithms. Morgan Kaufmann pp346–353
Watkins, C (1989). Learning From Delayed Rewards. Phd. Cambridge University
Wilson, S. W (1994) ZCS: A Zeroth-level Classifier System. Evolutionary Computation 2(1):1–18
Wilson, S. W (2000) Get Real! XCS with Continuous Valued Inputs. In P-L. Lanzi, W. Stolzmann & W. Wilson (eds) Learning Classifier Systems: From Foundations to Applications. Springer, pp209–222.
Wilson, S. W (2001) Function Approximation with a Classifier System. In L. Spector, M. Gen, S. Sen, M. Dorigo, S, Pezeshk, M. Garzon & E. Burke (eds) Proceedings of the Genetic and Evolutionary Computation Conference-GECCO 2001. Morgan Kaufmann, pp974–984.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hurst, J., Bull, L., Melhuish, C. (2002). TCS Learning Classifier System Controller on a Real Robot. In: Guervós, J.J.M., Adamidis, P., Beyer, HG., Schwefel, HP., Fernández-Villacañas, JL. (eds) Parallel Problem Solving from Nature — PPSN VII. PPSN 2002. Lecture Notes in Computer Science, vol 2439. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45712-7_57
Download citation
DOI: https://doi.org/10.1007/3-540-45712-7_57
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44139-7
Online ISBN: 978-3-540-45712-1
eBook Packages: Springer Book Archive