Abstract
This paper examines the performance of the ZCS Michigan-style classifier system in multi-agent environments. Using an abstract multi-agent model the effects of varying aspects of the performance, reinforcement and discovery components are examined. It is shown that small modifications to the basic ZCS architecture can improve its performance in environments with significant inter-agent dependence. Further, it is suggested that classifier systems have characteristics which make them more suitable to such non-stationary problem domains in comparison to other forms of reinforcement learning. Results from the initial use of ZCS as an adaptive economic trading agent within an artificial double-auction market are then presented, with the findings from the abstract model shown to improve the efficiency of the traders and hence the overall market.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
Arthur W B (1990), “A Learning Algorithm that Replicates Human Learning”, Technical Report 90-026, Santa Fe Institute.
Axlerod R (1987), “The Evolution of Strategies in the Iterated Prisoner's Dilemma,” in L Davis (ed.) Genetic Algorithms and Simulated Annealing, Pittman, pp32–42.
Booker L (1985), “Improving the Performance of Genetic Algorithms in Classifier Systems”, in J J Grefenstette (ed.) Proceedings of the First International Conference on Genetic Algorithms and their Applications, Lawrence Erlbaum, pp80–93.
Booker L (1989), “Triggered Rule Discovery in Classifier Systems”, in J D Schaffer (ed.) Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, pp265–275.
Bull L (1998), “Evolutionary Computing in Multi-Agent Environments: Operators,” in V W Porto, N Saravanan, D E Waagen & A E Eiben (eds.) Proceedings of the Seventh Annual Conference on Evolutionary Programming, Springer Verlag, to appear.
Bull L (1997), “Evolutionary Computing in Multi-Agent Environments: Partners,” in T Baeck (ed.) Proceedings of the Seventh International Conference on Genetic Algorithms, Morgan Kaufmann, pp370–377.
Bull L, Fogarty T C & Snaith M (1995), “Evolution in Multi-Agent Systems: Evolving Communicating Classifier Systems for Gait in a Quadrupedal Robot,” in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp382–388.
Carse B, Fogarty T C & Munro A (1995), “Adaptive Distributed Routing using Evolutionary Fuzzy Control”, in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp389–397.
Cedeno W & Vemuri V (1997), “On the Use of Niching for Dynamic Landscapes”, in Proceedings of the 1997 IEEE International Conference on Evolutionary Computation, IEEE, pp361–366.
Cliff D & Bruten J (1997), “Zero is Not Enough: On the Lower Limit of Agent Intelligence for Continuous Double Auction Markets”, HP Laboratories Technical Report HPL-97-141, HP Laboratories Bristol.
Dorigo M & Schnepf U (1992), “Genetics-based Machine Learning and Behaviour-based Robotics: A New Synthesis”, IEEE Trans. on Sys. Man and Cybernetics 22(6):141–154.
Dorigo M & Bersini H (1994), “A Comparison of Q-learning and Classifier Systems”, in D Cliff, P Husbands, J-A Meyer & S W Wilson (eds.) From Animals to Animats 3, MIT Press, pp248–255.
Dworman G (1994), “Games Computers Play: Simulating Characteristic Function Game Playing Agents with Classifier Systems”, in Proceedings of the 1994 IEEE Conference on Evolutionary Computing, IEEE.
Holland J H (ed.)(1975), Adaptation in Natural and Artificial Systems, University of Michigan Press.
Holland J H, Holyoak K J, Nisbett R E & Thagard P R (eds.)(1986), Induction: Processes of Inference, Learning and Discovery, MIT Press.
Kauffman S A (ed.)(1993), The Origins of Order: Self-organisation and Selection in Evolution, Oxford University Press.
Lin L-J (1992), “Self-improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching”, Machine Learning 8(3):293–322.
Marengo L & Tordjman H (1996), “Speculation, Heterogeneity and Learning: A Model of Exchange Rate Dynamics” KYKLOS 49(3):407–438.
Marimon R, McGrattan E & Sargent T (1990), “Money as a Medium of Exchange in an Economy with Artificially Intelligent Agents”, Economic Dynamics and Control (14):329–373.
Mitlohner J (1996), “Classifier Systems and Economic Modelling” APL Quote Quad 26(4)
Palmer R, Arthur W B, Holland J H, LeBaron B & Tayler P (1994), “Artificial Economic Life: A Simple Model of a Stockmarket”, Physica D 75:264–274.
Potter M, De Jong K & Grefenstette J (1995), “A Coevolutionary Approach to Learning Sequential Decision Rules”, in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp366–372.
Sandholm T & Crites R H (1995), “Multiagent Reinforcement Learning in the Iterated Prisoner's Dilemma”, BioSystems 37: 147–166.
Seredynski F, Cichosz P & Klebus G (1995), “Learning Classifier Systems in Multi-Agent Environments”, in Proceedings of the First IEE/IEEE Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, IEE, pp287–292.
Smith S F (1980), “A Learning System Based on Genetic Adaptive Algorithms”, PhD dissertation, University of Pittsburgh.
Smith V (ed.)(1992), Papers in Experimental Economics, Cambridge Press.
Syswerda G (1989), “Uniform Crossover in Genetic Algorithms”, in J D Schaffer (ed.) Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, pp2–9.
Tsetlin M (ed.)(1973), Automaton Theory and Modeling of Biological Systems, Academic Press.
Watkins C (1989), “Learning from Delayed Rewards”, PhD dissertation, University of Cambridge.
Weiss G (ed.)(1997), Distributed Artificial Intelligence Meets Machine Learning, Springer.
Wilson S W (1994), “ZCS: A Zeroth-level Classifier System”, Evolutionary Computation 2(1):1–18.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bull, L. (1998). On ZCS in multi-agent environments. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, HP. (eds) Parallel Problem Solving from Nature — PPSN V. PPSN 1998. Lecture Notes in Computer Science, vol 1498. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0056889
Download citation
DOI: https://doi.org/10.1007/BFb0056889
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65078-2
Online ISBN: 978-3-540-49672-4
eBook Packages: Springer Book Archive