Abstract
The neural network has been used extensively as a vehicle for both genetic algorithms and reinforcement learning. This paper shows a natural way to combine the two methods and suggests that reinforcement learning may be superior to random mutation as an engine for the discovery of useful substructures. The paper also describes a software experiment that applies this technique to produce an Othello-playing computer program. The experiment subjects a pool of Othello-playing programs to a regime of successive adaptation cycles, where each cycle consists of an evolutionary phase, based on the genetic algorithm, followed by a learning phase, based on reinforcement learning. A key idea of the genetic implementation is the concept of feature-level crossover. The regime was run for three months through 900,000 individual matches of Othello. It ultimately yielded a program that is competitive with a human-designed Othello-program that plays at roughly intermediate level.
Chapter PDF
References
G. Tesauro, “Temporal Difference Learning and TD-Gammon”. Communications of the ACM, vol. 38, no. 3, pp. 58–68, 1995
K. Chellapilla and D. B. Fogel, “Evolution, Neural Networks, Games, And Intelligence”. Proceedings of the IEEE, vol. 87,no. 9, pp. 1471–96, 1999
D. E. Moriarty and R. Miikkulainen, “Discovering Complex Othello Strategies Through Evolutionary Neural Networks”. Connection Science, vol. 7, no.3-4, pp. 195–209, 1995
Terry Jones, “Crossover, Macromutation, and Population-Based Search”, Proceedings of the Sixth International Conference on Genetic Algorithms
Dimitri Bertsekas and John Tsitsiklis, Neuro-Dynamic Programming. Belmont, Massachusetts: Athena Scientific, 1996.
John Koza, Genetic Programming. MIT, 1996
J. B. Pollack and A. D. Blair, “Co-Evolution in the Successful Learning of Backgammon Strategy”, Machine Learning, vol. 32,no. 3, pp. 225–40, 1998
Brian D. Ripley, Pattern Recognition And Neural Networks. Cambridge University Press, 1996.
Richard Sutton and Andrew Bartow, Reinforcement Learning. Windfall Software, 1999.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Singer, _.J.A. (2001). Co-evolving a Neural-Net Evaluation Function for Othello by Combining Genetic Algorithms and Reinforcement Learning. In: Alexandrov, V.N., Dongarra, J.J., Juliano, B.A., Renner, R.S., Tan, C.J.K. (eds) Computational Science - ICCS 2001. ICCS 2001. Lecture Notes in Computer Science, vol 2074. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45718-6_42
Download citation
DOI: https://doi.org/10.1007/3-540-45718-6_42
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42233-4
Online ISBN: 978-3-540-45718-3
eBook Packages: Springer Book Archive