Abstract
No-Limit Texas Hold’em Poker is a stochastic game of imperfect information. Each player receives cards dealt randomly and does not know which cards his opponents have been dealt. These simple features result in No-Limit Texas Hold’em Poker having a large decision space in comparison to other classic games such as Backgammon and Chess. Evolutionary algorithms and neural networks have been shown to find solutions in large and non-linear decision spaces and have proven to aid decision making in No-Limit Texas Hold’em Poker. In this paper, a hybrid method known as evolving neural networks is used by No-Limit Texas Hold’em Poker playing agents to make betting decisions. When selecting a new generation of agents, evolutionary forgetting can result in selecting an agent with betting behaviour that has previously been shown to be inferior. To prevent this from occurring, we utilize two heuristics: halls of fame and co-evolution. In addition, we evaluate agent fitness using three fitness functions based upon, respectively, the length of time an agent survives in a tournament, the number of hands won in a tournament, and the average amount of money won across all hands in a tournament. Results show that the length of time an agent survives is indeed an appropriate measure of fitness. Results also show that utilizing halls of fame and co-evolution serve to further improve the fitness of agents. Finally, through monitoring the evolutionary progress of agents, we find that the skill level of agents improves when using our evolutionary heuristics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barone, L., While, L.: An adaptive learning model for simplified poker using evolutionary algorithms. In: Proceedings of the Congress on Evolutionary Computation, vol. 1, pp. 153–160 (1999)
Beattie, B., Nicolai, G., Gerhard, D., Hilderman, R.: Pattern classification in No-Limit Poker: A head start evolutionary approach. In: Canadian Conference on AI, pp. 204–215 (2007)
Billings, D., Papp, D., Pena, L., Schaeffer, J., Szafron, D.: Using selective-sampling simulations in poker. In: AAAI Spring Symposium on Search Techniches for Problem Solving Under Uncertainty and Incomplete Information (1999)
Billings, D., Davidson, A., Schaeffer, J., Szafron, D.: The challenge of poker. Artificial Intelligence 134, 201–240 (2002)
Billings, D., Burch, N., Davidson, A., Holte, R., Schaeffer, J., Schauenberg, T., Szafron, D.: Approximating game-theoretic optimal strategies for full-scale poker. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (2003)
Billings, D.: Algorithms and Assessment in Computer Poker. PHD Dissertation. University of Alberta (2006)
Booker, L.: A No Limit Texas Hold’em poker playing agent. Master’s Thesis. University of London (2004)
Campbell, M., Hoane, A., Hsu, F.: Deep Blue. Artificial Intelligence 134, 57–83 (2002)
Donninger, C., Lorenz, U.: The Hydra project Xcell. Journal 53, 94–97 (2005)
Hauptman, A., Sipper, M.: GP-EndChess: Using genetic programming to evolve chess endgame players, booktitle. In: Proceedings of the 8th European Conference on Genetic Programming (2005)
Johanson, M.: Robust strategies and counter-strategies: Building a champion level computer poker player. Master’s thesis. University of Alberta (2007)
Kendall, G., Whitwell, G.: An evolutionary approach for tuning of a chess evaluation function using population dynamics. In: Proceedings of the 2001 IEEE Congress on Evolutionary Computation, pp. 995–1002 (2001)
Lubberts, A., Miikkulainen, R.: Co-evolving a go-playing neural network. In: Proceedings of the GECCO-01 Workshop on Coevolution: Turning Adaptive Algorithms Upon Themselves (2001)
Pollack, J., Blair, A.: Co-evolution in the successful learning of backgammon strategy. Machine Learning 32, 225–240 (1998)
Rosin, C.: Coevolutionary search among adversaries. PHD Dissertation. University fo California, San Diego (1997)
Samuel, A.L.: Some studies in machine learning using the game of checkers. IBM Journal of Research and Development (1959)
Schaeffer, J., Cublerson, J., Treloar, N., Knight, B., Lu, P., Szafron, D.: A word championship caliber checkers program. Artificial Intelligence 53, 273–289 (1992)
Schauenberg, T.: Opponent modelling and search in poker. Master’s thesis. University of Alberta (2006)
Tesauro, G.: Programming backgammon using self-teaching neural nets. Artificial Intelligence 134, 181–199 (2002)
Thrun, S.: Learning to play the game of chess. Advances in Neural Information Processing Systems 7, 1069–1076 (1995)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag GmbH Berlin Heidelberg
About this paper
Cite this paper
Nicolai, G., Hilderman, R. (2012). Countering Evolutionary Forgetting in No-Limit Texas Hold’em Poker Agents. In: Madani, K., Dourado Correia, A., Rosa, A., Filipe, J. (eds) Computational Intelligence. IJCCI 2010. Studies in Computational Intelligence, vol 399. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27534-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-27534-0_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-27533-3
Online ISBN: 978-3-642-27534-0
eBook Packages: EngineeringEngineering (R0)