Abstract
Recently a strong poker-playing algorithm called DeepStack was published, which is able to find an approximate Nash equilibrium during gameplay by using heuristic values of future states predicted by deep neural networks. This paper analyzes new ways of encoding the inputs and outputs of DeepStack’s deep counterfactual value networks based on traditional abstraction techniques, as well as an unabstracted encoding, which was able to increase the network’s accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
As in DeepStack, the inputs to the networks with 7 layers with 500 nodes each using parametric ReLUs and an outer network ensuring the zero-sum property are the respective encodings.
References
Bowling, M., Burch, N., Johanson, M., Tammelin, O.: Heads-up limit hold’em poker is solved. Commun. ACM 60(11), 81–88 (2017)
Burch, N., Bowling, M.: CFR-D: solving imperfect information games using decomposition. CoRR abs/1303.4441 (2013). http://arxiv.org/abs/1303.4441
Ganzfried, S., Sandholm, T.: Endgame solving in large imperfect-information games. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2015, pp. 37–45, Richland, SC (2015)
Gibson, R.: Regret minimization in games and the development of champion multiplayer computer poker-playing agents. Ph.D. thesis, University of Alberta (2014)
Gilpin, A., Sandholm, T., Sørensen, T.B.: A heads-up no-limit texas hold’em poker player: discretized betting models and automatically generated equilibrium-finding programs. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2, AAMAS 2008, pp. 911–918. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2008)
Hopner, P.: Analysis and optimization of deep counterfactual value networks. Bachelor’s thesis, Technische Universität Darmstadt (2018). http://www.ke.tu-darmstadt.de/bibtex/publications/show/3078
Hopner, P., Loza Mencía, E.: Analysis and optimization of deep counterfactual value networks (2018). http://arxiv.org/abs/1807.00900
Johanson, M.: Measuring the size of large no-limit poker games. CoRR abs/1302.7008 (2013). http://arxiv.org/abs/1302.7008
Johanson, M., Bard, N., Lanctot, M., Gibson, R., Bowling, M.: Efficient nash equilibrium approximation through Monte Carlo counterfactual regret minimization. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2, AAMAS 2012, pp. 837–846. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2012)
Johanson, M., Burch, N., Valenzano, R., Bowling, M.: Evaluating state-space abstractions in extensive-form games. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS 2013, pp. 271–278. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2013)
Johanson, M.B.: Robust strategies and counter-strategies: from superhuman to optimal play. Ph.D. thesis, University of Alberta (2016). http://johanson.ca/publications/theses/2016-johanson-phd-thesis/2016-johanson-phd-thesis.pdf
Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002). https://doi.org/10.1109/TPAMI.2002.1017616
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
Moravcík, M., et al.: Deepstack: expert-level artificial intelligence in no-limit poker. CoRR abs/1701.01724 (2017). http://arxiv.org/abs/1701.01724
Moravcík, M., et al.: Supplementary materials for deepstack: expert-level artificial intelligence in no-limit poker (2017). https://www.deepstack.ai/
Nash, J.: Non-cooperative games. Ann. Math. 54(2), 286–295 (1951)
Noam Brown, T.S.: Libratus: the superhuman AI for no-limit poker. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 5226–5228 (2017)
Schnizlein, D.P.: State translation in no-limit poker. Master’s thesis, University of Alberta (2009)
Tammelin, O.: Solving large imperfect information games using CFR+. CoRR abs/1407.5042 (2014). http://arxiv.org/abs/1407.5042
Zinkevich, M., Johanson, M., Bowling, M., Piccione, C.: Regret minimization in games with incomplete information. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems 20, pp. 1729–1736. Curran Associates, Inc. (2008). http://papers.nips.cc/paper/3306-regret-minimization-in-games-with-incomplete-information.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Hopner, P., Loza Mencía, E. (2018). Analysis and Optimization of Deep Counterfactual Value Networks. In: Trollmann, F., Turhan, AY. (eds) KI 2018: Advances in Artificial Intelligence. KI 2018. Lecture Notes in Computer Science(), vol 11117. Springer, Cham. https://doi.org/10.1007/978-3-030-00111-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-00111-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00110-0
Online ISBN: 978-3-030-00111-7
eBook Packages: Computer ScienceComputer Science (R0)