Neural Learning of Heuristic Functions for General Game Playing

  • Leo Ghignone
  • Rossella CancelliereEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10122)


The proposed model represents an original approach to general game playing, and aims at creating a player able to develop a strategy using as few requirements as possible, in order to achieve the maximum generality. The main idea is to modify the known minimax search algorithm removing its task-specific component, namely the heuristic function: this is replaced by a neural network trained to evaluate the game states using results from previous simulated matches. A method for simulating matches and extracting training examples from them is also proposed, completing the automatic procedure for the setup and improvement of the model. Part of the algorithm for extracting training examples is the Backward Iterative Deepening Search, a new original search algorithm which aims at finding, in a limited time, a high number of leaves along with their common ancestors.


Game playing Neural networks Reinforcement learning Online learning 


  1. 1.
    Silver, D., Huang, A., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)CrossRefGoogle Scholar
  2. 2.
    Draper, S., Rose, A.: Sancho GGP player.
  3. 3.
    Michulke, D.: Neural networks for high-resolution state evaluation in general game playing. In: IJCAI-11 Workshop on General Game Playing (GIGA11), pp. 31–37 (2011)Google Scholar
  4. 4.
    Schiffel, S., Thielscher, M.: Fluxplayer: a successful general game player. In: 22nd National Conference on Artificial intelligence, pp. 1191–1196. AAAI Press, Menlo Park (2007)Google Scholar
  5. 5.
    Świechowski, M., Mańdziuk, J.: Specialized vs. multi-game approaches to AI in games. In: Angelov, P., et al. (eds.) Intelligent Systems 2014. AISC, vol. 322, pp. 243–254. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-11313-5_23 Google Scholar
  6. 6.
    Świechowski, M., Park, H., Mańdziuk, J., Kim, K.-J.: Recent advances in general game playing. Sci. World J. 2015, Article ID 986262, 22 p. (2015). doi: 10.1155/2015/986262
  7. 7.
    Schmidt, W.F., Kraaijveld, M., Duin, R.P.W., et al.: Feedforward neural networks with random weights. In: International Conference on Pattern Recognition, Conference B: Pattern Recognition Methodology and Systems, pp. 1–4 (1992)Google Scholar
  8. 8.
    Pao, Y.H., Park, G.H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 163–180 (1994)CrossRefGoogle Scholar
  9. 9.
    Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17, 879–892 (2006)CrossRefGoogle Scholar
  10. 10.
    Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Netw. 17, 1411–1423 (2006)CrossRefGoogle Scholar
  11. 11.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice-Hall, Egnlewood Cliffs (1995)zbMATHGoogle Scholar
  12. 12.
    Penrose, R.: On best approximate solutions of linear matrix equations. Math. Proc. Camb. Philos. Soc. 52, 17–19 (1956)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Bishop, C.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  14. 14.
    Gherrity, M.: A game-learning machine. Ph.D. thesis, University of California, San Diego (1993)Google Scholar
  15. 15.
    Allis, L.W.: A knowledge-based approach of connect-four. Technical report, Vrije Universiteit, Subfaculteit Wiskunde en Informatica (1988)Google Scholar
  16. 16.
    British Othello Federation: Game Rules.
  17. 17.
    Cirasella, J., Kopec, D.: The History of Computer Games. CUNY Academic Works, New York (2006)Google Scholar
  18. 18.
    Mitchell, D.H.: Using features to evaluate positions in experts’ and novices’ Othello games. Masters thesis, Northwestern University, Evanston (1984)Google Scholar
  19. 19.
    MacGuire, S.: Strategy Guide for Reversi and Reversed Reversi.
  20. 20.
    Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of TurinTorinoItaly

Personalised recommendations