A Combined Position Evaluation Function in Chinese Chess Computer Game

  • Yulin He
  • Xizhao Wang
  • Tingting Fu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7420)


In Chinese-chess computer game (CCCG), the position evaluation function plays a crucial role in building a game playing program. Traditionally, there are two typical types of evaluation functions: standard heuristic evaluation function (SHEF) and self learning evaluation function (SLEF). The SHEF depends on the board position features to large extent, but it hardly includes all the features due to the limit of knowledge of the designer. The SLEF can explore the knowledge hidden in the current position which is difficult to find in the SHEF. In this paper, a combined position evaluation function (CPEF) is designed by weighted sum of SHEF and SLEF. SHEF considers the material balance and adjunctive value of position while SLEF takes the form of a three-layer fully-connected feed forward neural network. We use temporal difference learning (TDL) to train the neural network on professional game records. Based on the combined position evaluation function, a Chinese chess program HBUCHESS is developed. We experimentally validate that our CPEF is quite effective through competing with different kinds of testing players. With the help of CPEF, the intelligent level of HBUCHESS can be improved incrementally with the increase of number of professional game records SLEF learned. Furthermore, in the process of learning professional game records, we find that the performance of HBUCHESS is mainly relevant to the following four aspects: (1) the initial heuristic knowledge, (2) the number of nodes in hidden layer of neural network, (3) the trace decay parameter λ, and (4) the learning rate α.


Chinese-chess computer game ensemble position evaluation function standard heuristic evaluation function self-learning evaluation function temporal difference learning neural network professional game records 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shannon, C.E.: Programming a Computer for Playing Chess. Philos. Mag. 41, 256–275 (1950)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Gillogly, J.J.: The Technology Chess Program. Artif. Intell. 3, 145–163 (1972)zbMATHCrossRefGoogle Scholar
  3. 3.
    Tesauro, G.: TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play. Neural Comput. 6, 215–219 (1994)CrossRefGoogle Scholar
  4. 4.
    Samuel, A.L.: Some Studies in Machine Learning Using the Game of Checkers. IBM J. Res. Dev. 3, 210–229 (1959)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Sutton, R.S.: Learning to Predict by the Methods of Temporal Differences. Mach. Learn. 3, 9–44 (1988)Google Scholar
  6. 6.
    Tesauro, G.: Practical Issues in Temporal Difference Learning. Mach. Learn. 8, 257–277 (1992)zbMATHGoogle Scholar
  7. 7.
    Tesauro, G.: Temporal Difference Learning and TD-Gammon. Commun. ACM 38, 58–68 (1995)CrossRefGoogle Scholar
  8. 8.
    Schraudolph, N.N., Dayan, P., Sejnowski, T.J.: Temporal Difference Learning of Position Evaluation in the Game of Go. Adv. Neur. Infor. Proc. Syst. 6, 817–824 (1994)Google Scholar
  9. 9.
    Enzenberger, M.: The Integration of a Priori Knowledge into a Go Playing Neural Network (1996),
  10. 10.
    Baxter, J., Trigdell, A., Weaver, L.: Knightcap: a Chess Program that Learns by Combining TD(λ) with Game-Tree Search. In: Proc. Inter. Conf. Mach. Learn., pp. 28–36 (1998)Google Scholar
  11. 11.
    Baxter, J., Tridgell, A., Weaver, L.: Learning To Play Chess Using Temporal Differences. Mach. Learn. 40, 243–263 (2001)CrossRefGoogle Scholar
  12. 12.
    Trinh, T.B., Bashi, A.S., Deshpande, N.: Temporal Difference Learning in Chinese Chess. In: Mira, J., Moonis, A., de Pobil, A.P. (eds.) IEA/AIE 1998. LNCS, vol. 1416, pp. 612–618. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  13. 13.
    Olson, D.K.: Learning to Play Games from Experience: An Application of Artificial Neural Networks and Temporal Difference Learning. Master’s thesis. Pacific Lutheran University, Washington (1993)Google Scholar
  14. 14.
    Thrun, S.: Learning to Play the Game of Chess. Adv. Neur. Infor. Proc. Syst. 7, 1069–1076 (1995)Google Scholar
  15. 15.
    Chellapilla, K., Fogel, D.: Evolving Neural Network to Play Checkers without Expert Knowledge. IEEE Trans. Neur. Netw. 10, 1382–1391 (1999)CrossRefGoogle Scholar
  16. 16.
    Schaeffer, J., Hlynak, M., Jussila, V.: Temporal Difference Learning Applied to a High-Performance Game-Playing Program. In: Inter. Joint Conf. Artif. Intell., pp. 529–534 (2001)Google Scholar
  17. 17.
    Ghory, I.: Reinforcement Learning in Board Games. Technical Report: CSTR-04-004, Department of Computer Science, University of Bristol (2004)Google Scholar
  18. 18.
    Wiering, M.A., Patist, J.P.: Learning to Play Board Games using Temporal Difference Methods. Technical Report: UU-CS-2005-048. Institute of Information and Computing Sciences, Utrecht University (2005)Google Scholar
  19. 19.
    Tu, Z.J.: Design and Implementation of Computer Chess. Master thesis. Zhongshan University (2004)Google Scholar
  20. 20.
    Xu, X.H., Jiao Wang, J.: Key Technologies Analysis of Chinese Chess Computer Game. Mini-Micro Syst. 27, 961–969 (2006)Google Scholar
  21. 21.
    Wei, C.B.: Study and Implementation of Chinese Chess Computer Game System. Master thesis. Kunming University of Science and Technology (2008)Google Scholar
  22. 22.
    Qiang, F.: The Research of Chinese Chess Based On Reinforcement Learning. Master thesis. Changsha University of Science and Technology (2006)Google Scholar
  23. 23.
    Wang, Y.F.: Research and Implementation of a Chinese Chess Computer Game System with Self Learning. Master thesis. Harbin Engineering University (2007)Google Scholar
  24. 24.
    Mitchell, T.M.: Machine Learning. The McGraw-Hill Companies, Inc. (1997)Google Scholar
  25. 25.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  26. 26.
    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. J. of Artif. Intell. Res. 4, 237–285 (1996)Google Scholar
  27. 27.
    Lau, H.T.: Chinese Chess: An Introduction to China’s Ancient Game of Strategy. Tuttle Publishing (2003)Google Scholar
  28. 28.
    Greenblatt, R.D., Eastlake, D.E., Crocker, S.D.: The Greenblatt Chess Program. In: Fall Joint Compu. Conf. Proc., pp. 801–810 (1967)Google Scholar
  29. 29.
    Hsu, S.C.: Past and Future of Computer Chess and Computer Chinese Chess. J. Compu. 2, 1–8 (1990)Google Scholar
  30. 30.
    Knuth, D.E., Moore, R.W.: An Analysis of Alpha-Beta Pruning. Artif. Intell. 6, 293–326 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Schaeffer, J.: The History Heuristic and Alpha-Beta Search Enhancement in Practice. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1203–1212 (1989)CrossRefGoogle Scholar
  32. 32.
    Marsland, T.A.: A Review of Game-Tree Pruning. ICCA Journal 9, 3–19 (2001)Google Scholar
  33. 33.
    Su, P., Li, Y., Li, Y.J., Shiu, S.C.K.: An auto-adaptive convex map generating path-finding algorithm: Genetic Convex A*. Inter. J. Mach. Learn. & Cyber. (2012), doi: 10.1007/s13042-012-0120-xGoogle Scholar
  34. 34.
    Chen, W.Y., Heish, S.Y., Yen, C.Y., Kuo, D.Y.: The Chinese-Chess Image Identification Techniques on Spatial Domain. In: Proc. 2011 World Congr. Intell. Control and Autom., pp. 970–974 (2011)Google Scholar
  35. 35.
    Su, K.L., Chung, C.Y., Zou, J.T., Hsu, T.Y.: A* Search Algorithm Applied to a Chinese Chess Game. Artif. Life & Robot. 16, 132–136 (2011)CrossRefGoogle Scholar
  36. 36.
    Chen, B.N., Liu, P.F., Hsu, S.C., Hsu, T.S.: Aggregating Consistent endgame knowledge in Chinese Chess. Knowledge-Based Syst. (2012), doi: 10.1016/j.knosys.2011.11.020Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Yulin He
    • 1
  • Xizhao Wang
    • 1
  • Tingting Fu
    • 1
    • 2
  1. 1.College of Mathematics and Computer ScienceHebei UniversityBaodingChina
  2. 2.Department of Computer ScienceBeijing JiaoTong University Haibin CollegeHuanghuaChina

Personalised recommendations