Abstract
The assessment of chess players is both an increasingly attractive opportunity and an unfortunate necessity. The chess community needs to limit potential reputational damage by inhibiting cheating and unjustified accusations of cheating: there has been a recent rise in both. A number of counter-intuitive discoveries have been made by benchmarking the intrinsic merit of players’ moves: these call for further investigation. Is Capablanca actually, objectively the most accurate World Champion? Has ELO rating inflation not taken place? Stimulated by FIDE/ACP, we revisit the fundamentals of the subject to advance a framework suitable for improved standards of computational experiment and more precise results. Other games and domains look to chess as demonstrator of good practice, including the rating of professionals making high-value decisions under pressure, personnel evaluation by Multichoice Assessment and the organization of crowd-sourcing in citizen science projects. The ‘3P’ themes of performance, prediction and profiling pervade all these domains.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
This probably increased the apparent competence ac of RP e : draws exhibited higher ac.
- 2.
Over the last four years, the winners of the TCEC events [24] have been Houdini 1.5a, Houdini 3, Komodo 1142, Stockfish 170514 and Komodo 1333.
- 3.
UCI = Universal Chess Interface [19].
- 4.
An example of chess-position in XML format is given in [2].
- 5.
The notation here is: 〈 means ‘worse than’ and 〈〈 means ‘much worse than’.
References
Andrist, R.B., Haworth, G.McC.: Deeper model endgame analysis. Theor. Comput. Sci. 349(2), 158–167 (2005). doi:10.1016/j.tcs.2005.09.044. ISSN 0304-3975
Barnes, D.J., Hernandez-Castro, J.: On the limits of engine analysis for cheating detection in chess. Comput. Secur. 48, 58–73 (2015)
Bayes, T.: An essay towards solving a problem in the doctrine of chances. Phil. Trans. Royal Soc. 53, 370–418 (1763). doi:10.1098/rstl.1763.0053
Biswas, T., Regan, K.W.: Quantifying depth and complexity of thinking and knowledge. In: ICAART 2015, the 7th International Conference on Agents and Artificial Intelligence, January 2015, Lisbon, Portugal (2015)
Di Fatta, G., Haworth, G.McC.: Skilloscopy: Bayesian modeling of decision makers’ skill. IEEE Trans. Syst. Man Cybern. Syst. 43(6), 1290–1301 (2013). doi:10.1109/TSMC.2013.2252893. ISSN 0018-9472
Di Fatta, G., Haworth, G.McC., Regan, K.: Skill rating by bayesian inference. In: IEEE CIDM Symposium on Computational Intelligence and Data Mining (2009)
Ferreira, D.R.: The impact of search depth on chess playing strength. ICGA J. 36(2), 67–90 (2013)
Ferreira, D.R.: Determining the strength of chess players based on actual play. ICGA J. 35(1), 3–19 (2012)
FIDE Online Arena with AceGuard: http://www.fide.com/fide/7318-fide-online-arena.html, http://arena.myfide.net/ (2015)
FIDE/ACP. Anti-Cheating Guidelines approved by FIDE (2014). http://www.fide.com/images/-stories/NEWS_2014/FIDE_news/4th_PB_Sochi_Agenda_Minutes/Annex_50.pdf
Friedel, F.: Cheating in chess. In: Advances in Computer Games, vol. 9, pp. 327 − 346. Institute for Knowledge and Agent Technology (IKAT), Maastricht (2001)
Guid, M., Perez, A., Bratko, I.: How trustworthy is Crafty’s analysis of chess champions? ICGA J. 31(3), 131–144 (2008)
Guid, M., Bratko, I.: Computer analysis of world chess champions. ICGA J. 29(2), 65–73 (2006)
Haworth, G.McC., Biswas, T., Regan, K.W.: This article and related, evolving datasets including pgn and statistics files (2015). http://centaur.reading.ac.uk/39431/
Haworth, G.McC, Regan, K., Di Fatta, G.: Performance and prediction: bayesian modelling of fallible choice in chess. In: van den Herik, H.J., Spronck, P. (eds.) ACG 2009. LNCS, vol. 6048, pp. 99–110. Springer, Heidelberg (2010)
Haworth, G.McC.: Gentlemen, stop your engines! ICGA J. 30(3), 150–156 (2007)
Haworth, G.McC., Andrist, R.B.: Model endgame analysis. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) Advances in Computer Games: Many Games, Many Challenges, vol. 135(10), pp. 65–79. Kluwer Academic Publishers, Norwell (2004). ISBN 9781402077098
Haworth, G.McC.: Reference fallible endgame play. ICGA J. 26(2), 81–91 (2003)
Huber, R., Meyer-Kahlen, S.: Universal Chess Interface specification (2000). http://wbec-ridderkerk.nl/html/UCIProtocol.html
Regan, K.W., Biswas, T.: Psychometric modeling of decision making via game play. In: CIG 2013, the 2013 IEEE Conference on Computational Intelligence in Games, August 2013, Niagara Falls, Canada (2013)
Regan, K.W., Macieja, B., Haworth, G.McC.: Understanding distributions of chess performances. In: van den Herik, H.J., Plaat, A. (eds.) ACG 2011. LNCS, vol. 7168, pp. 230–243. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31866-5_20
Regan, K.W., Haworth, G.McC.: Intrinsic chess ratings. In: AAAI 2011, the 25th AAAI Conference on Artificial Intelligence, 07–11 August 2011, San Francisco, USA, pp. 834–839 (2011). ISBN: 9781-5773-5507-6
Riis, S.: Review of “Computer Analysis of World Champions” (2006). http://www.chessbase.com/newsdetail.asp?newsid=3465
TCEC: Thoresen’s Chess Engine Competition, Seasons 1–7 (2014). http://tcec.chessdom.com/archive.php
Acknowledgements
In addition to thanking various parties for stimulating discussions on skill assessment, the authors thank David Barnes and Julio Hernandez Castro of the University of Kent for advance and subsequent discussion of their paper [2]
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Haworth, G., Biswas, T., Regan, K. (2015). A Comparative Review of Skill Assessment: Performance, Prediction and Profiling. In: Plaat, A., van den Herik, J., Kosters, W. (eds) Advances in Computer Games. ACG 2015. Lecture Notes in Computer Science(), vol 9525. Springer, Cham. https://doi.org/10.1007/978-3-319-27992-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-27992-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27991-6
Online ISBN: 978-3-319-27992-3
eBook Packages: Computer ScienceComputer Science (R0)