Decision Support for an Adversarial Game Environment Using Automatic Hint Generation

  • Steven MooreEmail author
  • John Stamper
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11528)


The Hint Factory is a method of automatic hint generation that has been used to augment hints in a number of educational systems. Although the previous implementations were done in domains with largely deterministic environments, the methods are inherently useful in stochastic environments with uncertainty. In this work, we explore the game Connect Four as a simple domain to give decision support under uncertainty. We speculate how the implementation created could be extended to other domains including simulated learning environments and advanced navigational tasks.


Hint generation Educational data mining Reinforcement learning 


  1. Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward tutoring help seeking. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 227–239. Springer, Heidelberg (2004). Scholar
  2. Aleven, V., McLaren, B.M., Sewall, J., Koedinger, K.R.: The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 61–70. Springer, Heidelberg (2006). Scholar
  3. Allis, L.V.: A knowledge-based approach of connect-four. Vrije Universiteit, Subfaculteit Wiskunde en Informatica (1988)Google Scholar
  4. Anggara, K.: Connect Four (2018).
  5. Barnes, T., Stamper, J., Croy, M.: Using Markov decision processes for automatic hint generation. In: Handbook of Educational Data Mining, 467 (2011)Google Scholar
  6. Edelkamp, S., Kissmann, P.: Symbolic classification of general two-player games. In: Dengel, A.R., Berns, K., Breuel, T.M., Bomarius, F., Roth-Berghofer, T.R. (eds.) KI 2008. LNCS (LNAI), vol. 5243, pp. 185–192. Springer, Heidelberg (2008). Scholar
  7. Edwards, W.: The theory of decision making. Psychol. Bull. 51(4), 380 (1954)CrossRefGoogle Scholar
  8. Koedinger, K.R., Stamper, J.C., McLaughlin, E.A., Nixon, T.: Using data-driven discovery of better student models to improve student learning. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 421–430. Springer, Heidelberg (2013). Scholar
  9. Mosley, P., Kline, R.: Engaging students: a framework using lego robotics to teach problem solving. Inf. Technol. Learn. Perform. J. 24(1), 39–45 (2006)Google Scholar
  10. Paquette, L., Lebeau, J.-F., Beaulieu, G., Mayers, A.: Automating next-step hints generation using ASTUS. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 201–211. Springer, Heidelberg (2012). Scholar
  11. Razzaq, L., Heffernan, N.T.: Hints: is it better to give or wait to be asked? In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6094, pp. 349–358. Springer, Heidelberg (2010). Scholar
  12. Rivers, K.: Automated Data-Driven Hint Generation for Learning Programming (2017)Google Scholar
  13. Shih, B., Koedinger, K.R., Scheines, R.: A response time model for bottom-out hints as worked examples. In: Handbook of Educational Data Mining, pp. 201–212 (2011)CrossRefGoogle Scholar
  14. Stamper, J.C., Eagle, M., Barnes, T., Croy, M.: Experimental evaluation of automatic hint generation for a logic tutor. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 345–352. Springer, Heidelberg (2011a). Scholar
  15. Stamper, J., Barnes, T., Croy, M.: Enhancing the automatic generation of hints with expert seeding. Int. J. AI Educ. 21(1–2), 153–167 (2011b)Google Scholar
  16. Stamper, J., et al.: PSLC DataShop: a data analysis service for the learning science community. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6095, p. 455. Springer, Heidelberg (2010). Scholar
  17. Stamper, J., Barnes, T., Lehmann, L., Croy, M.: The hint factory: Automatic generation of contextualized help for existing computer aided instruction. In: 9th International Conf on Intelligent Tutoring Systems Young Researchers Track, pp. 71–78 (2008)Google Scholar
  18. Stamper, J.C., Barnes, T., Croy, M.: Extracting student models for intelligent tutoring systems. In: Proceedings of the National Conference on Artificial Intelligence, vol. 22, no. 2, p. 1900. AAAI Press; MIT Press, 1999 (2007)Google Scholar
  19. Stamper, J.: Automating the generation of production rules for intelligent tutoring systems. In: Proceedings of the 9th International Conference on Interactive Computer Aided Learning (ICL 2006). Kassel University Press (2006)Google Scholar
  20. Sutton, R., Barto, A.: Reinforcement Learning. The MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  21. Thill, M., Koch, P., Konen, W.: Reinforcement learning with N-tuples on the game connect-4. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 184–194. Springer, Heidelberg (2012). Scholar
  22. Tsovaltzi, D., et al.: Extending a virtual chemistry lab with a collaboration script to promote conceptual learning. Int. J. Technol. Enhanc. Learn. 2(1–2), 91–110 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.HCIICarnegie Mellon UniversityPittsburghUSA

Personalised recommendations