Minimizing Computing Costs of Policy Trees in a POMDP-based Intelligent Tutoring System

  • Fangju WangEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 739)


Uncertainties exist in intelligent tutoring. The partially observable Markov decision process (POMDP) model may provide useful tools for handling uncertainties. The model may enable an intelligent tutoring system (ITS) to choose optimal actions when uncertainties occur. A major difficulty in applying the POMDP model to intelligent tutoring is its computational complexity. Typically, when a technique of policy trees is used, in making a decision the number of policy trees to evaluate is exponential, and the cost of evaluating a tree is also exponential. To overcome the difficulty, we develop a new technique of policy trees, based on the features of tutoring processes. The technique can minimize the number of policy trees to evaluate in making a decision, and minimize the costs of evaluating individual trees.


Intelligent tutoring system Partially observable markov decision process Policy tree Computing cost 



This research is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).


  1. 1.
    Anderson, J.R., Corbett, A.T., Koedinger, K.R., Peeelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)CrossRefGoogle Scholar
  2. 2.
    Braziunas, D.: POMDP solution methods: a survey. Technical report, Department of Computer Science, University of Toronto (2003)Google Scholar
  3. 3.
    Carlin, A., Zilberstein, S.: Observation compression in DEC-POMDP policy trees. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-agent Systems, pp. 31–45(2008)Google Scholar
  4. 4.
    Cassandra, A. A survey of POMDP applications. In: Working Notes of AAAI 1998 Fall Symposium on Planning with Partially Observable Markov Decision Process, pp. 17–24(1998)Google Scholar
  5. 5.
    Cheung, B., Hui, L., Zhang, J., Yiu, S.M.: SmartTutor: an intelligent tutoring system in web-based adult education. J. Syst. Softw. 68, 11–25 (2003). ElsevierCrossRefGoogle Scholar
  6. 6.
    Chinaei, H.R., Chaib-draa, B., Lamontagne, L.: Learning observation models for dialogue POMDPs. In: Kosseim, L., Inkpen, D. (eds.) AI 2012. LNCS, vol. 7310, pp. 280–286. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-30353-1_24 CrossRefGoogle Scholar
  7. 7.
    Folsom-Kovarik, J.T., Sukthankar, G., Schatz, S.: Tractable POMDP representations for intelligent tutoring systems. ACM Trans. Intell. Syst. Technol. (TIST) - Special Section on Agent Communication, Trust in Multiagent Systems, Intelligent Tutoring and Coaching Systems Archive 4(2), 29 (2013)Google Scholar
  8. 8.
    Jeremic, Z., Jovanovic, J., Gasevic, D.: Evaluating an intelligent tutoring systemfor design patterns: the DEPTHS experience. Educ. Technol. Soc. 12(2), 111–130 (2009)Google Scholar
  9. 9.
    Littman, M.L.: A tutorial on partially observable Markov decision processes. J. Math. Psychol. 53, 119–125 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    McLaren, B.M., DeLeeuw, K.E., Mayer, R.E.: Polite web-based intelligent tutors: can they improve learning in classrooms? Comput. Educ. 56(3), 574–584 (2011)CrossRefGoogle Scholar
  11. 11.
    McLaren, B.M., van Gog, T., Ganoe, C., Karabinos, M., Yaron, D.: The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in classroom experiments. Comput. Hum. Behav. 55, 87–99 (2016)CrossRefGoogle Scholar
  12. 12.
    Rafferty, A.N., Brunskill, E., Griffiths, T.L., Shafto, P.: Faster teaching by POMDP planning. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 280–287. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-21869-9_37 CrossRefGoogle Scholar
  13. 13.
    Theocharous, G., Beckwith, R., Butko, N., Philipose, M.: Tractable POMDP planning algorithms for optimal teaching in SPAIS. In: IJCAI PAIR Workshop (2009)Google Scholar
  14. 14.
    VanLehn, K.: Intelligent turoring systems for continuous, embedded assessment. In: Dwyer, C. (ed.) The Future of Assessment: Shaping Teaching and Learning, pp. 113–138. Erlbaum, Mahwah (2008)Google Scholar
  15. 15.
    VanLehn, K., van de Sande, B., Shelby, R., Gershman, S.: The Andes physics tutoring system: an experiment in Freedom. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems. SCI, vol. 308, pp. 421–443. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-14363-2_21 CrossRefGoogle Scholar
  16. 16.
    Wang, F.: Handling exponential state space in a POMDP-based intelligent tutoring system. In: Proceedings of 6th International Conference on E-Service and Knowledge Management (IIAI ESKM 2015), pp. 67–72 (2015)Google Scholar
  17. 17.
    Williams, J.D., Poupart, P., Young, S.: Factored partially observable Markov decision processes for dialogue management. In: Proceedings of Knowledge and Reasoning in Practical Dialogue Systems (2005)Google Scholar
  18. 18.
    Williams, J.D., Young, S.: Partially observable Markov decision processes for spoken dialog systems. Comput. Speech Lang. 21, 393–422 (2007). ElsevierCrossRefGoogle Scholar
  19. 19.
    Woolf, B.P.: Building Intelligent Interactive Tutors. Morgan Kaufmann Publishers, Burlington (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of GuelphGuelphCanada

Personalised recommendations