Survival Analysis on Duration Data in Intelligent Tutors
Effects such as student dropout and the non-normal distribution of duration data confound the exploration of tutor efficiency, time-in-tutor vs. tutor performance, in intelligent tutors. We use an accelerated failure time (AFT) model to analyze the effects of using automatically generated hints in Deep Thought, a propositional logic tutor. AFT is a branch of survival analysis, a statistical technique designed for measuring time-to-event data and account for participant attrition. We found that students provided with automatically generated hints were able to complete the tutor in about half the time taken by students who were not provided hints. We compare the results of survival analysis with a standard between-groups mean comparison and show how failing to take student dropout into account could lead to incorrect conclusions. We demonstrate that survival analysis is applicable to duration data collected from intelligent tutors and is particularly useful when a study experiences participant attrition.
KeywordsITS EDM Survival Analysis Efficiency Duration Data
Unable to display preview. Download preview PDF.
- 2.Anderson, J.R., Reiser, B.J.: The lisp tutor. Byte 10(4), 159–175 (1985)Google Scholar
- 3.Blischke, W.R., Murthy, D.P.: Reliability: Modeling, prediction, and optimization, vol. 767. Wiley (2011)Google Scholar
- 5.Crow, E.L., Shimizu, K.: Lognormal distributions: Theory and applications, vol. 88. CRC Press, LLC (1988)Google Scholar
- 6.Croy, M.J.: Graphic interface design and deductive proof construction. J. Comput. Math. Sci. Teach. 18, 371–385 (1999)Google Scholar
- 9.Meeker, W.Q., Escobar, L.A.: Statistical methods for reliability data, vol. 314. Wiley. com (1998)Google Scholar
- 10.Miller, R.B., Hollist, C.S.: Attrition bias (2007)Google Scholar
- 12.Stamper, J., Eagle, M., Barnes, T., Croy, M.: Experimental evaluation of automatic hint generation for a logic tutor. International Journal of Artificial Intelligence in Education (IJAIED) 22(1), 3–18 (2012)Google Scholar
- 15.Therneau, T.M.: A Package for Survival Analysis in S, R package version 2.37-7 (2014)Google Scholar