DataShopping for Performance Predictions
Mathematical models of learning have been created to capitalize on the regularities that are seen when individuals acquire new skills, which could be useful if implemented in learning management systems. One such mathematical model is the Predictive Performance Equation (PPE). It is the intent that PPE will be used to predict the performance of individuals to inform real-world education and training decisions. However, in order to improve mathematical models of learning, data from multiple samples are needed. Online data repositories, such as Carnegie Mellon University’s DataShop, provide data from multiple studies at fine levels of granularity. In this paper, we describe results from a set of analyses ranging across levels of granularity in order to assess the predictive validity of PPE in educational contexts available in the repository.
KeywordsPerformance prediction Datashop Repository Learning optimization Mathematical models
We would like to thank Carnegie Learning, Inc., for providing the Cognitive Tutoring data supporting this analysis. We used the “Handwriting/Examples Dec 2006”, “Geometry Area (1996–1997)”, “USNA Physics Fall 2006”, “USNA Physics Fall 2007”, “USNA Introductory Physics Fall 2009”, “USNA Physics Spring 2007”, “USNA Physics Spring 2008”, “USNA Introductory Physics Spring 2010”, “Watchung Hills Regional High School AP Physics 2007–2008”, “Watchung Hills Regional High School AP Physics 2008–2009”, “Watchung Hills Regional High School Honors Physics 2007–2008”, “Watchung Hills Regional High School Honors Physics 2008–2009” and “Watchung Hills Regional High School Honors Physics 2009–2010” accessed via DataShop. We would also like to thank Michael Krusmark for the work he was done implementing the PPE into R. This research was supported by the Air Force Research Laboratory through AFOSR grant 13RH15COR and through Early Career Award Winner funds provided to the third author.
- 1.Anderson, J.R., Schunn, C.: Implications of the ACT-R learning theory: no magic bullets. In: Glaser, R. (ed.) Advances in Instructional Psychology: Educational Design and Cognitive Science, vol. 5, pp. 1–33. Erlbaum, Mahwah (2000)Google Scholar
- 4.Jastrzembski, T.S., Gluck, K.A., Gunzelmann, G.: Knowledge tracing and prediction of future trainee performance. In: 40th I/ITSEC Annual Meeting, pp. 1498–1508. National Training Systems Association, Orlando, FL (2006)Google Scholar
- 5.Jastrzembski, T.S., Gluck, K.A.: A formal comparison of model variants for performance prediction. In: Proceedings of the International Conference on Cognitive Modeling (ICCM), Manchester, England (2009)Google Scholar
- 6.Jastrzembski, T.S., Addis, K., Krusmark, M., Gluck, K.A., Rodgers, S.: Prediction intervals for performance prediction. In: International Conference on Cognitive Modeling (ICCM), pp. 109–114, Philadelphia, PA (2010)Google Scholar
- 7.Koedinger, K.R., Baker, R.S.J.D., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC DATASHOP. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.D. (eds.) Handbook of Educational Data Mining, pp. 43–56. CRC Press, Boca Raton (2010)CrossRefGoogle Scholar
- 8.Newell, A., Rosenbloom, P.S.: Mechanisms of skill acquisition and the law of practice. In: Anderson, J.R. (ed.) Cognitive Skills and Their Acquisition, vol. 1, pp. 1–55. Lawrence Erlbaum Associates, Inc., Hillsdale (1981)Google Scholar