Advertisement

Personalizing Algebra to Students’ Individual Interests in an Intelligent Tutoring System: Moderators of Impact

  • Candace WalkingtonEmail author
  • Matthew L. Bernacki
Article

Abstract

Students experience mathematics in their day-to-day lives as they pursue their individual interests in areas like sports or video games. The present study explores how connecting to students’ individual interests can be used to personalize learning using an Intelligent Tutoring System (ITS) for algebra. We examine the idea that the effects of personalization may be moderated by students’ depth of quantitative engagement with their out-of-school interests. We also examine whether math problems designed to draw upon students’ knowledge of their individual interests at a deep level (i.e., actual quantitative experiences) or surface level (i.e., superficial changes to problem topic) have differential effects. Results suggest that connecting math instruction to students’ out-of-school interests can be beneficial for learning in an ITS and reduces gaming the system. However, benefits may only be realized when students’ degree of quantitative engagement with their out-of-school interests matches the depth at which the personalized problems are written. Students whose quantitative engagement with their interests is minimal may benefit most when problems draw upon superficial aspects of their interest areas. Students who report significant quantitative engagement with their interests may benefit most when individual interests are deeply incorporated into the quantitative structure of math problems. We also find that problems with deeper personalization may spur positive affective states and ward off negative ones for all students. Findings suggest depth is a critical feature of personalized learning with implications for theory and AI instructional design.

Keywords

Personalization Personalized learning Individual interest Intelligent tutoring systems 

Notes

Acknowledgements

This work was supported by the Pittsburgh Science of Learning Center, which was supported by the National Science Foundation, Award #0836012. Thank you to Gail Kusbit, Ryan Baker, Steve Ritter, Steve Fancsali, Vincent Aleven, and Timothy Nokes-Malach for their generous assistance with this research. Thank you to Carnegie Learning for their collaboration and support of this project. Thank you to Elizabeth Howell and Alyssa Holland for their assistance with data transcription and coding. And finally, thank you to the classroom teachers involved in this study.

References

  1. Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.Google Scholar
  2. Baker, R., & de Carvalho, A. (2008). Labeling student behavior faster and more precisely with text replays. In R. S. J. de Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining: 1st international conference on educational data mining, proceedings (pp. 38–47). Montreal, Quebec, Canada: International Educational Data Mining Society.Google Scholar
  3. Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z. (2004) Off-task behavior in the cognitive tutor classroom: When students "game the system." Proceedings of ACM CHI 2004: Computer-Human Interactions, 383–390.Google Scholar
  4. Baker, R. S., Corbett, A., Koedinger, K., & Roll, I. (2005). Detecting when students game the system, across tutor subjects and classroom cohorts. Proceedings of User Modeling, 2005, 220–224.Google Scholar
  5. Baker, R.S.J.d., Corbett, A.T., Roll, I., Koedinger, K.R. (2008a) Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, 18, 3, 287–314.Google Scholar
  6. Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008b). Why students engage in" gaming the system" behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2), 185–224.Google Scholar
  7. Baker, R. S. J.d., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of Learners' cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223–241.Google Scholar
  8. Baker, R.S.J.d., Gowda, S., Corbett, A.T. (2011) Towards predicting future transfer of learning. Proceedings of 15th International Conference on Artificial Intelligence in Education, 23–30.Google Scholar
  9. Baker, R.S.J.d., Gowda, S.M., Wixon, M., Kalka, J., Wagner, A.Z., Salvi, A., Aleven, V., Kusbit, G., Ocumpaugh, J., Rossi, L. (2012) Towards sensor-free affect detection in cognitive tutor algebra. Proceedings of the 5th International Conference on Educational Data Mining, 126–133.Google Scholar
  10. Bates, E., & Wiest, L. (2004). The impact of personalization of mathematical word problems on student performance. The Mathematics Educator, 14(2), 17–26.Google Scholar
  11. Bernacki, M. L., & Walkington, C. (2018). The role of situational interest in personalized learning. Journal of Education & Psychology, 110(6), 864–881.  https://doi.org/10.1037/edu0000250.
  12. Cakir, O., & Simsek, N. (2010). A comparative analysis of computer and paper-based personalization on student achievement. Computers & Education, 55, 1524–1531.Google Scholar
  13. Carnegie Learning. (2016). Cognitive Tutor Algebra. [software]. Pittsburgh, PA.Google Scholar
  14. Civil, M. (2007). Building on community knowledge: An avenue to equity in mathematics education. In N. Nassir. and P. Cobb (Eds.) Improving access to mathematics: Diversity and equity in the classroom (pp. 105–117).Google Scholar
  15. Common Core State Standards Initiative. (2010). Common Core State Standards (Mathematics Standards). Retrieved 6 July 2010 from http://www.corestandards.org/the-standards/mathematics
  16. Cordova, D., & Lepper, M. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715–730.Google Scholar
  17. Davis-Dorsey, J., Ross, S., & Morrison, G. (1991). The role of rewording and context personalization in the solving of mathematical word problems. Journal of Educational Psychology, 83(1), 61–68.Google Scholar
  18. Durik, A. M., & Harackiewicz, J. M. (2007). Different strokes for different folks: How individual interest moderates the effects of situational factors on task interest. Journal of Educational Psychology, 99(3), 597–610.Google Scholar
  19. Eccles, J., Fredricks, F., & Epstein, A. (2015). Understanding well-developed interests and activity commitment. In A. Renninger, M. Nieswandt, & S. Hidi (Eds,) Interest in Mathematics and Science Learning (pp. 315–330), American Educational Research Association, Washington D.C.Google Scholar
  20. Fancsali, S. E., & Ritter, S. (2014, March). Context personalization, preferences, and performance in an intelligent tutoring system for middle school mathematics. In Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (pp. 73–77). ACM.Google Scholar
  21. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.Google Scholar
  22. Fredricks, J. A., & Eccles, J. (2002). Children’s competence and value beliefs from childhood through adolescence: Growth trajectories in two male-sex-typed domains. Developmental Psychology, 38, 519–533.Google Scholar
  23. Frenzel, A. C., Goetz, T., Pekrun, R., & Watt, H. M. G. (2010). Development of mathematics interest in adolescence: Influences of gender, family, and school context. Journal of Research on Adolescence, 20, 507–537.Google Scholar
  24. Goldstone, R. L., & Son, J. Y. (2005). The transfer of scientific principles using concrete and idealized simulations. The Journal of the Learning Sciences, 14(1), 69–110.Google Scholar
  25. Gutstein, E. (2006). Reading and writing the world with mathematics: Toward a pedagogy for social justice. Taylor & Francis.Google Scholar
  26. Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414–434.Google Scholar
  27. Heilman, M., Collins-Thompson, K., Callan, J., Eskenazi, M., Juffs, A., & Wilson, L. (2010). Personalization of reading passages improves vocabulary acquisition. International Journal of Artificial Intelligence in Education, 20(1), 73–98.Google Scholar
  28. Hidi, S., & Ainley, M. (2008). Interest and self-regulation. The relationships between two variables that influence learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and application (pp. 77–109). New York: Erlbaum.Google Scholar
  29. Hidi, S., & Renninger, K. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.Google Scholar
  30. Honaker, J., King, G., & Blackwell, M. (2011). Amelia II: A program for missing data. Journal of Statistical Software, 45(7), 1–47. http://www.jstatsoft.org/v45/i07/.
  31. Høgheim, S., & Reber, R. (2015). Supporting interest of middle school students in mathematics through context personalization and example choice. Contemporary Educational Psychology, 42, 17–25.Google Scholar
  32. Kaput, J. J. (2000). Teaching and learning a new algebra with understanding. U.S.; Massachusetts: National Center for Improving Student Learning and Achievement.Google Scholar
  33. Koedinger, K. (2001). Cognitive tutors as modeling tool and instructional model. In K. D. Forbus & P. J. Feltovich (Eds.), Smart Machines in Education: The coming revolution in educational technology. Menlo Park: AAAI/MIT Press.Google Scholar
  34. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239–264.Google Scholar
  35. Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A data repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.) Handbook of Educational Data Mining. Boca Raton: CRC Press.Google Scholar
  36. Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge-learning-instruction (KLI) framework: Toward bridging the science practice chasm to enhance robust student learning. Cognitive Science, 36, 1–42.Google Scholar
  37. Ku, H., & Sullivan, H. (2000). Personalization of mathematics word problems in Taiwan. Educational Technology Research and Development, 48(3), 49–59.Google Scholar
  38. Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains. Educational and psychological measurement, 70(4), 647–671.Google Scholar
  39. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.Google Scholar
  40. McCoy, L. P. (2005). Effect of demographic and personal variables on achievement in eighth-grade algebra. Journal of Educational Research, 98(3), 131–135.Google Scholar
  41. Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory into practice, 31(2), 132–141.Google Scholar
  42. Moses, R., & Cobb, C. (2001). Radical equations: Math literacy and civil rights. Boston: Beacon Press.Google Scholar
  43. National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. US Department of Education. Retrieved from http://www2.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf.
  44. Ocumpaugh, J., Baker, R. S. J. D., & Rodrigo, M. M. T. (2012). Baker-Rodrigo Observation Method Protocol (BROMP) 1.0. Training Manual version 1.0. Technical report. New York: EdLab. Manila, Philippines: Ateneo Laboratory for the Learning Sciences.Google Scholar
  45. Pardos, Z. A., Baker, R. S. J.d., San Pedro, M. O. C. Z., Gowda, S. M., & Gowda, S. M. (2013). Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge, 117–124.Google Scholar
  46. Reber, R., Hetland, H., Chen, W., Norman, E., & Kobbeltvedt, T. (2009). Effects of example choice on interest, control, and learning. The Journal of the Learning Sciences, 18(4), 509–548.Google Scholar
  47. Renninger, K., & Pozos-Brewer, R. (2015). Psychology of interest. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences 2 nd edition Volume 12 (pp. 378–385). Oxford: Elsevier.Google Scholar
  48. Renninger, K., & Su, S. (2012). Interest and its development. In R. M. Ryan (Ed.), Handbook of human motivation. NY: Oxford University Press.Google Scholar
  49. Renninger, K. A., & Hidi, S. (2016). The power of interest for motivation and engagement. New York, NY: Routledge.Google Scholar
  50. Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M [superscript 2]: Media in the Lives of 8-to 18-year-olds. Henry J. Kaiser Family Foundation. Retrieved from http://files.eric.ed.gov/fulltext/ED527859.pdf
  51. Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26(2), 582–599.Google Scholar
  52. Romero, C., Ventura, S., Pechenizky, M., Baker, R. (2010). Handbook of educational data mining. 2010. Editorial Chapman and Hall/CRC Press, Taylor & Francis Group. Data Mining and Knowledge Discovery Series.Google Scholar
  53. San Pedro, M.O.Z., Baker, R.S.J.d., Bowers, A.J., Heffernan, N.T. (2013) Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. Proceedings of the 6th International Conference on Educational Data Mining, 177–184.Google Scholar
  54. San Pedro, M.O.Z., Baker, R.S.J.d., Mercedes, M.M.T. (2014) Carelessness and affect in an intelligent tutoring system for mathematics. International Journal of Artifiical Intelligence in Education, 24, 189–210.Google Scholar
  55. Sansone, C., Fraughton, T., Zachary, J., Butner, J., & Heiner, C. (2011). Self-regulation of motivation when learning online: The important of who, why, and how. Educational Technology Research and Development, 59, 199–212.Google Scholar
  56. Santos, O. C., Kravcik, M., & Boticario, J. G. (2016). Preface to special issue on user modelling to support personalization in enhanced educational settings. International Journal of Artificial Intelligence in Education, 26(3), 809–820.Google Scholar
  57. Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research. Educational Psychology Review, 13(1), 23–52.Google Scholar
  58. Simsek, N., & Cakir, O. (2009). Effect of personalization on students’ achievement and gender factor in mathematics education. International Journal of Social Science, 4, 278–282.Google Scholar
  59. Turner, E., Drake, C., McDuffie, A., Aguirre, J., Bartell, T., & Foote, M. (2012). Promoting equity in mathematics teacher preparation: A framework for advancing teacher learning of children’s multiple mathematics knowledge bases. Journal of Mathematics Teacher Education, 15, 67–82.Google Scholar
  60. Walkington, C. (2013). Using learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105(4), 932–945.Google Scholar
  61. Walkington, C., & Bernacki, M. (2014). Motivating students by “personalizing” learning around individual interests: A consideration of theory, design, and implementation issues. In S. Karabenick & T. Urdan (eds.) Advances in Motivation and Achievement Volume 18 (pp. 139–176), Emerald Group Publishing.Google Scholar
  62. Walkington, C., & Bernacki, M. (2015). Students authoring personalized “algebra stories”: Problem-posing in the context of out-of-school interests. The Journal of Mathematical Behavior, 40B, 171–191.Google Scholar
  63. Walkington, C., & Hayata, C. (2017). Designing learning personalized to students’ interests: Balancing rich experiences with mathematical goals. ZDM Mathematics Education, 49(4), 519–530.  https://doi.org/10.1007/s11858-017-0842-z.Google Scholar
  64. Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of story problems: Coordinating situation-based reasoning with algebraic representation. The Journal of Mathematical Behavior, 31(2), 174–195.Google Scholar
  65. Walkington, C., Cooper, J., & Howell, E. (2013). The effects of visual representations and interest-based personalization on solving percent problems. In M. Martinez & A. Castro Superfine (Eds.), Proceedings of the 35th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 533–536). Chicago: University of Illinois at Chicago.Google Scholar
  66. Walkington, C., Sherman, M., & Howell, E. (2014). Personalized learning in algebra. Mathematics Teacher, 108(4), 272–279.Google Scholar
  67. Walkington, C., Clinton, V., Ritter, S., & Nathan, M. J. (2015a). How readability and topic incidence relate to performance on mathematics story problems in computer-based curricula. Journal of Educational Psychology, 107(4), 1051–1074.Google Scholar
  68. Walkington, C., Cooper, J., Nathan, M. J., & Alibali, M. A. (2015b). The effects of visual representations and interest-based personalization on solving mathematics story problems. In T. Bartell, K. Bieda, R. Putnam, K. Bradfield, & H. Dominguez (Eds.), Proceedings of the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (p. 127). East Lansing: Michigan State University.Google Scholar
  69. Walkington, C., Clinton, V., & Mingle, L. (2016). Considering cognitive factors in interest research: Context personalization and illustrations in math curricula. In M. B. Wood, E. E. Turner, M. Civil, & J. A. Eli (Eds.), Proceedings of the 38th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 89–96). Tucson, AZ: The University of Arizona.Google Scholar

Copyright information

© International Artificial Intelligence in Education Society 2018

Authors and Affiliations

  1. 1.Southern Methodist UniversityDallasUSA
  2. 2.University of Nevada, Las VegasLas VegasUSA

Personalised recommendations