# Personalizing Algebra to Students’ Individual Interests in an Intelligent Tutoring System: Moderators of Impact

## Abstract

Students experience mathematics in their day-to-day lives as they pursue their individual interests in areas like sports or video games. The present study explores how connecting to students’ individual interests can be used to personalize learning using an Intelligent Tutoring System (ITS) for algebra. We examine the idea that the effects of personalization may be moderated by students’ depth of quantitative engagement with their out-of-school interests. We also examine whether math problems designed to draw upon students’ knowledge of their individual interests at a deep level (i.e., actual quantitative experiences) or surface level (i.e., superficial changes to problem topic) have differential effects. Results suggest that connecting math instruction to students’ out-of-school interests can be beneficial for learning in an ITS and reduces gaming the system. However, benefits may only be realized when students’ degree of quantitative engagement with their out-of-school interests matches the depth at which the personalized problems are written. Students whose quantitative engagement with their interests is minimal may benefit most when problems draw upon superficial aspects of their interest areas. Students who report significant quantitative engagement with their interests may benefit most when individual interests are deeply incorporated into the quantitative structure of math problems. We also find that problems with deeper personalization may spur positive affective states and ward off negative ones for all students. Findings suggest depth is a critical feature of personalized learning with implications for theory and AI instructional design.

## Keywords

Personalization Personalized learning Individual interest Intelligent tutoring systems## Notes

### Acknowledgements

This work was supported by the Pittsburgh Science of Learning Center, which was supported by the National Science Foundation, Award #0836012. Thank you to Gail Kusbit, Ryan Baker, Steve Ritter, Steve Fancsali, Vincent Aleven, and Timothy Nokes-Malach for their generous assistance with this research. Thank you to Carnegie Learning for their collaboration and support of this project. Thank you to Elizabeth Howell and Alyssa Holland for their assistance with data transcription and coding. And finally, thank you to the classroom teachers involved in this study.

## References

- Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a cognitive tutor.
*International Journal of Artificial Intelligence in Education, 16*(2), 101–128.Google Scholar - Baker, R., & de Carvalho, A. (2008). Labeling student behavior faster and more precisely with text replays. In R. S. J. de Baker, T. Barnes, & J. E. Beck (Eds.),
*Educational data mining: 1st international conference on educational data mining, proceedings*(pp. 38–47). Montreal, Quebec, Canada: International Educational Data Mining Society.Google Scholar - Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z. (2004) Off-task behavior in the cognitive tutor classroom: When students "game the system."
*Proceedings of ACM CHI 2004: Computer-Human Interactions*, 383–390.Google Scholar - Baker, R. S., Corbett, A., Koedinger, K., & Roll, I. (2005). Detecting when students game the system, across tutor subjects and classroom cohorts.
*Proceedings of User Modeling, 2005*, 220–224.Google Scholar - Baker, R.S.J.d., Corbett, A.T., Roll, I., Koedinger, K.R. (2008a) Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, 18, 3, 287–314.Google Scholar
- Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008b). Why students engage in" gaming the system" behavior in interactive learning environments.
*Journal of Interactive Learning Research, 19*(2), 185–224.Google Scholar - Baker, R. S. J.d., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of Learners' cognitive-affective states during interactions with three different computer-based learning environments.
*International Journal of Human-Computer Studies, 68*(4), 223–241.Google Scholar - Baker, R.S.J.d., Gowda, S., Corbett, A.T. (2011) Towards predicting future transfer of learning.
*Proceedings of 15th International Conference on Artificial Intelligence in Education*, 23–30.Google Scholar - Baker, R.S.J.d., Gowda, S.M., Wixon, M., Kalka, J., Wagner, A.Z., Salvi, A., Aleven, V., Kusbit, G., Ocumpaugh, J., Rossi, L. (2012) Towards sensor-free affect detection in cognitive tutor algebra.
*Proceedings of the 5th International Conference on Educational Data Mining*, 126–133.Google Scholar - Bates, E., & Wiest, L. (2004). The impact of personalization of mathematical word problems on student performance.
*The Mathematics Educator, 14*(2), 17–26.Google Scholar - Bernacki, M. L., & Walkington, C. (2018). The role of situational interest in personalized learning.
*Journal of Education & Psychology, 110*(6), 864–881. https://doi.org/10.1037/edu0000250. - Cakir, O., & Simsek, N. (2010). A comparative analysis of computer and paper-based personalization on student achievement.
*Computers & Education, 55*, 1524–1531.Google Scholar - Carnegie Learning. (2016). Cognitive Tutor Algebra. [software]. Pittsburgh, PA.Google Scholar
- Civil, M. (2007). Building on community knowledge: An avenue to equity in mathematics education. In N. Nassir. and P. Cobb (Eds.)
*Improving access to mathematics: Diversity and equity in the classroom*(pp. 105–117).Google Scholar - Common Core State Standards Initiative. (2010). Common Core State Standards (Mathematics Standards). Retrieved 6 July 2010 from http://www.corestandards.org/the-standards/mathematics
- Cordova, D., & Lepper, M. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice.
*Journal of Educational Psychology, 88*(4), 715–730.Google Scholar - Davis-Dorsey, J., Ross, S., & Morrison, G. (1991). The role of rewording and context personalization in the solving of mathematical word problems.
*Journal of Educational Psychology, 83*(1), 61–68.Google Scholar - Durik, A. M., & Harackiewicz, J. M. (2007). Different strokes for different folks: How individual interest moderates the effects of situational factors on task interest.
*Journal of Educational Psychology, 99*(3), 597–610.Google Scholar - Eccles, J., Fredricks, F., & Epstein, A. (2015). Understanding well-developed interests and activity commitment. In A. Renninger, M. Nieswandt, & S. Hidi (Eds,)
*Interest in Mathematics and Science Learning*(pp. 315–330), American Educational Research Association, Washington D.C.Google Scholar - Fancsali, S. E., & Ritter, S. (2014, March). Context personalization, preferences, and performance in an intelligent tutoring system for middle school mathematics. In
*Proceedings of the Fourth International Conference on Learning Analytics and Knowledge*(pp. 73–77). ACM.Google Scholar - Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses.
*Behavior Research Methods, 41*, 1149–1160.Google Scholar - Fredricks, J. A., & Eccles, J. (2002). Children’s competence and value beliefs from childhood through adolescence: Growth trajectories in two male-sex-typed domains.
*Developmental Psychology, 38*, 519–533.Google Scholar - Frenzel, A. C., Goetz, T., Pekrun, R., & Watt, H. M. G. (2010). Development of mathematics interest in adolescence: Influences of gender, family, and school context.
*Journal of Research on Adolescence, 20*, 507–537.Google Scholar - Goldstone, R. L., & Son, J. Y. (2005). The transfer of scientific principles using concrete and idealized simulations.
*The Journal of the Learning Sciences, 14*(1), 69–110.Google Scholar - Gutstein, E. (2006). Reading and writing the world with mathematics: Toward a pedagogy for social justice. Taylor & Francis.Google Scholar
- Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning.
*Journal of Educational Psychology, 90*(3), 414–434.Google Scholar - Heilman, M., Collins-Thompson, K., Callan, J., Eskenazi, M., Juffs, A., & Wilson, L. (2010). Personalization of reading passages improves vocabulary acquisition.
*International Journal of Artificial Intelligence in Education, 20*(1), 73–98.Google Scholar - Hidi, S., & Ainley, M. (2008). Interest and self-regulation. The relationships between two variables that influence learning. In D. H. Schunk & B. J. Zimmerman (Eds.),
*Motivation and self-regulated learning: Theory, research, and application*(pp. 77–109). New York: Erlbaum.Google Scholar - Hidi, S., & Renninger, K. (2006). The four-phase model of interest development.
*Educational Psychologist, 41*(2), 111–127.Google Scholar - Honaker, J., King, G., & Blackwell, M. (2011). Amelia II: A program for missing data.
*Journal of Statistical Software, 45*(7), 1–47. http://www.jstatsoft.org/v45/i07/. - Høgheim, S., & Reber, R. (2015). Supporting interest of middle school students in mathematics through context personalization and example choice.
*Contemporary Educational Psychology, 42*, 17–25.Google Scholar - Kaput, J. J. (2000).
*Teaching and learning a new algebra with understanding*. U.S.; Massachusetts: National Center for Improving Student Learning and Achievement.Google Scholar - Koedinger, K. (2001). Cognitive tutors as modeling tool and instructional model. In K. D. Forbus & P. J. Feltovich (Eds.),
*Smart Machines in Education: The coming revolution in educational technology*. Menlo Park: AAAI/MIT Press.Google Scholar - Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors.
*Educational Psychology Review, 19*, 239–264.Google Scholar - Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A data repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.)
*Handbook of Educational Data Mining*. Boca Raton: CRC Press.Google Scholar - Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge-learning-instruction (KLI) framework: Toward bridging the science practice chasm to enhance robust student learning.
*Cognitive Science, 36*, 1–42.Google Scholar - Ku, H., & Sullivan, H. (2000). Personalization of mathematics word problems in Taiwan.
*Educational Technology Research and Development, 48*(3), 49–59.Google Scholar - Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains.
*Educational and psychological measurement, 70*(4), 647–671.Google Scholar - Mayer, R. E. (2009).
*Multimedia learning*(2nd ed.). New York: Cambridge University Press.Google Scholar - McCoy, L. P. (2005). Effect of demographic and personal variables on achievement in eighth-grade algebra.
*Journal of Educational Research, 98*(3), 131–135.Google Scholar - Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms.
*Theory into practice, 31*(2), 132–141.Google Scholar - Moses, R., & Cobb, C. (2001).
*Radical equations: Math literacy and civil rights*. Boston: Beacon Press.Google Scholar - National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathematics Advisory Panel.
*US Department of Education. Retrieved from*http://www2.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf. - Ocumpaugh, J., Baker, R. S. J. D., & Rodrigo, M. M. T. (2012).
*Baker-Rodrigo Observation Method Protocol (BROMP) 1.0. Training Manual version 1.0. Technical report*. New York: EdLab. Manila, Philippines: Ateneo Laboratory for the Learning Sciences.Google Scholar - Pardos, Z. A., Baker, R. S. J.d., San Pedro, M. O. C. Z., Gowda, S. M., & Gowda, S. M. (2013). Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes.
*Proceedings of the 3rd International Conference on Learning Analytics and Knowledge*, 117–124.Google Scholar - Reber, R., Hetland, H., Chen, W., Norman, E., & Kobbeltvedt, T. (2009). Effects of example choice on interest, control, and learning.
*The Journal of the Learning Sciences, 18*(4), 509–548.Google Scholar - Renninger, K., & Pozos-Brewer, R. (2015). Psychology of interest. In J. D. Wright (Ed.),
*International Encyclopedia of the Social & Behavioral Sciences 2*^{nd}*edition Volume 12*(pp. 378–385). Oxford: Elsevier.Google Scholar - Renninger, K., & Su, S. (2012). Interest and its development. In R. M. Ryan (Ed.),
*Handbook of human motivation*. NY: Oxford University Press.Google Scholar - Renninger, K. A., & Hidi, S. (2016). The power of interest for motivation and engagement. New York, NY: Routledge.Google Scholar
- Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M [superscript 2]: Media in the Lives of 8-to 18-year-olds. Henry J
*.*Kaiser Family Foundation. Retrieved from http://files.eric.ed.gov/fulltext/ED527859.pdf - Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education.
*International Journal of Artificial Intelligence in Education, 26*(2), 582–599.Google Scholar - Romero, C., Ventura, S., Pechenizky, M., Baker, R. (2010). Handbook of educational data mining
*.*2010. Editorial Chapman and Hall/CRC Press, Taylor & Francis Group. Data Mining and Knowledge Discovery Series.Google Scholar - San Pedro, M.O.Z., Baker, R.S.J.d., Bowers, A.J., Heffernan, N.T. (2013) Predicting college enrollment from student interaction with an intelligent tutoring system in middle school.
*Proceedings of the 6th International Conference on Educational Data Mining*, 177–184.Google Scholar - San Pedro, M.O.Z., Baker, R.S.J.d., Mercedes, M.M.T. (2014) Carelessness and affect in an intelligent tutoring system for mathematics.
*International Journal of Artifiical Intelligence in Education,*24, 189–210.Google Scholar - Sansone, C., Fraughton, T., Zachary, J., Butner, J., & Heiner, C. (2011). Self-regulation of motivation when learning online: The important of who, why, and how.
*Educational Technology Research and Development, 59*, 199–212.Google Scholar - Santos, O. C., Kravcik, M., & Boticario, J. G. (2016). Preface to special issue on user modelling to support personalization in enhanced educational settings.
*International Journal of Artificial Intelligence in Education, 26*(3), 809–820.Google Scholar - Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research.
*Educational Psychology Review, 13*(1), 23–52.Google Scholar - Simsek, N., & Cakir, O. (2009). Effect of personalization on students’ achievement and gender factor in mathematics education.
*International Journal of Social Science, 4*, 278–282.Google Scholar - Turner, E., Drake, C., McDuffie, A., Aguirre, J., Bartell, T., & Foote, M. (2012). Promoting equity in mathematics teacher preparation: A framework for advancing teacher learning of children’s multiple mathematics knowledge bases.
*Journal of Mathematics Teacher Education, 15*, 67–82.Google Scholar - Walkington, C. (2013). Using learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes.
*Journal of Educational Psychology, 105*(4), 932–945.Google Scholar - Walkington, C., & Bernacki, M. (2014). Motivating students by “personalizing” learning around individual interests: A consideration of theory, design, and implementation issues. In S. Karabenick & T. Urdan (eds.)
*Advances in Motivation and Achievement Volume 18*(pp. 139–176), Emerald Group Publishing.Google Scholar - Walkington, C., & Bernacki, M. (2015). Students authoring personalized “algebra stories”: Problem-posing in the context of out-of-school interests.
*The Journal of Mathematical Behavior, 40B*, 171–191.Google Scholar - Walkington, C., & Hayata, C. (2017). Designing learning personalized to students’ interests: Balancing rich experiences with mathematical goals.
*ZDM Mathematics Education, 49*(4), 519–530. https://doi.org/10.1007/s11858-017-0842-z.Google Scholar - Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of story problems: Coordinating situation-based reasoning with algebraic representation.
*The Journal of Mathematical Behavior, 31*(2), 174–195.Google Scholar - Walkington, C., Cooper, J., & Howell, E. (2013). The effects of visual representations and interest-based personalization on solving percent problems. In M. Martinez & A. Castro Superfine (Eds.),
*Proceedings of the 35th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education*(pp. 533–536). Chicago: University of Illinois at Chicago.Google Scholar - Walkington, C., Sherman, M., & Howell, E. (2014). Personalized learning in algebra.
*Mathematics Teacher, 108*(4), 272–279.Google Scholar - Walkington, C., Clinton, V., Ritter, S., & Nathan, M. J. (2015a). How readability and topic incidence relate to performance on mathematics story problems in computer-based curricula.
*Journal of Educational Psychology, 107*(4), 1051–1074.Google Scholar - Walkington, C., Cooper, J., Nathan, M. J., & Alibali, M. A. (2015b). The effects of visual representations and interest-based personalization on solving mathematics story problems. In T. Bartell, K. Bieda, R. Putnam, K. Bradfield, & H. Dominguez (Eds.),
*Proceedings of the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (p. 127)*. East Lansing: Michigan State University.Google Scholar - Walkington, C., Clinton, V., & Mingle, L. (2016). Considering cognitive factors in interest research: Context personalization and illustrations in math curricula. In M. B. Wood, E. E. Turner, M. Civil, & J. A. Eli (Eds.),
*Proceedings of the 38th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education*(pp. 89–96). Tucson, AZ: The University of Arizona.Google Scholar