Current online learning approaches are sometimes criticized for a “one- size- fits -all” approach and insufficient feedback, which may result in low levels of satisfaction and high dropout rates. To mitigate these shortcomings, we implemented a set of principles for designing personalized motivational feedback based on students’ achievement goals. A sequential explanatory mixed-methods design was used to collect data. The results indicated students who received this feedback demonstrated significantly higher levels of motivation and satisfaction. However, their performance scores were not significantly higher than students in the control group. Post-interviews showed this personalized feedback helped students regulate their learning goals and behaviors and thus improved their learning. The evidence suggests this personalized feedback may benefit learning by improving motivation. While its effect on performance outcomes was not significant, we speculate that more complex factors may have obscured any performance effect.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Note: Learning style was criticized later as an invalid learning inctruct. See more details on page 7.
Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help-seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.
Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2017). Instruction based on adaptive learning technologies. In R. E. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction (2nd ed., pp. 522–560). New York: Routledge.
Allen, E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment report 2017. Babson Park, MA: Babson Survey Research Group.
Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84(3), 261–271.
An, D., & Carr, M. (2017). Learning styles theory fails to explain learning and achievement: Recommendations for alternative approaches. Personality and Individual Differences, 116, 410–416.
Arroyo, I., Beck, J. E., Beal, C. R., Wing, R., & Woolf, B. P. (2001). Analyzing students’ response to help provision in an elementary mathematics intelligent tutoring system. Papers of the AIED-2001 workshop on help provision and help-seeking in interactive learning environments, 34–46.
Bawa, P. (2016). Retention in online courses: exploring issues and solutions. SAGE Open, 6(1), 1–11.
Berlin, N., & Dargnies, M. P. (2016). Gender differences in reactions to feedback and willingness to compete. Journal of Economic Behavior & Organization, 130, 320–336.
Brusilovsky, P. (2001). Adaptive educational hypermedia. Proceedings of Tenth International PEG Conference, 8–12.
Budd, R. W., Thorp, R. K., & Donohew, L. (1967). Content analysis of communications. New York: The Macmillan Co.
Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. Journal of Educational Psychology, 79(4), 474.
Covington, M. V. (2000). Goal theory, motivation, and school achievement: An integrative review. Annual Review of Psychology, 51, 171–200.
Creswell, J. W. (2013). Qualitative inquiry and research design: choosing among five approaches. Thousand Oaks, CA: Sage Publications.
Daniels, L. M., Haynes, T. L., Stupnisky, R. H., Perry, R. P., Newall, N. E., & Pekrun, R. (2008). Individual differences in achievement goals: A longitudinal study of cognitive, emotional, and achievement outcomes. Contemporary Educational Psychology, 33(4), 584–608.
DeCoster, J., Gallucci, M., & Iselin, A. M. R. (2011). Best practices for using median splits, artificial categorization, and their continuous alternatives. Journal of experimental psychopathology, 2(2), 197–209.
Defalco, J., Georgoulas-Sherry, V., Paquette, L., Baker, R., Rowe, J., Mott, B., & Lester, J. (2016). Motivational feedback messages as interventions to frustration in GIFT. Proceedings of the Fourth GIFT User Symposium (pp. 25–35).
Dominicis, S., Sokoloski, R., Jaeger, C. M., & Schultz, P. W. (2019). Making the smart meter social promotes long-term energy conservation. Palgrave Communications, 5(1), 1–8.
Duffy, M. C., & Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior, 52, 338–348.
Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology Copyright, 72(1), 218–232.
Elliot, A. J., & McGregor, H. A. (2001). A 2 X 2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519.
Elliot, A., Murayama, K., Kobeisy, A., & Lichtenfeld, S. (2015). Potential-based achievement goals. British Journal of Educational Psychology, 85(2), 192–206.
Eng, J. (2003). Sample size estimation: How many individuals should be studied? Radiology, 227(2), 309–313.
Fasihuddin, H., Skinner, G., & Athauda, R.(2016). A comprehensive review of adaptive, open, and online learning literature. International Conference on Computer Science Education Innovation & Technology (pp. 1–10).
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191.
Feng, S. L., & Tuan, H. L. (2005). Using ARCS Model to promote 11th graders motivation and achievement in learning about acids and bases. International Journal of Science and Mathematics Education, 3(3), 463–484.
Goldin, I., Narciss, S., Foltz, P., & Bauer, M. (2017). New directions in formative feedback in interactive learning environments. International Journal of Artificial Intelligence in Education, 27(3), 385–392.
Hancock, T. E., Stock, W. A., & Kulhavy, R. W. (1992). Predicting feedback effects from response- certitude estimates. Bulletin of the Psychonomic Society, 30(2), 173–176.
Harackiewicz, J. M., & Linnenbrink, E. A. (2005). Multiple achievement goals and multiple pathways for learning: The agenda and impact of Paul R. Pintrich. Educational Psychologist, 40(2), 75–84.
Hattie, J., & Timperley, H. (2007). The power of feedback. Medical Education, 77(1), 81–112.
Hedberg, J. G., & McNamara, S. E. (1985). Matching feedback and cognitive style in visual CAI tasks. The Meeting of the American Educational Research Association (pp. 1–29).
Hornstra, L., Majoor, M., & Peetsma, T. (2017). Achievement goal profiles and developments in effort and achievement in upper elementary school. British Journal of Educational Psychology, 87(4), 606–629.
Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20.
Luo, W., Paris, S. G., Hogan, D., & Luo, Z. (2011). Do performance goals promote learning? A pattern analysis of Singapore students’ achievement goals. Contemporary Educational Psychology, 36(2), 165–176.
Keller, J. M. (2006). Development of two measures of learner motivation. Florida State University.
Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention - A literature review. EdMedia: World Conference on Educational Media and Technology (pp. 1305–1313).
Kim, C., & Keller, J. M. (2007). Effects of motivational and volitional email messages (MVEM) with personal messages on undergraduate students? Motivation, study habits and achievement. British Journal of Educational Technology, 39(1), 36–51.
Kirschner, P. A. (2017). Stop propagating the learning styles myth. Computers & Education, 106, 166–171.
Koedinger, K. R., Brunskill, E., Baker, R. S., McLaughlin, E. A., & Stamper, J. (2013). New potentials for data-driven intelligent tutoring system development and optimization. AI Magazine, 34(3), 27–41.
Mahvelati, E. H. (2020). Field-Dependent/Field-Independent Learners’ Information Processing Behavior in an Implicit Learning Task: Evidence from Iranian EFL Learners. Journal of psycholinguistic research, 1–19.
Martinez, M. A. (1999). An investigation into successful learning: Measuring the impact of learning orientation, a primary learner-difference variable. Provo, UT: Brigham Young University.
Martinez, M. A. (2001). Key design considerations for personalized learning on the web. Educational Technology & Society, 4(1), 26–40.
Mascret, N., Elliot, A. J., & Cury, F. (2015). Extending the 3 × 2 achievement goal model to the sport domain: The 3 × 2 achievement goal questionnaire for sport. Psychology of Sport and Exercise, 17, 7–14.
Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Hicks Anderman, L., & Roeser, R. (1998). The Development and Validation of Scales Assessing Students’ Achievement Goal Orientations. Contemporary Educational Psychology, 23, 113–131.
Molden, D. C., Lucas, G. M., Finkel, E. J., Kumashiro, M., & Rusbult, C. (2009). Perceived support for promotion-focused and prevention-focused goals: Associations with well-being in unmarried and married couples. Psychological Science, 20(7), 787–793.
Mory, E. H. (1994). Adaptive feedback in computer-based instruction: Effects of response certitude on performance, feedback-study time, and efficiency. Educational Computing Research, 11(3), 263–290.
Nakic, J., Granic, A., & Glavinic, V. (2015). Anatomy of student models in adaptive learning systems: A systematic literature review of individual differences from 2001 to 2013. Journal of Educational Computing Research, 51(4), 459–489.
Narciss, S. (2004). The impact of informative tutoring feedback and self-efficacy on motivation and achievement in concept learning. Experimental Psychology, 51(3), 214–228.
Narciss, S. (2008). Feedback strategies for interactive learning tasks. In Handbook of Research on Educational Communications and Technology (4th ed., pp. 125–144). Springer.
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers and Education, 71, 56–76.
Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. Journal of Online Learning and Teaching, 11(2), 309–319.
Pailai, J., Wunnasri, W., Hayashi, Y., & Hirashima, T. (2018, June). Correctness-and confidence-based adaptive feedback of kit-build concept map with confidence tagging. In International Conference on Artificial Intelligence in Education (pp. 395–408). Springer, Cham.
Pintrich, P. R. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470.
Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92(3), 545–555.
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, M. Zeidner, & P. R. Pintrich (Eds.), Handbook of Self-Regulation (pp. 451–502). San Diego, CA: Academic Press.
Polso, K. M., Tuominen, H., Hellas, A., & Ihantola, P. (2020). Achievement goal orientation profiles and performance in a programming MOOC. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (pp. 411–417).
Privitera, G. J. (2016). Research methods for the behavioral sciences. Sage Publications.
Sadler, D. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84.
Schunk, D. H. (1995). Self-efficacy, motivation, and performance. Journal of Applied Sport Psychology, 7(2), 112–137.
Senko, C., & Harackiewicz, J. M. (2005). Regulation of achievement goals: The role of competence feedback. Journal of Educational Psychology, 97(3), 320–336.
Senko, C., & Miles, K. M. (2008). Pursuing their own learning agenda: How mastery-oriented students jeopardize their class performance. Contemporary Educational Psychology, 33(4), 561–583.
Senko, C. (2016). Achievement goal theory: A story of early promises, eventual discords, and future possibilities. Chapter to appear in K. Wentzel & D. Miele (Eds.), Handbook of Motivation at School (Vol. 2, pp. 75–95). New York, NY: Routledge.
Shin, J., Lee, Y., & Seo, E. (2017). The effects of feedback on students’ achievement goals: Interaction between reference of comparison and regulatory focus. Learning and Instruction, 49, 21–31.
Shin, T. S., & Dickson, W. P. (2010). The effects of peer- and self-referenced feedback on students’ motivation and academic performance in online learning environments. Journal of Online Learning and Teaching, 6(1), 1–11.
Sommet, N., & Elliot, A. J. (2017). Achievement goals, reasons for goal pursuit, and achievement goal complexes as predictors of beneficial outcomes: Is the influence of goals reducible to reasons? Journal of Educational Psychology, 109(8), 1141.
Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. (2015). Personalisation of MOOCs: The state of the art. Proceedings of the 7th International Conference on Computer Supported Education, Lisbon, Portugal (pp. 88–97).
Tobias, S. (1994). Interest, prior knowledge, and learning. Review of Educational Research, 64(1), 37–54.
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
Vasilyeva, E., De Bra, P., Pechenizkiy, M., & Puuronen, S. (2008). Tailoring feedback in online assessment: Influence of learning styles on the feedback preferences and elaborated feedback effectiveness. Eighth IEEE International Conference on Advanced Learning Technologies (pp. 834–838).
Vasilyeva, E., Pechenizkiy, M., & DeBra, P. (2008). Adaptation of elaborated feedback in e-learning. Lecture Notes in Computer Science, 5149, 235–244.
Weidinger, A. F., Spinath, B., & Steinmayr, R. (2016). Why does intrinsic motivation decline following negative feedback? The mediating role of ability self-concept and its moderation by goal orientations. Learning and Individual Differences, 47, 117–128.
Zimmerman, B. J., & Schunk, D. H. (2011). Motivational sources and outcomes of self-regulated learning and performance. Handbook of self-regulation of learning and performance, 5(3), 49–64.
We thank Tim Newby, Jennifer Richardson, and Judith Lewandowski who gave scientific guidance and participated in discussions.
Conflict of interest
We have no conflicts of interest to disclose.
Personalized online feedback study project.
Thank you for participating in the interview and completing the pre-survey in advance. I appreciate you taking the time out of your busy schedule! As a reminder the purpose of this study is to study the students' perception of the feedback they received from Instructors/TAs in their online classes and then we will use these responses to improve the feedback design.
Just to let you know participation is voluntary and you are free to withdraw at any time. The data collected for this research project will be kept confidential; once we have provided a transcript of your interview to you for verification of responses, your name will be removed from the data and you will not be identified as an individual in any way. Before we get started, do you have any questions? Do I have your permission to tape this session (once permission is granted then turn on recorder and verify so they are on "tape" granting permission)?
Part 1. Background information questions (You have responded to this question in the pre-survey. We may briefly go through this part in the interview.)
Have you taken any online graduate level courses previously? If so, what were they (category), when did you take them, and what institutions provided them?
Please describe your overall perceptions/feelings of the feedback received from these course instructors (TA). Prompt: Here are some examples of possible feedback, such as instructor feedback to the assignments, feedbacks in instructor email or feedback comments embedded in your online assignment documents, feedback information in course announcement, etc.
Could you talk about the positive/good/useful feedback examples and their features (in time, frequency, helpfulness…) and negative/useless feedback example and their features.
Part 2. Perceptions about the feedback in this course, EDCI [target course ID]
How do you feel about the feedback you received from the instructors/TAs in this course? (As you may still remember, in most of your assignments, you get instructor/TA's regular feedback text and some chart/graphs feedback.) Prompt: What is the impact of the course EDCI [target course ID] feedback on your learning motivation, the way you manage your learning process, your feelings during learning process, and the overall learning results?)
Was it (the course feedback) overall helpful for your learning online?
Can you talk about which aspects/parts of the feedback helped you?
Can you provide some examples?
If possible, would you show provide me the screenshots of these examples later?
How did it help you?
How did you use this feedback information in your following learning once you received them in your learning?
Why do you think it was helpful for you?
If some feedback may need to be improved,
Can you provide some feedback examples that need to improve? If possible, would you provide me the screenshots of these examples later?
If you are one of the TAs or instructors, what kind of feedback would you give to your students? Please give some the example(s)?
Part 3. Further suggestions to improve online feedback
What is the ideal feedback (good/useful feedback) in your opinions, what are the features (in time, frequency, usefulness, etc.) of ideal feedback?
How do you think the feedback you received in this course could be improved? What suggestions do you have?
What additional feedback information do you expect to get from the course instructor/TA?
Is there anything else you would like to share that may be related to effective feedback in the online learning that I have not asked?
Great! That's all for the interview.
Thank you again for your time in participating in this interview.
Please feel free to reach out to us by email or phone if you have any questions or concerns.
We will contact you for verification as we conduct the analyses.
About this article
Cite this article
Wang, H., Lehman, J.D. Using achievement goal-based personalized motivational feedback to enhance online learning. Education Tech Research Dev (2021). https://doi.org/10.1007/s11423-021-09940-3
- Personalized feedback
- Online learning
- Achievement goals