Using achievement goal-based personalized motivational feedback to enhance online learning

Abstract

Current online learning approaches are sometimes criticized for a “one- size- fits -all” approach and insufficient feedback, which may result in low levels of satisfaction and high dropout rates. To mitigate these shortcomings, we implemented a set of principles for designing personalized motivational feedback based on students’ achievement goals. A sequential explanatory mixed-methods design was used to collect data. The results indicated students who received this feedback demonstrated significantly higher levels of motivation and satisfaction. However, their performance scores were not significantly higher than students in the control group. Post-interviews showed this personalized feedback helped students regulate their learning goals and behaviors and thus improved their learning. The evidence suggests this personalized feedback may benefit learning by improving motivation. While its effect on performance outcomes was not significant, we speculate that more complex factors may have obscured any performance effect.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    Note: Learning style was criticized later as an invalid learning inctruct. See more details on page 7.

References

  1. Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help-seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.

    Google Scholar 

  2. Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2017). Instruction based on adaptive learning technologies. In R. E. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction (2nd ed., pp. 522–560). New York: Routledge.

    Google Scholar 

  3. Allen, E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment report 2017. Babson Park, MA: Babson Survey Research Group.

    Google Scholar 

  4. Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84(3), 261–271.

    Article  Google Scholar 

  5. An, D., & Carr, M. (2017). Learning styles theory fails to explain learning and achievement: Recommendations for alternative approaches. Personality and Individual Differences, 116, 410–416.

    Article  Google Scholar 

  6. Arroyo, I., Beck, J. E., Beal, C. R., Wing, R., & Woolf, B. P. (2001). Analyzing students’ response to help provision in an elementary mathematics intelligent tutoring system. Papers of the AIED-2001 workshop on help provision and help-seeking in interactive learning environments, 34–46.

  7. Bawa, P. (2016). Retention in online courses: exploring issues and solutions. SAGE Open, 6(1), 1–11.

    Article  Google Scholar 

  8. Berlin, N., & Dargnies, M. P. (2016). Gender differences in reactions to feedback and willingness to compete. Journal of Economic Behavior & Organization, 130, 320–336.

    Article  Google Scholar 

  9. Brusilovsky, P. (2001). Adaptive educational hypermedia. Proceedings of Tenth International PEG Conference, 8–12.

  10. Budd, R. W., Thorp, R. K., & Donohew, L. (1967). Content analysis of communications. New York: The Macmillan Co.

    Google Scholar 

  11. Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. Journal of Educational Psychology, 79(4), 474.

    Article  Google Scholar 

  12. Covington, M. V. (2000). Goal theory, motivation, and school achievement: An integrative review. Annual Review of Psychology, 51, 171–200.

    Article  Google Scholar 

  13. Creswell, J. W. (2013). Qualitative inquiry and research design: choosing among five approaches. Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  14. Daniels, L. M., Haynes, T. L., Stupnisky, R. H., Perry, R. P., Newall, N. E., & Pekrun, R. (2008). Individual differences in achievement goals: A longitudinal study of cognitive, emotional, and achievement outcomes. Contemporary Educational Psychology, 33(4), 584–608.

    Article  Google Scholar 

  15. DeCoster, J., Gallucci, M., & Iselin, A. M. R. (2011). Best practices for using median splits, artificial categorization, and their continuous alternatives. Journal of experimental psychopathology, 2(2), 197–209.

    Article  Google Scholar 

  16. Defalco, J., Georgoulas-Sherry, V., Paquette, L., Baker, R., Rowe, J., Mott, B., & Lester, J. (2016). Motivational feedback messages as interventions to frustration in GIFT. Proceedings of the Fourth GIFT User Symposium (pp. 25–35).

  17. Dominicis, S., Sokoloski, R., Jaeger, C. M., & Schultz, P. W. (2019). Making the smart meter social promotes long-term energy conservation. Palgrave Communications, 5(1), 1–8.

    Article  Google Scholar 

  18. Duffy, M. C., & Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior, 52, 338–348.

    Article  Google Scholar 

  19. Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology Copyright, 72(1), 218–232.

    Article  Google Scholar 

  20. Elliot, A. J., & McGregor, H. A. (2001). A 2 X 2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519.

    Article  Google Scholar 

  21. Elliot, A., Murayama, K., Kobeisy, A., & Lichtenfeld, S. (2015). Potential-based achievement goals. British Journal of Educational Psychology, 85(2), 192–206.

    Article  Google Scholar 

  22. Eng, J. (2003). Sample size estimation: How many individuals should be studied? Radiology, 227(2), 309–313.

    Article  Google Scholar 

  23. Fasihuddin, H., Skinner, G., & Athauda, R.(2016). A comprehensive review of adaptive, open, and online learning literature. International Conference on Computer Science Education Innovation & Technology (pp. 1–10).

  24. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191.

    Article  Google Scholar 

  25. Feng, S. L., & Tuan, H. L. (2005). Using ARCS Model to promote 11th graders motivation and achievement in learning about acids and bases. International Journal of Science and Mathematics Education, 3(3), 463–484.

    Article  Google Scholar 

  26. Goldin, I., Narciss, S., Foltz, P., & Bauer, M. (2017). New directions in formative feedback in interactive learning environments. International Journal of Artificial Intelligence in Education, 27(3), 385–392.

    Article  Google Scholar 

  27. Hancock, T. E., Stock, W. A., & Kulhavy, R. W. (1992). Predicting feedback effects from response- certitude estimates. Bulletin of the Psychonomic Society, 30(2), 173–176.

    Article  Google Scholar 

  28. Harackiewicz, J. M., & Linnenbrink, E. A. (2005). Multiple achievement goals and multiple pathways for learning: The agenda and impact of Paul R. Pintrich. Educational Psychologist, 40(2), 75–84.

    Article  Google Scholar 

  29. Hattie, J., & Timperley, H. (2007). The power of feedback. Medical Education, 77(1), 81–112.

    Google Scholar 

  30. Hedberg, J. G., & McNamara, S. E. (1985). Matching feedback and cognitive style in visual CAI tasks. The Meeting of the American Educational Research Association (pp. 1–29).

  31. Hornstra, L., Majoor, M., & Peetsma, T. (2017). Achievement goal profiles and developments in effort and achievement in upper elementary school. British Journal of Educational Psychology, 87(4), 606–629.

    Article  Google Scholar 

  32. Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20.

    Article  Google Scholar 

  33. Luo, W., Paris, S. G., Hogan, D., & Luo, Z. (2011). Do performance goals promote learning? A pattern analysis of Singapore students’ achievement goals. Contemporary Educational Psychology, 36(2), 165–176.

    Article  Google Scholar 

  34. Keller, J. M. (2006). Development of two measures of learner motivation. Florida State University.

  35. Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention - A literature review. EdMedia: World Conference on Educational Media and Technology (pp. 1305–1313).

  36. Kim, C., & Keller, J. M. (2007). Effects of motivational and volitional email messages (MVEM) with personal messages on undergraduate students? Motivation, study habits and achievement. British Journal of Educational Technology, 39(1), 36–51.

    Google Scholar 

  37. Kirschner, P. A. (2017). Stop propagating the learning styles myth. Computers & Education, 106, 166–171.

    Article  Google Scholar 

  38. Koedinger, K. R., Brunskill, E., Baker, R. S., McLaughlin, E. A., & Stamper, J. (2013). New potentials for data-driven intelligent tutoring system development and optimization. AI Magazine, 34(3), 27–41.

    Article  Google Scholar 

  39. Mahvelati, E. H. (2020). Field-Dependent/Field-Independent Learners’ Information Processing Behavior in an Implicit Learning Task: Evidence from Iranian EFL Learners. Journal of psycholinguistic research, 1–19.

  40. Martinez, M. A. (1999). An investigation into successful learning: Measuring the impact of learning orientation, a primary learner-difference variable. Provo, UT: Brigham Young University.

    Google Scholar 

  41. Martinez, M. A. (2001). Key design considerations for personalized learning on the web. Educational Technology & Society, 4(1), 26–40.

    Google Scholar 

  42. Mascret, N., Elliot, A. J., & Cury, F. (2015). Extending the 3 × 2 achievement goal model to the sport domain: The 3 × 2 achievement goal questionnaire for sport. Psychology of Sport and Exercise, 17, 7–14.

    Article  Google Scholar 

  43. Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Hicks Anderman, L., & Roeser, R. (1998). The Development and Validation of Scales Assessing Students’ Achievement Goal Orientations. Contemporary Educational Psychology, 23, 113–131.

    Article  Google Scholar 

  44. Molden, D. C., Lucas, G. M., Finkel, E. J., Kumashiro, M., & Rusbult, C. (2009). Perceived support for promotion-focused and prevention-focused goals: Associations with well-being in unmarried and married couples. Psychological Science, 20(7), 787–793.

    Article  Google Scholar 

  45. Mory, E. H. (1994). Adaptive feedback in computer-based instruction: Effects of response certitude on performance, feedback-study time, and efficiency. Educational Computing Research, 11(3), 263–290.

    Article  Google Scholar 

  46. Nakic, J., Granic, A., & Glavinic, V. (2015). Anatomy of student models in adaptive learning systems: A systematic literature review of individual differences from 2001 to 2013. Journal of Educational Computing Research, 51(4), 459–489.

    Article  Google Scholar 

  47. Narciss, S. (2004). The impact of informative tutoring feedback and self-efficacy on motivation and achievement in concept learning. Experimental Psychology, 51(3), 214–228.

    Article  Google Scholar 

  48. Narciss, S. (2008). Feedback strategies for interactive learning tasks. In Handbook of Research on Educational Communications and Technology (4th ed., pp. 125–144). Springer.

  49. Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers and Education, 71, 56–76.

    Article  Google Scholar 

  50. Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. Journal of Online Learning and Teaching, 11(2), 309–319.

    Google Scholar 

  51. Pailai, J., Wunnasri, W., Hayashi, Y., & Hirashima, T. (2018, June). Correctness-and confidence-based adaptive feedback of kit-build concept map with confidence tagging. In International Conference on Artificial Intelligence in Education (pp. 395–408). Springer, Cham.

  52. Pintrich, P. R. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470.

    Article  Google Scholar 

  53. Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92(3), 545–555.

    Article  Google Scholar 

  54. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, M. Zeidner, & P. R. Pintrich (Eds.), Handbook of Self-Regulation (pp. 451–502). San Diego, CA: Academic Press.

    Google Scholar 

  55. Polso, K. M., Tuominen, H., Hellas, A., & Ihantola, P. (2020). Achievement goal orientation profiles and performance in a programming MOOC. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (pp. 411–417).

  56. Privitera, G. J. (2016). Research methods for the behavioral sciences. Sage Publications.

  57. Sadler, D. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84.

    Google Scholar 

  58. Schunk, D. H. (1995). Self-efficacy, motivation, and performance. Journal of Applied Sport Psychology, 7(2), 112–137.

    Article  Google Scholar 

  59. Senko, C., & Harackiewicz, J. M. (2005). Regulation of achievement goals: The role of competence feedback. Journal of Educational Psychology, 97(3), 320–336.

    Article  Google Scholar 

  60. Senko, C., & Miles, K. M. (2008). Pursuing their own learning agenda: How mastery-oriented students jeopardize their class performance. Contemporary Educational Psychology, 33(4), 561–583.

    Article  Google Scholar 

  61. Senko, C. (2016). Achievement goal theory: A story of early promises, eventual discords, and future possibilities. Chapter to appear in K. Wentzel & D. Miele (Eds.), Handbook of Motivation at School (Vol. 2, pp. 75–95). New York, NY: Routledge.

  62. Shin, J., Lee, Y., & Seo, E. (2017). The effects of feedback on students’ achievement goals: Interaction between reference of comparison and regulatory focus. Learning and Instruction, 49, 21–31.

    Article  Google Scholar 

  63. Shin, T. S., & Dickson, W. P. (2010). The effects of peer- and self-referenced feedback on students’ motivation and academic performance in online learning environments. Journal of Online Learning and Teaching, 6(1), 1–11.

    Google Scholar 

  64. Sommet, N., & Elliot, A. J. (2017). Achievement goals, reasons for goal pursuit, and achievement goal complexes as predictors of beneficial outcomes: Is the influence of goals reducible to reasons? Journal of Educational Psychology, 109(8), 1141.

    Article  Google Scholar 

  65. Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. (2015). Personalisation of MOOCs: The state of the art. Proceedings of the 7th International Conference on Computer Supported Education, Lisbon, Portugal (pp. 88–97).

  66. Tobias, S. (1994). Interest, prior knowledge, and learning. Review of Educational Research, 64(1), 37–54.

    Article  Google Scholar 

  67. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.

    Article  Google Scholar 

  68. Vasilyeva, E., De Bra, P., Pechenizkiy, M., & Puuronen, S. (2008). Tailoring feedback in online assessment: Influence of learning styles on the feedback preferences and elaborated feedback effectiveness. Eighth IEEE International Conference on Advanced Learning Technologies (pp. 834–838).

  69. Vasilyeva, E., Pechenizkiy, M., & DeBra, P. (2008). Adaptation of elaborated feedback in e-learning. Lecture Notes in Computer Science, 5149, 235–244.

    Article  Google Scholar 

  70. Weidinger, A. F., Spinath, B., & Steinmayr, R. (2016). Why does intrinsic motivation decline following negative feedback? The mediating role of ability self-concept and its moderation by goal orientations. Learning and Individual Differences, 47, 117–128.

    Article  Google Scholar 

  71. Zimmerman, B. J., & Schunk, D. H. (2011). Motivational sources and outcomes of self-regulated learning and performance. Handbook of self-regulation of learning and performance, 5(3), 49–64.

    Google Scholar 

Download references

Acknowledgement

We thank Tim Newby, Jennifer Richardson, and Judith Lewandowski who gave scientific guidance and participated in discussions.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Huanhuan Wang.

Ethics declarations

Conflict of interest

We have no conflicts of interest to disclose.

Appendices

Appendix A

Interview protocol

Personalized online feedback study project.

Interview protocol

Thank you for participating in the interview and completing the pre-survey in advance. I appreciate you taking the time out of your busy schedule! As a reminder the purpose of this study is to study the students' perception of the feedback they received from Instructors/TAs in their online classes and then we will use these responses to improve the feedback design.

Just to let you know participation is voluntary and you are free to withdraw at any time. The data collected for this research project will be kept confidential; once we have provided a transcript of your interview to you for verification of responses, your name will be removed from the data and you will not be identified as an individual in any way. Before we get started, do you have any questions? Do I have your permission to tape this session (once permission is granted then turn on recorder and verify so they are on "tape" granting permission)?

Part 1. Background information questions (You have responded to this question in the pre-survey. We may briefly go through this part in the interview.)

  1. 1.

    Have you taken any online graduate level courses previously? If so, what were they (category), when did you take them, and what institutions provided them?

  2. 2.

    Please describe your overall perceptions/feelings of the feedback received from these course instructors (TA). Prompt: Here are some examples of possible feedback, such as instructor feedback to the assignments, feedbacks in instructor email or feedback comments embedded in your online assignment documents, feedback information in course announcement, etc.

  3. 3.

    Could you talk about the positive/good/useful feedback examples and their features (in time, frequency, helpfulness…) and negative/useless feedback example and their features.

Part 2. Perceptions about the feedback in this course, EDCI [target course ID]

  1. 1)

    How do you feel about the feedback you received from the instructors/TAs in this course? (As you may still remember, in most of your assignments, you get instructor/TA's regular feedback text and some chart/graphs feedback.) Prompt: What is the impact of the course EDCI [target course ID] feedback on your learning motivation, the way you manage your learning process, your feelings during learning process, and the overall learning results?)

  2. 2)

    Was it (the course feedback) overall helpful for your learning online?

    If yes,

    1. 1)

      Can you talk about which aspects/parts of the feedback helped you?

    2. 2)

      Can you provide some examples?

      If possible, would you show provide me the screenshots of these examples later?

    3. 3)

      How did it help you?

      How did you use this feedback information in your following learning once you received them in your learning?

    4. 4)

      Why do you think it was helpful for you?

If some feedback may need to be improved,

  1. 1)

    Can you provide some feedback examples that need to improve? If possible, would you provide me the screenshots of these examples later?

  2. 2)

    If you are one of the TAs or instructors, what kind of feedback would you give to your students? Please give some the example(s)?

Part 3. Further suggestions to improve online feedback

  1. 1)

    What is the ideal feedback (good/useful feedback) in your opinions, what are the features (in time, frequency, usefulness, etc.) of ideal feedback?

  2. 2)

    How do you think the feedback you received in this course could be improved? What suggestions do you have?

  3. 3)

    What additional feedback information do you expect to get from the course instructor/TA?

Closing out

Is there anything else you would like to share that may be related to effective feedback in the online learning that I have not asked?

Great! That's all for the interview.

Thank you again for your time in participating in this interview.

Please feel free to reach out to us by email or phone if you have any questions or concerns.

We will contact you for verification as we conduct the analyses.

Thank you!

Appendix B

Pre-interview Questionnaire

figured
figuree
figuref

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Lehman, J.D. Using achievement goal-based personalized motivational feedback to enhance online learning. Education Tech Research Dev (2021). https://doi.org/10.1007/s11423-021-09940-3

Download citation

Keywords

  • Personalized feedback
  • Online learning
  • Motivation
  • Effectiveness
  • Achievement goals