Skip to main content
SpringerLink
Log in

Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

The main learning activity provided by intelligent tutoring systems is problem solving, although several recent projects investigated the effectiveness of combining problem solving with worked examples. Previous research has shown that learning from examples is an effective learning strategy, especially for novice learners. A worked example provides step-by-step explanations of how a problem is solved. Many studies have compared learning from examples to unsupported problem solving, and suggested presenting worked examples to students in the initial stages of learning, followed by problem solving once students have acquired enough knowledge. This paper presents a study in which we compare a fixed sequence of alternating worked examples and tutored problem solving with a strategy that adapts learning tasks to students’ needs. The adaptive strategy determines the type of the task (a worked example, a faded example or a problem to be solved) based on how much assistance the student received on the previous problem. The results show that students in the adaptive condition learnt significantly more than their peers who were presented with a fixed sequence of worked examples and problem solving. Novices from the adaptive condition learnt faster than novices from the control group, while the advanced students from the adaptive condition learnt more than their peers from the control group.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Cognitive efficiency \(=\) Performance / Mental Effort Rating.

  2. Please note that P is scaled to the range [1,9].

  3. Normalised learning gain \(=\) (post test—pre test) / (max score—pre test).

References

  • Aleven, V., Koedinger, K.R.: An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor. Cognit. Sci. 26(2), 147–179 (2002)

    Article  Google Scholar 

  • Anthony, L., Mankoff, J., Mitchell, T., Gross, M.: Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning. Doctoral Dissertation, Carnegie Mellon University (2008)

  • Atkinson, R.K., Derry, S.J., Renkl, A., Wortham, D.: Learning from examples: instructional principles from the worked examples research. Rev. Educ. Res. 70(2), 181–214 (2000)

    Article  Google Scholar 

  • Brown, A.L., Kane, M.J.: Preschool children can learn to transfer: learning to learn and learning from example. Cognit. Psychol. 20(4), 493–523 (1988)

    Article  Google Scholar 

  • Brusilovsky, P.: Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for meta-adaptation. Br. J. Educ. Technol. 34(4), 487–497 (2003)

    Article  Google Scholar 

  • Burow, R., Weber, G.: Example explanation in learning environments. In: Frasson, C., Gauthier, G., Lesgoed, A. (eds.) Proceedings of the 3rd International Conference Intelligent Tutoring Systems, Montreal, Canada, pp. 457–465. Springer, Berlin (1996)

    Google Scholar 

  • Chandler, P., Sweller, J.: Cognitive load theory and the format of instruction. Cognit. Instr. 8(4), 293–332 (1991)

    Article  Google Scholar 

  • Chi, M., De Leeuw, N., Chiu, M.H., LaVancher, C.: Eliciting self-explanations improves understanding. Cognit. Sci. 18(3), 439–477 (1994)

    Google Scholar 

  • Chi, M.T.H., Bassok, M., Lewis, M.W., Reimann, P., Glaser, R.: Self-explanations: how students study and use examples in learning to solve problems. Cognit. Sci. 13(2), 145–182 (1989)

    Article  Google Scholar 

  • Chi, M.T.H., VanLehn, K.A.: The content of physics self-explanations. J. Learn. Sci. 1(1), 69–105 (1991)

    Article  Google Scholar 

  • Clark, R.C., Nguyen, F., Sweller, J.: Efficiency in learning: evidence-based guidelines to manage cognitive load. Jossey-Bass, San Francisco (2006)

    Google Scholar 

  • Corbett, A., Kauffman, L., MacLaren, B., Wagner, A., Jones, E.: A cognitive tutor for genetics problem solving: learning gains and student modeling. J Educ Comput Res. 42, 219–239 (2010)

    Article  Google Scholar 

  • Corbett, A., MacLaren, B., Wagner, A., Kauffman, L., Mitchell, A., d Baker, R.S.: Differential Impact of Learning Activities Designed to Support Robust Learning in the Genetics Cognitive Tutor. In: Lane, H.C., Yacef, K., Mostow, J. (Eds.) Proceedings of 16th Artificial Intelligence in Education, Memphis, pp. 319–328 (2013)

  • Davidovic, A., Warren, J., Trichina, E.: Learning benefits of structural example based adaptive tutoring systems. IEEE Trans Educ. 46(2), 241–251 (2003)

    Article  Google Scholar 

  • Deeks, A.: Web Based Assignments in Structural Analysis. Nanyang Technological University, Deeks, Centre for Educational Development (2000)

    Google Scholar 

  • Hattie, J.: Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. Routledge, New York (2009)

    Google Scholar 

  • Hausmann, R., Nokes, T., VanLehn, K., Gershman, S.: The design of self-explanation prompts: the fit hypothesis. In: Proceedings of 31st Annual Conference of the Cognitive Science Society, pp. 2626–2631 (2009)

  • Hilbert, T.S., Renkl, A.: Learning how to use a computer-based concept-mapping tool: self-explaining examples helps. Comput. Hum. Behav. 25(2), 267–274 (2009)

    Article  Google Scholar 

  • Hsiao, I.H., Bakalov, F., Brusilovsky, P., König-Ries, B.: Progressor: social navigation support through open social student modeling. New Rev. Hypermed. Multimed. 19(2), 112–131 (2013)

    Article  Google Scholar 

  • Kalyuga, S.: Expertise reversal effect and its implications for learner-tailored instruction. Educ. Psychol. Rev. 19, 509–539 (2007)

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., Tuovinen, J., Sweller, J.: When problem solving is superior to studying worked examples. Educ. Psychol. 93(3), 579–588 (2001)

    Article  Google Scholar 

  • Kalyuga, S., Sweller, J.: Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning. Educ. Technol. Res. Dev. 53(3), 83–93 (2005)

    Article  Google Scholar 

  • Karampiperis, P., Sampson, D.G.: Adaptive Instructional Planning Using Ontologies. In: Proceedings of 4th IEEE International Conference Advanced Learning Technologies Joensuu, Vol. 4, pp. 126-1 (2004)

  • Kim, R.S., Weitz, R., Heffernan, N.T., Krach, N.: Tutored Problem Solving vs . “ Pure ” Worked Examples. In: Proceedings of 31st Annual Conference of the Cognitive Science Society, pp. 3121–3126. Cognitive Science Society, Austin, TX (2007)

  • Koedinger, K., Aleven, V.: Exploring the assistance dilemma in experiments with cognitive tutors. Educ. Psychol. Rev. 19(3), 239–264 (2007)

    Article  Google Scholar 

  • Kostons, D., van Gog, T., Paas, F.: Self-assessment and task selection in learner-controlled instruction: differences between effective and ineffective learners. Comput. Educ. 54(4), 932–940 (2010)

    Article  Google Scholar 

  • Kyun, S., Kalyuga, S., Sweller, J.: The effect of worked examples when learning to write essays in English literature. J. Exp. Educ 81(3), 385–408

  • Martin, B., Mitrovic, A.: Using learning curves to mine student models. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) Proceedings of 10th International Conference User Modeling, pp. 79–88. Springer, Edinburgh (2005)

    Google Scholar 

  • Masthoff, J.: Design and evaluation of a navigation agent with a mixed locus of control. In: Cerri, S.A., Gouarderes, G., Paraguacu, F. (eds.) Proceedings of International Conference Intelligent Tutoring Systems, pp. 982–991. Springer, Berlin (2002)

    Chapter  Google Scholar 

  • McLaren, B.M., van Gog, T., Ganoe, C., Karabinos, M., Yaron, D.: The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in classroom experiments. Comput. Hum. Behav. 55, 87–99 (2016)

    Article  Google Scholar 

  • McLaren, B., Isotani, S.: When Is It Best to Learn with All Worked Examples? In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) Proceedings of 15th International Conference Artificial Intelligence in Education, pp. 222–229. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • McLaren, B.M., Lim, S.-J., Koedinger, K.R.: When and how often should worked examples be given to students? New results and a summary of the current state of research. In: Proceedings of Conference Cognitive Science Society, pp. 2176–2181. Cognitive Science Society, Austin, TX (2008)

  • Mitrovic, A.: An intelligent SQL tutor on the web. Int. J. Artif. Intell. Educ. 13(2–4), 173–197 (2003)

    Google Scholar 

  • Mitrovic, A., Martin, B.: Evaluating the effectiveness of feedback in SQL-Tutor. In: Kinshuk, Jesshope, C., Okamoto, T. (eds.) Proceedings International Workshop on Advanced Learning Technologies, pp. 143–144. Palmerston North (2000)

  • Mitrovic, A., Martin, B.: Scaffolding and fading problem selection in SQL-Tutor. In: Hoppe, U., Verdejo, F., Kay, J. (eds.) Proceedings of 11th International Conference Artificial Intelligence in Education, pp. 479–481. Australia, Sydney (2003)

    Google Scholar 

  • Mitrovic, A., Weerasinghe, A.: Revisiting Ill-Definedness and the Consequences for ITSs. In: Dimitrova, V., Mizoguchi, R. (eds.) Proceedings of 14th International Conference on Artificial Intelligence in Education, pp. 375–382. IOS Press, Brighton (2009)

    Google Scholar 

  • Najar, A.S., Mitrovic, A.: Do novices and advanced students benefit differently from worked examples and ITS? In: Wong, L.H., Liu, C.-C., Hirashima, T., Sumedi, P., Lukman, M. (eds.) Proceedings of International Conference ICCE, pp. 20–29. Bali (2013a)

  • Najar, A.S., Mitrovic, A.: In: Yacef, K., et al. (eds.) Examples and Tutored Problems: How can Self-Explanation Make a Difference to Learning? Proceedings of 16th International Conference AIED, pp. 339–348. Springer, Heidelberg (2013b)

  • Najar, A.S., Mitrovic, A.: Examples and tutored problems: is alternating examples and problems the best instructional strategy? Res. Pract. Technol. Enhanc. Learn. 9(3), 439–459 (2014)

    Google Scholar 

  • Najar, A.S., Mitrovic, A., McLaren, B.M.: Adaptive support versus alternating worked examples and tutored problems: which leads to better learning? In: Dimitrova, V., et al. (eds.) User Modeling, Adaptation, and Personalization, pp. 171–182. Springer International Publishing, Aalborg (2014)

    Google Scholar 

  • Najar, A.S., Mitrovic, A., McLaren, B.M.: Examples and tutored problems: adaptive support using assistance scores. In: Proceedings of 24th International Joint Conference on Artificial Intelligence Buenos Aires, pp. 4317-4323. AAAI Press (2015)

  • Paas, F.G.W.C., Van Merrienboer, J.J.G.: The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Hum. Factors 35, 737–743 (1993)

    Google Scholar 

  • Peachey, D., McCalla, G.: Using planning techniques in intelligent tutoring systems. Int. J. Man-Mach. Stud. 24, 77–98 (1986)

    Article  Google Scholar 

  • Roediger, H.L., Karpicke, J.D.: Test-enhanced learning taking memory tests improves long-term retention. Psychol. Sci. 17(3), 249–255 (2006)

    Article  Google Scholar 

  • Rourke, A., Sweller, J.: The worked-example effect using ill-defined problems: learning to recognise designers’ styles. Learn. Instr. 19(2), 185–199 (2009)

    Article  Google Scholar 

  • Salden, R., Aleven, V., Renkl, A., Schwonke, R.: Worked examples and tutored problem solving: redundant or synergistic forms of support? Topics Cognit. Sci. 1(1), 203–213 (2009)

    Article  Google Scholar 

  • Salden, R., Koedinger, K., Renkl, A., Aleven, V., McLaren, B.: Accounting for beneficial effects of worked examples in tutored problem solving. Educ. Psychol. Rev. 22(4), 379–392 (2010)

    Article  Google Scholar 

  • Santos, O.C.: (Ed.) Educational Recommender Systems and Technologies: Practices and Challenges: Practices and Challenges. IGI Global, Hershey (2011)

  • Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., Salden, R.: The worked-example effect: not an artefact of lousy control conditions. Comput. Hum. Behav. 25, 258–266 (2009)

    Article  Google Scholar 

  • Schworm, S., Renkl, A.: Computer-supported example-based learning: when instructional explanations reduce self-explanations. Comput. Educ. 46, 426–445 (2006)

    Article  Google Scholar 

  • Sosnovsky, S., Brusilovsky, P.: Evaluation of topic-based adaptation and student modeling in QuizGuide. User Model. User-Adapt. Interact. 25(4), 371–424 (2015)

    Article  Google Scholar 

  • Sweller, J.: The worked example effect and human cognition. Learn. Instr. 16(2), 165–169 (2006)

    Article  Google Scholar 

  • Sweller, J., Ayres, P., Kalyuga, S.: Cognitive Load Theory. Springer, New York (2011)

    Book  Google Scholar 

  • Sweller, J., Cooper, G.A.: The use of worked examples as a substitute for problem solving in learning algebra. Cognit. Instr. 2, 59–89 (1985)

    Article  Google Scholar 

  • Trafton, J. G., Reiser, B. J.: The Contributions of Studying Examples and Solving Problems to Skill Acquisition. In: Proceedings of 15th Annual Conference of the Cognitive Science Society, ACM Press, pp. 1017–1022 (1993)

  • Van Gog, T.: Effects of identical example-problem and problem-example pairs on learning. Comput. Educ. 57(2), 1775–1779 (2011)

    Article  MathSciNet  Google Scholar 

  • Van Gog, T., Kester, L., Paas, F.: Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemp. Educ. Psychol. 36(3), 212–218 (2011)

    Article  Google Scholar 

  • Van Gog, T., Paas, F.: Instructional efficiency: revisiting the original construct in educational research. Educ. Psychol. 43(1), 16–26 (2008)

    Article  Google Scholar 

  • Van Gog, T., Rummel, N.: Example-based learning: integrating cognitive and social-cognitive research perspectives. Educ. Psychol. Rev. 22(2), 155–174 (2010)

    Article  Google Scholar 

  • Vassileva, J.: Reactive instructional planning to support interacting teaching strategies. In: Proceedings of 7th Conference Artificial Intelligence in Education, pp. 334–342 (1995)

  • Webb, N.M.: Peer interaction and learning in small groups. Educ. Res. 13(1), 21–39 (1989)

    Google Scholar 

  • Weber, G.: Examples and remindings in a case-based help system. In: European Workshop on Advances in Case-Based Reasoning, pp. 165–177. Springer, Berlin (1994)

  • Weber, G., Brusilovsky, P.: ELM-ART-an interactive and intelligent web-based electronic textbook. Int. J. Artif. Intell. Educ. 26(1), 72–81 (2016)

    Article  Google Scholar 

  • Weerasinghe, A., Mitrovic, A.: Facilitating deep learning through self-explanation in an open-ended domain. Knowl.-Based Intell. Eng. Syst. 10(1), 3–19 (2006)

    Google Scholar 

  • Weerasinghe, A., Mitrovic, A., Martin, B.: Towards individualized dialogue support for ill-defined domains. Int. J. Artifici. Intell. Educ. 19(4), 357–379 (2009)

    Google Scholar 

  • Weerasinghe, A., Mitrovic, A.: Facilitating adaptive tutorial dialogues in EER-tutor. In: International Conference on Artificial Intelligence in Education, pp. 630–631. Springer, Berlin (2011)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Software Engineering, Intelligent Computer Tutoring Group, University of Canterbury, Christchurch, New Zealand

    Amir Shareghi Najar & Antonija Mitrovic

  2. Department of Computer Science and Software Engineering, Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, USA

    Bruce M. McLaren

Authors
  1. Amir Shareghi Najar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonija Mitrovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruce M. McLaren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonija Mitrovic.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Najar, A.S., Mitrovic, A. & McLaren, B.M. Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum. User Model User-Adap Inter 26, 459–491 (2016). https://doi.org/10.1007/s11257-016-9181-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-016-9181-y

Keywords

Search

Navigation