Skip to main content

Advertisement

SpringerLink
Log in

Exploring the Assistance Dilemma in Experiments with Cognitive Tutors

  • Original Article
  • Published:
Educational Psychology Review Aims and scope Submit manuscript

Abstract

Intelligent tutoring systems are highly interactive learning environments that have been shown to improve upon typical classroom instruction. Cognitive Tutors are a type of intelligent tutor based on cognitive psychology theory of problem solving and learning. Cognitive Tutors provide a rich problem-solving environment with tutorial guidance in the form of step-by-step feedback, specific messages in response to common errors, and on-demand instructional hints. They also select problems based on individual student performance. The learning benefits of these forms of interactivity are supported, to varying extents, by a growing number of results from experimental studies. As Cognitive Tutors have matured and are being applied in new subject-matter areas, they have been used as a research platform and, particularly, to explore interactive methods to support metacognition. We review experiments with Cognitive Tutors that have compared different forms of interactivity and we reinterpret their results as partial answers to the general question: How should learning environments balance information or assistance giving and withholding to achieve optimal student learning? How best to achieve this balance remains a fundamental open problem in instructional science. We call this problem the “assistance dilemma” and emphasize the need for further science to yield specific conditions and parameters that indicate when and to what extent to use information giving versus information withholding forms of interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. While texts come with an at least implicit request to have students try to understand them, they do not explicitly request students to produce a written or spoken output. The student “learning event” that may result from a passive instructional event may be active (e.g., taking notes), but passive instructional events do not explicitly prompt for such activity.

  2. This effort was not the first to incorporate examples in Cognitive Tutors as the original LISP tutor had hypercard declarative instruction and examples interspersed with problem-solving practice in the tutor. Use of worked examples in the Cognitive Tutor Algebra course, both in text materials and in the tutor, were discouraged by our collaborating instructors (cf., Koedinger et al. 1997) because it was thought that urban students, if they processed the examples at all, would do so shallowly, which would impede deeper conceptual understanding that might better come from classroom discussion and collaborative projects.

  3. As described above, even excellent college students at Carnegie Mellon University benefited in time savings from the immediate feedback of the LISP tutor.

  4. In fact, immediate feedback tutors do allow students to learn from their errors—after all students are required to correct their errors and can and often do so without further assistance. However, they are not given the opportunity to learn from the downstream consequences of their errors.

References

  • Aleven, V., & Koedinger, K. R. (2000a). Limitations of student control: Do students know when they need help? In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Proceedings of the 5th International Conference on Intelligent Tutoring Systems, ITS 2000 (pp. 292–303). Berlin: Springer.

    Google Scholar 

  • Aleven, V., & Koedinger, K. R. (2000b). The need for tutorial dialog to support self-explanation. In C. P. Rose & R. Freedman (Eds.), Building dialogue systems for tutorial applications, papers of the 2000 AAAI Fall Symposium (pp. 65–73). Technical Report FS-00-01. Menlo Park, CA: AAAI.

  • Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2):147–179.

    Article  Google Scholar 

  • Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence and Education, 16, 101–128.

    Google Scholar 

  • Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. M. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(2), 277–320.

    Article  Google Scholar 

  • Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition and the Lisp Tutor. Cognitive Science, 13, 467–505.

    Article  Google Scholar 

  • Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), 167–207.

    Article  Google Scholar 

  • Anderson, J. R., & Lebiere, C. (1998). The Atomic Components of Thought. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated learning and education. Educational Researcher, 25(4), 5–11.

    Google Scholar 

  • Anderson, J. R., Reder, L. M., & Simon, H. A. (1998). Radical constructivism and cognitive psychology. In D. Ravitch (Ed.), Brookings papers on education policy 1998. Washington, DC: Brookings Institute.

    Google Scholar 

  • Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.

    Google Scholar 

  • Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of Educational Psychology, 95(4), 774–783.

    Article  Google Scholar 

  • Baker, R., Corbett, A., Koedinger, K. R., Evenson, S., Roll, I., Wagner, A., et al. (2006). Adapting to when students game an intelligent tutoring system. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 392–401). Berlin: Springer.

    Google Scholar 

  • Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the Cognitive Tutor classroom: When students “game the system.” Proceedings of ACM CHI 2004: Computer-Human Interaction (pp. 383–390). New York: ACM.

  • Bloom, B. S. (1984). The 2-sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 4–16.

    Google Scholar 

  • Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.

    Article  Google Scholar 

  • Chi, M. T. H., de Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.

    Article  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2003). e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. San Francisco: Jossey-Bass.

    Google Scholar 

  • Corbett, A. (2001). Cognitive computer tutors: solving the two-sigma problem. In M. Bauer, P. J. Gmytrasiewicz, & J. Vassileva (Eds.), Proceedings of the 2001 International Conference on User Modeling (pp. 137–147). Berlin: Springer.

    Google Scholar 

  • Corbett, A., & Anderson, J. R. (1995). Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253–278.

    Article  Google Scholar 

  • Crowley, R. S., Legowski, E., Medvedeva, O., & Tseytlin, E. (2005). An ITS for medical classification problem-solving: Effects of tutoring and representations. In C. K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.), Proceedings of the 12th International Conference on Artificial Intelligence. Amsterdam, The Netherlands: AIED.

  • Crowley, R. S., & Medvedeva, O. (2006). An intelligent tutoring system for visual classification problem solving. Artificial Intelligence in Medicine, 36(1), 85–117.

    Article  PubMed  Google Scholar 

  • Duch, B., Gron, S., & Allen, D. (2001). The Power of Problem-Based Learning; A Practical “How To” For Teaching Undergraduate Courses in Any Discipline. Sterling, Virginia: Stylus.

    Google Scholar 

  • Eberts, R. E. (1997). Computer-based instruction. In M. G. Helander, T. K. Landauer, & P. V. Prabhu (Eds.), Handbook of Human–Computer Interaction (pp. 825–847). Amsterdam, The Netherlands: Elsevier.

    Google Scholar 

  • Guskey, T. R. (1987). The essential elements of mastery learning. Journal of Classroom Interaction, 22(2), 19–22.

    Google Scholar 

  • Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579–588.

    Article  Google Scholar 

  • Krajcik, J., & Starr, M. (2001). Learning science content in a project-based environment. In R. Tinker & J. S. Krajcik (Eds.), Portable technologies: Science learning in context. The Netherlands: Kluwer.

    Google Scholar 

  • Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.

    Article  Google Scholar 

  • Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661–667.

    Article  PubMed  Google Scholar 

  • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback intervention on performance: A historical review, a meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 112(2), 254–284.

    Article  Google Scholar 

  • Koedinger, K. R. (2002). Toward evidence for instructional design principles: Examples from Cognitive Tutor Math 6. In D. S. Mewborn, P. Sztajn, D. Y. White, H. G. Wiegel, R. L. Bryant, & K. Nooney (Eds.), Proceedings of twenty-fourth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education Vol. 1 (pp. 21–49). Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education.

    Google Scholar 

  • Koedinger, K. R., & Anderson, J. R. (1993). Reifying implicit planning in geometry: Guidelines for model-based intelligent tutoring system design. In S. Lajoie & S. Derry (Eds.), Computers as cognitive tools. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.

    Google Scholar 

  • Koedinger, K. R., & Corbett, A. T. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences. Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Lepper, M. R., & Malone, T. W. (1987). Intrinsic motivation and instructional effectiveness in computer-based education. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning and instruction: Volume III Conative and affective process analyses. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Mathan, S. A. (2003). Recasting the feedback debate: Benefits of tutoring error detection and correction skills. Unpublished dissertation, Carnegie Mellon University, Pittsburgh, PA.

  • Mathan, S. A., & Koedinger, K. R. (2005) Fostering the Intelligent Novice: Learning from errors with metacognitive tutoring. Educational Psychologist, 40(4), 257–265.

    Article  Google Scholar 

  • McKendree, J. E. (1990). Effective feedback content for tutoring complex skills. Human Computer Interaction, 5, 381–414.

    Article  Google Scholar 

  • McLaren, B. M., Lim, S., Gagnon, F., Yaron, D., & Koedinger, K. R. (2006). Studying the effects of personalized language and worked examples in the context of a web-based intelligent tutor. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 318–328). Berlin: Springer.

    Google Scholar 

  • Merrill, D. C., Reiser, B. J., Ranney, M., & Trafton, J. G. (1992). Effective tutoring techniques: Comparison of human tutors and intelligent tutoring systems. Journal of the Learning Sciences, 2(3), 277–305.

    Article  Google Scholar 

  • Morgan, P., & Ritter, S. (2002). An experimental study of the effects of Cognitive Tutor® Algebra I on student knowledge and attitude (Available from Carnegie Learning, Inc., 1200 Penn Avenue, Suite 150, Pittsburgh, PA 15222). Available at: http://www.carnegielearning.com.

  • Nathan, M. J. (1998). Knowledge and situational feedback in a learning environment for algebra story problem solving. Interactive Learning Environments, 5, 135–159.

    Article  Google Scholar 

  • Ohlsson, S. (1996). Learning from performance errors. Psychological Review, 103(2), 241–262.

    Article  Google Scholar 

  • Ohlsson, S., & Mitrovic, A. (2006). Constraint-based knowledge representation for individualized instruction. Computer Science and Information Systems, 3(1), 1–22.

    Article  Google Scholar 

  • Paas, F., & Van Merrienboer, J. (1994). Variability of worked examples and transfer of geometry problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.

    Article  Google Scholar 

  • Pavlik, Jr., P. I. (2007). Timing is an order: Modeling order effects in the learning of information. In F. E. Ritter, J. Nerb, T. O’Shea & E. Lehtinen (Eds.), In order to learn: How the sequences of topics affect learning, (pp. 137–150). New York, NY: Oxford University Press.

    Google Scholar 

  • Plano, G. S. (2004). The Effects of the Cognitive Tutor Algebra on student attitudes and achievement in a 9th grade Algebra course. Unpublished doctoral dissertation, Seton Hall University, South Orange, NJ.

  • Renkl, A. (2002). Learning from worked-out examples: Instructional explanations supplement self-explanations. Learning & Instruction, 12, 529–556.

    Article  Google Scholar 

  • Renkl, A., Atkinson, R. K., & Große, C. S. (2004) How fading worked solution steps works—A cognitive load perspective. Instructional Science, 32, 59–82.

    Article  Google Scholar 

  • Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23, 90–108.

    Article  PubMed  Google Scholar 

  • Roll, I., Aleven, V., McLaren, B. M., Ryu, E., Baker, R., & Koedinger, K. R. (2006). The help tutor: Does metacognitive feedback improve students’ help-seeking actions, skills and learning? In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 360–369). Berlin: Springer.

    Google Scholar 

  • Sarkis, H. (2004). Cognitive Tutor Algebra 1 program evaluation, Miami-Dade County Public Schools. Lighthouse Point, FL: The Reliability Group.

  • Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207–217.

    Article  Google Scholar 

  • Schoenfeld, A. (1983). Beyond the purely cognitive: Belief systems, social cognitions, and metacognitions as driving forces in intellectual performance. Cognitive Science, 7, 329–363.

    Article  Google Scholar 

  • Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522.

    Article  Google Scholar 

  • Schwonke, R., Wittwer, J., Aleven, V., Salden, R. J. C. M., Krieg, C., & Renkl, A. (2007). Can tutored problem solving benefit from faded worked-out examples? Paper presented at the European Cognitive Science Conference 2007, May 23–;27. Delphi, Greece.

  • Schworm, S., & Renkl, A. (2002). Learning by solved example problems: Instructional explanations reduce self-explanation activity. In W. D. Gray & C. D. Schunn (Eds.), Proceeding of the 24th Annual Conference of the Cognitive Science Society (pp.816–821). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Shneyderman, A. (2001). Evaluation of the Cognitive Tutor Algebra I program, Miami-Dade County Public Schools, Office Of Evaluation And Research. Retrieved from: http://oer.dadeschools.net/algebra.pdf#search=%22Shneyderman%20miami%20cognitive%20tutor%22.

  • Siegler, R. S. (2002). Microgenetic studies of self-explanation. In N. Garnott & J. Parziale (Eds.), Microdevelopment: A process-oriented perspective for studying development and learning (pp. 31–58). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604.

    Article  Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.

    Article  Google Scholar 

  • Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89.

    Article  Google Scholar 

  • Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples and solving problems to skill acquisition. In M. Polson (Ed.), Proceedings of the Fifteenth annual conference of the Cognitive Science Society (pp. 1017–1022). Hillsdale, N.J.: Erlbaum.

    Google Scholar 

  • VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16, 227–265.

    Google Scholar 

  • VanLehn, K., Jones, R. M., & Chi, M. T. H. (1992). A model of the self-explanation effect. The Journal of the Learning Sciences, 2, 1–59.

    Article  Google Scholar 

  • Vanlehn, K., Lynch, C., Schultz, K., Shapiro, J. A., Shelby, R. H., Taylor, L., et al. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147–204.

    Google Scholar 

  • Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and Instruction, 7, 1–39.

    Article  Google Scholar 

  • White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3–118.

    Article  Google Scholar 

  • Zhu, X., & Simon, H. A. (1987). Learning mathematics from examples and by doing. Cognition and Instruction, 4(3), 137–166.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Human-Computer Interaction Institute and Psychology Department, Carnegie Mellon University, CMU, 5000 Forbes Avenue, Pittsburgh, PA, 15213, USA

    Kenneth R. Koedinger

  2. Human-Computer Interaction Institute, Carnegie Mellon University, CMU, 5000 Forbes Avenue, Pittsburgh, PA, 15213, USA

    Kenneth R. Koedinger & Vincent Aleven

Authors
  1. Kenneth R. Koedinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent Aleven
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenneth R. Koedinger.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Koedinger, K.R., Aleven, V. Exploring the Assistance Dilemma in Experiments with Cognitive Tutors. Educ Psychol Rev 19, 239–264 (2007). https://doi.org/10.1007/s10648-007-9049-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10648-007-9049-0

Keywords

Search

Navigation