Regulative Loops, Step Loops and Task Loops
This commentary suggests a generalization of the conception of the behavior of tutoring systems, which the target article characterized as having an outer loop that was executed once per task and an inner loop that was executed once per step of the task. A more general conception sees these two loops as instances of regulative loops, which repeatedly compare students’ performances to a gold standard and give advice on how to bring that performance closer to the standard.
KeywordsIntelligent tutoring systems Computer supported collaborative learning Coaching Assessment Self-regulation Mirroring
The subject of this commentary, The Behavior of Tutoring Systems (VanLehn 2006), was originally intended to be the first chapter of a textbook on Intelligent Tutoring Systems, but the book was never written. Perhaps it was just as well, because such a book and its introduction should be based on a concept eloquently articulated by Soller et al. (2005). The Soller et al. article surveys work in computer-support collaborative learning (CSCL). It turns out that everything the article has to say about CSCL holds for intelligent tutoring systems (ITS) as well, and everything that the Behavior of Tutoring Systems says about ITS also holds for the systems reviewed by Soller et al. This commentary presents the outline of the introductory chapter of a textbook, as yet unwritten, that treats both types of systems together.
Soller et al. point out that work in CSCL can be divided into two basic types: structural and regulative. Structural work explores novel tasks or learning activities whose structure is intended to elicit improved learning from students. Regulative work explores the augmentation of existing tasks with regulatory mechanisms that aim to improve students’ learning by interacting with them as their activity occurs. Following Soller et al., let us focus only on regulative approaches.
If the students execute the whole regulative loop themselves, the activity is called self-regulation.
If the system monitors the students’ performance and presents it back to the students, then the activity is called mirroring.
If the system both monitors the students’ performance, compares it to a gold standard and reports the comparison to the students, then the activity is called formative assessment.
If the system monitors the performance, compares it to a gold standard, and generates some advice to the students or modifies the task or context, then the activity is called coaching.
Although Soller et al. intended this classification to apply only to collaborative learning activities, it applies equally well to learning activities that involve just one student. For instance, the coaching activity is just what an intelligent tutoring system typically does. Mirroring is often a good characterization of both reflective debriefing (Katz et al. 1998; Pon-Barry et al. 2006; Stottler et al. 2001) and open learner modeling (Bull et al. 2007), where the system presents a display of the student’s performance, often as a timeline. If the display also shows correctness, for instance by coloring red the incorrect episodes on the timeline, then the overall activity matches the assessment activity of Fig. 1.
Four dimensions for classifying regulative loop systems, in addition to the main classification dimension that is shown in Fig. 1
• Type of knowledge: Cognitive, meta-cognitive, collaborative skill, motivations
• Recipient: Student, teacher, activity structure (interventions only)
• Type of change process: Reminding, persuasion, teaching, remediation
• Granularity: Step, task, module
The second dimension of the classification concerns the gray arrows of Fig. 1. They show that information from a mirror, assessment or advice can affect the students’ knowledge. This suggests that the information is viewed by the students directly, and that is certainly the case with a traditional ITS, assessment or mirroring system. However, the information could be given to a teacher, who then may pass it on to the students, or enact one of the next phases in the regulative loop. For instance, MTDashboard monitors the performance of students and small groups, and presents the information to the teacher on a handheld dashboard (Martinez-Maldonado et al. 2014). The teacher may choose which groups to visit and help; or choose to move the whole class onto a different activity. In the case of interventions (but not mirrors or formative assessments), the system can restructure the students’ activity without giving advice to either the teacher or the student. For instance, when the students have completed a task, some ITS can assign a next task that addresses a weakness in the students’ knowledge. As an example from CSCL, Tchounikine et al. (2010) propose that the macro-script that structures a group’s activity be changed dynamically on the basis of an analysis of the group’s interactions.
Reminding: The student has merely forgotten to apply the piece of knowledge, so its retrieval cues need to be modified. The pointing hints described in the Behavior of Tutoring Systems are examples of advice intended to cause this kind of change. Some mirroring could also be aimed at reminding.
Persuasion: A persuasive hint, such as “you would probably finish your homework faster if you followed the strategy I taught you,” assumes that the students already possess a piece of knowledge and merely need to be convinced to use it.
Teaching: This kind of advice assumes that the students do not know a piece of knowledge, but know all the ingredient or pre-requisite concepts, so the knowledge can just be stated in a simple form.
Remediation: Sometimes a simple statement of knowledge won’t suffice because the students lack pre-requisite knowledge. For instance, the hints of Why2-Atlas, a physics tutoring system (VanLehn et al. 2007), assumed that the students already knew about vectors. When it was used to teach students who did not know vectors, it was less effective than human tutors, who were able to adapt their advice and teach the required remedial concepts.
A step is the smallest user interface action for which it makes sense to call it correct or incorrect. In physics, typing in an equation is a step, but entering the first character of such an equation is too small to be a step.
A task is the smallest unit of user activity that is logically and informationally independent of the tasks/units that precede and follow it. For instance, a complete solution of a physics problem is a task, but entering a single equation is not because it depends on variable definitions and other equations.
A module, chapter or unit is a larger grain size, roughly defined to be learning activities that all address the same topic.
The Behavior of Tutoring Systems is known in part for distinguishing the inner loop from the outer loop. The inner loop is just an instance of the regulative loop of Fig. 1 with step as the grain size. The outer loop is another instance of the regulative loop, but at the grain size of a task.
By viewing the inner and outer loops as regulative loops, new design options open up. For instance, the Behavior of Tutoring Systems considered only a coaching version of the outer loop. That is, the computer would either choose the next task for the student or display its recommendations and let the student choose a task to do next. An alternative would be a task loop that merely assessed the students’ choice of task by displaying the task choices that the student made and an evaluation of each choice relative to some gold standard for task selection.
By seeing regulative CSCL systems as similar to ITS in that both implement one or more regulative loops, some of the insights from ITS development recorded in Behavior of Tutoring Systems can be applied to CSCL systems. For instance, the whole discussion of when to give hints along with the related concepts of gaming the system, help abuse and help refusal, can be easily applied to CSCL systems that give advice to students on how to collaborate. The many options for reviewing solutions, also listed in Behavior of Tutoring Systems, could also be considered with designing a CSCL system that does assessment or mirroring of collaboration.
For future reference, instructional systems that conform to Fig. 1 could be called “regulative loop systems.” This class of systems probably includes most ITS but only some CSCL systems, namely those that take a regulative approach instead of a structural one. For example, a CSCL system that asks students to follow scripts would be structural, and yet it would become a regulative loop system if it also determined whether the students actually followed the scripts, and either mirrored, assessed or intervened with the students or their teachers.
In short, although the fields of CSCL and ITS have often had little contact over the years, they share many of the same problems and solutions. Perhaps the framework sketched here will be useful in transplanting ideas from one field to the other.
The research reported here was supported by the National Science Foundation under grants IIS-1123823 and DUE-1140901, by the Office of Naval Research under grant N00014-13-C-0029, by the US Army under contract W911NF-04-D-0004, and by the Bill and Melinda Gates Foundation under grant OPP1061281.
- Bull, S., Dimitrova, V., & McCalla, G. (2007). Open learner models: research questions (preface to special issue of ijaied, part 1). International Journal of Artificial Intelligence in Education, 17(2), 89–120.Google Scholar
- Conati, C., & VanLehn, K. (2000). Further results from the evaluation of an intelligent computer tutor to coach self-explanation. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent tutoring systems: 5th international conference, its 2000 (pp. 304–313). Berlin: Springer.CrossRefGoogle Scholar
- Du Boulay, B., Avramides, K., Luckin, R., Martinez-Miron, E., & Rebolledo-Mendez, G. (2010). Towards systems that care: a conceptual framework based on motivation, metacognition and affect. International Journal of Artificial Intelligence in Education.Google Scholar
- Katz, S., Lesgold, A., Hughes, E., Peters, D., Eggan, G., Gordin, M., & Greenberg, L. (1998). Sherlock 2: An intelligent tutoring system built on the lrdc framework. In C. P. Bloom & R. B. Loftin (Eds.), Facilitating the development and use of interactive learning environments (pp. 227–258). Hillsdale: Erlbaum.Google Scholar
- Lepper, M. R., Woolverton, M., Mumme, D. L., & Gurtner, J.-L. (1993). Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 75–105). Hillsdale: Lawrence Erlbaum Associates.Google Scholar
- Martinez-Maldonado, R., Clayphan, A., Yacef, K., & Kay, J. (2014). Towards providing notifications to enhance teacher’s awareness in the classroom. In S. Trausan-Matu (Ed.), Intelligent tutoring sysems its 2014 (pp. 510–515). Switzerland: Springer.Google Scholar
- Pon-Barry, H., Schultz, K., Bratt, E. O., Clark, B., & Peters, S. (2006). Responding to student uncertainty in spoken tutorial dialogue systems. International Journal of Artificial Intelligence in Education, 16, 171–194.Google Scholar
- Roll, I., Aleven, V., McLaren, B., & Koedinger, K.R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 267–280.Google Scholar
- Soller, A., Martinez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: a review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15, 261–290.Google Scholar
- Stottler, D., Fu, D., Ramachandran, S., & Jackson, T. (2001). Applying a generic intelligent tutoring system authoring tool to specific military domains tao its, a case study Paper presented at the ITSEC.Google Scholar
- VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16, 227–265.Google Scholar