The Original Paper and its Context

The 1998 paper in question was conceived from an engineering and integration perspective: The article did not focus on one specific innovative idea and approach but combined several different contributions and perspectives with a strong focus a on integrative system engineering. The hope was to extrapolate AIED systems and approaches in such a way as to “permeate” and enrich interactive and cooperative learning environments of various types. The concept was backed by multiple example implementations.

In this integrative perspective, the article addressed the following topics:

  • interaction/cooperation analysis building on “multiple student modelling” and further addressing collaboration patterns,

  • general architectures and synchronization mechanisms for open distributed LEs,

  • integrated face-to-face classroom environments including ubiquitous computing technologies (“computer integrated classrooms” or CiC’s).

The transfer and application of intelligent diagnosis, learner modelling and support mechanisms as core AIED and ITS techniques to CSCL environments (later subsumed under the notion of I-CSCL by Hoppe and Plötzner 1999) with a special focus on collaboration patterns defined the context of Martin Mühlenbrock’s dissertation on “action based collaboration analysis for group learning” (Mühlenbrock 2001). This part of our research built on prior work on “multiple student modelling” (Hoppe 1995), an approach to configuring and informing group learning situations based on individually assessed learner models. Martin Mühlenbrock’s specific contributions were related to defining and detecting collaboration patterns in multi-user activities and to the interfacing of intelligent mechanisms with the “un-intelligent” distributed collaborative environment as part of the overall multi-agent architecture. Frank Tewissen’s part in this paper was particularly related to the distributed computing environment and synchronization mechanisms, prominently applied and exemplified in the ubiquitous computing classroom environment developed in the NIMIS project (Tewissen et al. 2000; Hoppe et al. 2000a, b).

The paper uses the acronym ODLE (Open Distributed Learning Environments) to characterize the intended educational target environments. In this context, “openness” was meant to indicate the opening up of closed loop ITS style scenarios in which single students would interact with one intelligent system in the role of a private tutor. The alternative idea was to rather embed interactive media and intelligent support mechanism into existing and established learning scenarios, especially face-to-face classrooms and other group settings. In other words, the goal was to integrate intelligent learning support with “ecologically valid” contexts in an educational sense. The idea of openness was a trending topic in AIED at that time. Another interpretation of openness was, e.g., the opening up of student models towards inspectability and negotiation (cf. Dimitrova et al. 1999, or Bull and Kay 2007, for a more recent perspective). A common feature for both interpretations of openness is the necessity to give up the idea of full understanding and control of the learning process on the part of the supposedly intelligent system.

Repercussion and Take-Up

In the next few years, the article was mainly referred to and taken up related to issues and challenges of building and providing frameworks for the engineering of intelligently supported or adaptive learning environments.

In her 2001 UMUAI paper on major trends in learner-adapted teaching systems entitled “Learner Control”, Judy Kay refers to our 1998 paper as follows: “A promising approach to constructing sophisticated learning systems is to build generic tools which can support core activities. Some work in this area involves architectures which permit flexible, interchangeable components.”

From an HCI perspective, Fidas et al. (2001) gave the following characterization of our approach: “The framework system proposed by Mühlenbrock et al. is characterized by the combination of intelligent support with interactive learning environments, by the provision of reusable components and by a distributed multi-agent architecture.”

Both citations correctly and adequately put our work in the context of component engineering and multi-agent architectures for learning support systems.

The most specific take-up of ideas from our paper happened in the Intelligent IntraNet Peer Help-Desk (later developed into I-Help) at University of Saskatchewan, Canada. However, the merit of our 1998 paper in this context is relative, since the primary point of reference for this follow-up was the idea of multiple student modeling developed in the context of the earlier COSOFT project (Hoppe 1995). Greer et al. (1998) characterize this work in then following way: “Hoppe’s COSOFT is the first ambitious project to address several issues related to the use of student modelling in order to parameterize human-human collaboration. Later Mühlenbrock et al. (1998) pursued this research further. The questions raised by Hoppe in 1995 include the composition of a learning group from a known set of students, and especially the selection of a peer-helper, the identification of problems to be dealt with in a collaborative session or the selection of tasks that are adequate for a given learning group. Hoppe’s approach has been primarily targeted at exploring possible improvements to group student modelling to support human collaboration.”

Martin Mühlenbrock’s further PhD work was based on the proposed architecture and mechanisms yet focused less on the underlying engineering but on the conceptualization of collaboration patterns and on computational interaction analysis (cf. Mühlenbrock and Hoppe 1999, a paper presented at the CSCL conference). Related to the general topic of technology support for CSCL, a group of young researchers involving Martin Mühlenbrock, Patrick Jerman, Amy Soller, and later also Alejandra Martínez engaged in an exchange of ideas around their own PhD projects. The ensuing comparison and systematic classification of their own and other systems led to an often cited publication under the title “from mirroring to guiding” (Soller et al. 2005).

I found a less evident and somewhat positively surprising reference to our 1998 paper in a UMUAI article by Goodman et al. (2005). This paper deals with the application of dialogue modeling and analysis to collaborative learning environments (CLEs). Here, the type of system and the focus of study is clearly dialogue-centric, whereas our system examples were based on interactive-constructive modeling and co-construction without including dialogues and dialogue interpretation as part of our system environment. In this context of dialogue-centric CLEs, the authors raise the following important question: “The question becomes, can the skills of an ITS be applied in a CLE to optimize individual learning, given that it will now have the added task of facilitating and perhaps teaching small group interaction, and that it will now have an added limitation on its ability to interpret student actions, namely it will have an incomplete understanding of student-student natural language interaction (Lesgold et al. 1992; Mühlenbrock et al. 1998)?” This problem of inherent partial understanding is indeed characteristic of all situations in which the system is not in full control of the task specification and of the assignment of credits to the participating individual learners. Accordingly a precise learner modeling on an individual level is not possible. This is a consequence of openness (as discussed before), and it appears in both the interpretation of dialogues as well as with concrete (modeling) activities.

Practical Impact and Applications

The architectural principles, including the multi-agent architecture and distributed execution and logging of actions, described in our 1998 paper have been used in several European cooperation projects. In these projects that combined new research and development with practical applications in school settings, the architectural and engineering principles elaborated in our original article were actually used and adapted rather than being redefined. The common technical basis for these projects were distributed Java-based applications that would typically run in classroom and school networks.

The earliest EU project of this series was NIMIS (“Networked Interactive Media in Schools”, 1998–2000). The NIMIS environment featured a ubiquitous computing classroom with a large interactive display and pen-based tablets at the student desks. The target was “early learning” in the first years of primary school. There were installations of the same basic environment in Germany, Portugal and UK with different curricular orientations and specific applications. The German environment was particularly used to support the initial acquisition of reading and writing skills based on the “reading through writing” approach (Tewissen et al. 2000). Figure 1 shows scenes from the NIMIS classroom with the reading through writing application and a corresponding action log.

Fig. 1
figure 1

Reading through writing in the NIMIS classroom

Full size image

The later projects SEED (2001–2004), COLDEX (2002–2005), and Argunaut (2005–2008) elaborated in different ways on collaborative classroom tools based on a variety of visual languages for modeling, discussion and argumentation, still using very much the same distributed architecture and basic cooperation and logging support. This underlying architecture was extended with a general framework for representing visual languages with heterogeneous semantics and cooperation support (Pinkwart et al. 2002).

Problems and Unfulfilled Expectations

In 1998, there was hope that AIED techniques could be extrapolated in such a way as to encompass fully integrated practical learning environments beyond systems of the ITS type - including “computer-integrated” face-to-face classrooms as well as virtual environments. This has not happened. Instead, more general approaches based on interactive learning environments (supporting knowledge building, inquiry or guided discovery learning) and more recently web-based learning environments have been the most prominent approaches adapted in practical educational settings. Here, the role of intelligent support needs to be conceived in such a way as to hook up the AI-based components on existing architectures not originally conceived as AI systems.

In this sense, today we can state that Gordon McCalla (2000) was largely right when he predicted: “The notion of an AIEd shell that has been popular throughout the last 15 years (…) is increasingly being subsumed into research into communication protocols and ontologies that coordinate the fragmented activities of many AIEd components and learners (e.g., Ritter and Koedinger 1996; Breuker 1997; Mühlenbrock et al. 1998; Wasson, 1998; Vassileva, 1998). By 2010 general protocols will have been developed that guarantee that the humans and various software fragments that have come together to form a learning environment to achieve a particular set of learning goals can compatibly work together to achieve these goals. Research into enhancing these general protocols will be part of the distributed computation research community, not AIEd.” [secondary citations not listed in the references]

Meanwhile we have seen several generations of general technologies for distributed computing and collaboration support as well as mobile and ubiquitous computing frameworks. Many of these have been adapted to and used in educational applications, but as Gordon McCalla predicted these basic tools have been invented elsewhere (from the AIED point of view). In the next section, I will briefly and selectively elaborate on how the heritage of 1998 was still important under these new premises.

Another issue has to do with the further development of our scientific community (or communities): Our 1998 paper was conceived with the intention to bridge over between AIED and CSCL and thus to contribute to a better synergy between both fields. From today’s point of view, we have to state that AIED and CSCL have rather drifted apart. AIED is certainly more open to computational aspects than CSCL, but even nowadays’ AIED conferences do no longer manifest a high interest in aspects of system architectures and system engineering principles.

Re-Conceptualization and Lessons Learned

The continuous evolution of the described architectural and engineering principles for open distributed learning environments with intelligent support has been seriously challenged by new technical ingredients such as mobile and wireless technologies as well as by pure web-based environments. First, the move towards mobile, hand-held devices (cf. Pinkwart et al. 2003) went along with frequent discontinuities in the programming platforms for the client environments, which required repeated revisions and re-implementations of interfaces and protocols. This called for more independence and loose coupling of the system components. In our COLLIDE research group, this has led to a re-conceptualization of our basic architectural principles along the following lines:

Maintaining a basic multi-agent approach we have moved to a blackboard architecture to avoid direct agent-agent communication and interfacing. For this move, we have initially used the IBM T-Spaces implementation (Wyckoff et al. 1998) of Gelernter’s Tuple Space concept (Gelernter 1985). Blackboard architectures and accordingly Tuple Spaces allow for a loose coupling of active components (i.e., agents) that communicate indirectly through notes in the space as a shared memory. To support the usage of different programming languages not only between client and server but also between different clients or agents, we have provided generic clients for a variety of relevant languages such as Java, JavaScript, Prolog, Python, C# and others. To cope with additional functional requirements and to be independent of the T-Spaces implementation we have replaced the tuple space core with our own implementation called SQLSpaces. In Giemza et al. (2007), we have shown how a classical AIED approach to error diagnosis (Hoppe 1994) could be re-implemented on this basis with little extra effort. To achieve this, the original Prolog program was added to the predefined Prolog client to form an agent that communicated with a Java-based front-end through a space.

Meanwhile SQLSpaces have been used as part of a “big” application in the European project SCY (“Science Created by You”, cf. de Jong et al. 2010) supporting collaborative inquiry learning in science. Weinbrenner et al. (2010) report on using this multi-agent architecture based on SQLSpaces for monitoring and detecting (un-) systematic behavior of learners when working with a simulation tool.

Another recurring issue already addressed in our 1998 paper has been action logging as a basis for interaction analysis. In the context of the European Network of Excellence Kaleidoscope, a dedicated working group has specified a standardized format for action logging particularly for collaborative and social analysis (Harrer et al. 2009). However, again more general quasi standards have been established in connection with web-based technologies and social networking platforms. In the on-going EU project Go-Lab this has led us to abandoning the “common format” defined in Kaleidoscope in favor of using Activity Streams. As a result of this recent work, Manske et al. (2014) describe a flexible framework for the authoring of reusable and portable learning analytics gadgets, which is based on SQLSpaces and Activity Streams. Figure 2 illustrates the underlying processing chain in which a general analysis procedure or “workflow” (depicted at the left-hand side) is applied to a set of learner-generated concept maps to form an aggregate or overlay map. This overlay in turn is presented to the learners as a metacognitive scaffold embedded in a specific web-based “learning space” to support group reflection.

Fig. 2
figure 2

Creation of an embedded learning analytics gadget based on a predefined analysis workflow

Full size image

At the end of the day, this example shows how the bouquet of themes already present in our 1998 paper has undergone a series of changes and redefinitions, some surface-level, others more conceptual, but the essence of the issues at hand and also the spirit of the solution remains very much related and similar.