A Framework System for Intelligent Support in Open Distributed Learning Environments—a Look Back from 16 Years Later

Article

Abstract

The 1998 paper by Martin Mühlenbrock, Frank Tewissen, and myself introduced a multi-agent architecture and a component engineering approach for building open distributed learning environments to support group learning in different types of classroom settings. It took up prior work on “multiple student modeling” as a method to configure and inform group learning situations based on individually assessed learner models. Additionally, methods for detecting collaboration patterns in group action logs were introduced. The approach was exemplified with several applications in the areas of mathematics and general problems solving. The commentary traces a line of development from this work to current mobile and web-based learning architectures and approaches to action logging for interaction analysis. “Lessons learned” are discussed and briefly illustrated with examples from recent work on intelligently enhanced learning environments.

Keywords

Open distributed learning environments Interaction analysis Collaboration patterns 

The Original Paper and its Context

The 1998 paper in question was conceived from an engineering and integration perspective: The article did not focus on one specific innovative idea and approach but combined several different contributions and perspectives with a strong focus a on integrative system engineering. The hope was to extrapolate AIED systems and approaches in such a way as to “permeate” and enrich interactive and cooperative learning environments of various types. The concept was backed by multiple example implementations.

In this integrative perspective, the article addressed the following topics:
  • interaction/cooperation analysis building on “multiple student modelling” and further addressing collaboration patterns,

  • general architectures and synchronization mechanisms for open distributed LEs,

  • integrated face-to-face classroom environments including ubiquitous computing technologies (“computer integrated classrooms” or CiC’s).

The transfer and application of intelligent diagnosis, learner modelling and support mechanisms as core AIED and ITS techniques to CSCL environments (later subsumed under the notion of I-CSCL by Hoppe and Plötzner 1999) with a special focus on collaboration patterns defined the context of Martin Mühlenbrock’s dissertation on “action based collaboration analysis for group learning” (Mühlenbrock 2001). This part of our research built on prior work on “multiple student modelling” (Hoppe 1995), an approach to configuring and informing group learning situations based on individually assessed learner models. Martin Mühlenbrock’s specific contributions were related to defining and detecting collaboration patterns in multi-user activities and to the interfacing of intelligent mechanisms with the “un-intelligent” distributed collaborative environment as part of the overall multi-agent architecture. Frank Tewissen’s part in this paper was particularly related to the distributed computing environment and synchronization mechanisms, prominently applied and exemplified in the ubiquitous computing classroom environment developed in the NIMIS project (Tewissen et al. 2000; Hoppe et al. 2000a, b).

The paper uses the acronym ODLE (Open Distributed Learning Environments) to characterize the intended educational target environments. In this context, “openness” was meant to indicate the opening up of closed loop ITS style scenarios in which single students would interact with one intelligent system in the role of a private tutor. The alternative idea was to rather embed interactive media and intelligent support mechanism into existing and established learning scenarios, especially face-to-face classrooms and other group settings. In other words, the goal was to integrate intelligent learning support with “ecologically valid” contexts in an educational sense. The idea of openness was a trending topic in AIED at that time. Another interpretation of openness was, e.g., the opening up of student models towards inspectability and negotiation (cf. Dimitrova et al. 1999, or Bull and Kay 2007, for a more recent perspective). A common feature for both interpretations of openness is the necessity to give up the idea of full understanding and control of the learning process on the part of the supposedly intelligent system.

Repercussion and Take-Up

In the next few years, the article was mainly referred to and taken up related to issues and challenges of building and providing frameworks for the engineering of intelligently supported or adaptive learning environments.

In her 2001 UMUAI paper on major trends in learner-adapted teaching systems entitled “Learner Control”, Judy Kay refers to our 1998 paper as follows: “A promising approach to constructing sophisticated learning systems is to build generic tools which can support core activities. Some work in this area involves architectures which permit flexible, interchangeable components.”

From an HCI perspective, Fidas et al. (2001) gave the following characterization of our approach: “The framework system proposed by Mühlenbrock et al. is characterized by the combination of intelligent support with interactive learning environments, by the provision of reusable components and by a distributed multi-agent architecture.”

Both citations correctly and adequately put our work in the context of component engineering and multi-agent architectures for learning support systems.

The most specific take-up of ideas from our paper happened in the Intelligent IntraNet Peer Help-Desk (later developed into I-Help) at University of Saskatchewan, Canada. However, the merit of our 1998 paper in this context is relative, since the primary point of reference for this follow-up was the idea of multiple student modeling developed in the context of the earlier COSOFT project (Hoppe 1995). Greer et al. (1998) characterize this work in then following way: “Hoppe’s COSOFT is the first ambitious project to address several issues related to the use of student modelling in order to parameterize human-human collaboration. Later Mühlenbrock et al. (1998) pursued this research further. The questions raised by Hoppe in 1995 include the composition of a learning group from a known set of students, and especially the selection of a peer-helper, the identification of problems to be dealt with in a collaborative session or the selection of tasks that are adequate for a given learning group. Hoppe’s approach has been primarily targeted at exploring possible improvements to group student modelling to support human collaboration.”

Martin Mühlenbrock’s further PhD work was based on the proposed architecture and mechanisms yet focused less on the underlying engineering but on the conceptualization of collaboration patterns and on computational interaction analysis (cf. Mühlenbrock and Hoppe 1999, a paper presented at the CSCL conference). Related to the general topic of technology support for CSCL, a group of young researchers involving Martin Mühlenbrock, Patrick Jerman, Amy Soller, and later also Alejandra Martínez engaged in an exchange of ideas around their own PhD projects. The ensuing comparison and systematic classification of their own and other systems led to an often cited publication under the title “from mirroring to guiding” (Soller et al. 2005).

I found a less evident and somewhat positively surprising reference to our 1998 paper in a UMUAI article by Goodman et al. (2005). This paper deals with the application of dialogue modeling and analysis to collaborative learning environments (CLEs). Here, the type of system and the focus of study is clearly dialogue-centric, whereas our system examples were based on interactive-constructive modeling and co-construction without including dialogues and dialogue interpretation as part of our system environment. In this context of dialogue-centric CLEs, the authors raise the following important question: “The question becomes, can the skills of an ITS be applied in a CLE to optimize individual learning, given that it will now have the added task of facilitating and perhaps teaching small group interaction, and that it will now have an added limitation on its ability to interpret student actions, namely it will have an incomplete understanding of student-student natural language interaction (Lesgold et al. 1992; Mühlenbrock et al. 1998)?” This problem of inherent partial understanding is indeed characteristic of all situations in which the system is not in full control of the task specification and of the assignment of credits to the participating individual learners. Accordingly a precise learner modeling on an individual level is not possible. This is a consequence of openness (as discussed before), and it appears in both the interpretation of dialogues as well as with concrete (modeling) activities.

Practical Impact and Applications

The architectural principles, including the multi-agent architecture and distributed execution and logging of actions, described in our 1998 paper have been used in several European cooperation projects. In these projects that combined new research and development with practical applications in school settings, the architectural and engineering principles elaborated in our original article were actually used and adapted rather than being redefined. The common technical basis for these projects were distributed Java-based applications that would typically run in classroom and school networks.

The earliest EU project of this series was NIMIS (“Networked Interactive Media in Schools”, 1998–2000). The NIMIS environment featured a ubiquitous computing classroom with a large interactive display and pen-based tablets at the student desks. The target was “early learning” in the first years of primary school. There were installations of the same basic environment in Germany, Portugal and UK with different curricular orientations and specific applications. The German environment was particularly used to support the initial acquisition of reading and writing skills based on the “reading through writing” approach (Tewissen et al. 2000). Figure 1 shows scenes from the NIMIS classroom with the reading through writing application and a corresponding action log.
Fig. 1

Reading through writing in the NIMIS classroom

The later projects SEED (2001–2004), COLDEX (2002–2005), and Argunaut (2005–2008) elaborated in different ways on collaborative classroom tools based on a variety of visual languages for modeling, discussion and argumentation, still using very much the same distributed architecture and basic cooperation and logging support. This underlying architecture was extended with a general framework for representing visual languages with heterogeneous semantics and cooperation support (Pinkwart et al. 2002).

Problems and Unfulfilled Expectations

In 1998, there was hope that AIED techniques could be extrapolated in such a way as to encompass fully integrated practical learning environments beyond systems of the ITS type - including “computer-integrated” face-to-face classrooms as well as virtual environments. This has not happened. Instead, more general approaches based on interactive learning environments (supporting knowledge building, inquiry or guided discovery learning) and more recently web-based learning environments have been the most prominent approaches adapted in practical educational settings. Here, the role of intelligent support needs to be conceived in such a way as to hook up the AI-based components on existing architectures not originally conceived as AI systems.

In this sense, today we can state that Gordon McCalla (2000) was largely right when he predicted: “The notion of an AIEd shell that has been popular throughout the last 15 years (…) is increasingly being subsumed into research into communication protocols and ontologies that coordinate the fragmented activities of many AIEd components and learners (e.g., Ritter and Koedinger 1996; Breuker 1997; Mühlenbrock et al. 1998; Wasson, 1998; Vassileva, 1998). By 2010 general protocols will have been developed that guarantee that the humans and various software fragments that have come together to form a learning environment to achieve a particular set of learning goals can compatibly work together to achieve these goals. Research into enhancing these general protocols will be part of the distributed computation research community, not AIEd.” [secondary citations not listed in the references]

Meanwhile we have seen several generations of general technologies for distributed computing and collaboration support as well as mobile and ubiquitous computing frameworks. Many of these have been adapted to and used in educational applications, but as Gordon McCalla predicted these basic tools have been invented elsewhere (from the AIED point of view). In the next section, I will briefly and selectively elaborate on how the heritage of 1998 was still important under these new premises.

Another issue has to do with the further development of our scientific community (or communities): Our 1998 paper was conceived with the intention to bridge over between AIED and CSCL and thus to contribute to a better synergy between both fields. From today’s point of view, we have to state that AIED and CSCL have rather drifted apart. AIED is certainly more open to computational aspects than CSCL, but even nowadays’ AIED conferences do no longer manifest a high interest in aspects of system architectures and system engineering principles.

Re-Conceptualization and Lessons Learned

The continuous evolution of the described architectural and engineering principles for open distributed learning environments with intelligent support has been seriously challenged by new technical ingredients such as mobile and wireless technologies as well as by pure web-based environments. First, the move towards mobile, hand-held devices (cf. Pinkwart et al. 2003) went along with frequent discontinuities in the programming platforms for the client environments, which required repeated revisions and re-implementations of interfaces and protocols. This called for more independence and loose coupling of the system components. In our COLLIDE research group, this has led to a re-conceptualization of our basic architectural principles along the following lines:

Maintaining a basic multi-agent approach we have moved to a blackboard architecture to avoid direct agent-agent communication and interfacing. For this move, we have initially used the IBM T-Spaces implementation (Wyckoff et al. 1998) of Gelernter’s Tuple Space concept (Gelernter 1985). Blackboard architectures and accordingly Tuple Spaces allow for a loose coupling of active components (i.e., agents) that communicate indirectly through notes in the space as a shared memory. To support the usage of different programming languages not only between client and server but also between different clients or agents, we have provided generic clients for a variety of relevant languages such as Java, JavaScript, Prolog, Python, C# and others. To cope with additional functional requirements and to be independent of the T-Spaces implementation we have replaced the tuple space core with our own implementation called SQLSpaces. In Giemza et al. (2007), we have shown how a classical AIED approach to error diagnosis (Hoppe 1994) could be re-implemented on this basis with little extra effort. To achieve this, the original Prolog program was added to the predefined Prolog client to form an agent that communicated with a Java-based front-end through a space.

Meanwhile SQLSpaces have been used as part of a “big” application in the European project SCY (“Science Created by You”, cf. de Jong et al. 2010) supporting collaborative inquiry learning in science. Weinbrenner et al. (2010) report on using this multi-agent architecture based on SQLSpaces for monitoring and detecting (un-) systematic behavior of learners when working with a simulation tool.

Another recurring issue already addressed in our 1998 paper has been action logging as a basis for interaction analysis. In the context of the European Network of Excellence Kaleidoscope, a dedicated working group has specified a standardized format for action logging particularly for collaborative and social analysis (Harrer et al. 2009). However, again more general quasi standards have been established in connection with web-based technologies and social networking platforms. In the on-going EU project Go-Lab this has led us to abandoning the “common format” defined in Kaleidoscope in favor of using Activity Streams. As a result of this recent work, Manske et al. (2014) describe a flexible framework for the authoring of reusable and portable learning analytics gadgets, which is based on SQLSpaces and Activity Streams. Figure 2 illustrates the underlying processing chain in which a general analysis procedure or “workflow” (depicted at the left-hand side) is applied to a set of learner-generated concept maps to form an aggregate or overlay map. This overlay in turn is presented to the learners as a metacognitive scaffold embedded in a specific web-based “learning space” to support group reflection.
Fig. 2

Creation of an embedded learning analytics gadget based on a predefined analysis workflow

At the end of the day, this example shows how the bouquet of themes already present in our 1998 paper has undergone a series of changes and redefinitions, some surface-level, others more conceptual, but the essence of the issues at hand and also the spirit of the solution remains very much related and similar.

References

  1. Breuker, J. (1997). Presentation on ontologies for AIEd systems. Panel discussion. Eighth International Conference on Artificial Intelligence in Education, August 1997. Kobe, Japan.Google Scholar
  2. Bull, S., & Kay, J. (2007). Student models that invite the learner in: The SMILI:() open learner modelling framework. International Journal of Artificial Intelligence in Education, 17(2), 89–120.Google Scholar
  3. de Jong, T., van Joolingen, W. R., Giemza, A., Girault, I., Hoppe, H. U., Kindermann, J., & van der Zanden, M. (2010). Learning by creating and exchanging objects: the SCY experience. British Journal of Educational Technology, 41(6), 909–921.CrossRefGoogle Scholar
  4. Dimitrova, V., Self, J., & Brna, P. (1999). The interactive maintenance of open learner models. In S. P. Lajoie & M. Vivet (Eds.), Artificial Intelligence in Education - Proceedings of AIED’99 (pp. 405–412). Le Mans: Ios Press.Google Scholar
  5. Fidas, C., Komis, V., & Avouris, N. M. (2001). Design of collaboration-support tools for group problem solving. Proceedings of PC HCI (Patras, Greece, December 2001), pp. 263–268.Google Scholar
  6. Gelernter, D. (1985). Generative communication in Linda. ACM Transactions on Programming Languages and Systems, 7(1), 80–112.CrossRefMATHGoogle Scholar
  7. Giemza, A., Weinbrenner, S., Engler, J., & Hoppe, H. U. (2007). Tuple Spaces as a flexible integration platform for distributed learning environments. In T. Hirashima, H. U. Hoppe, & S.-C. Young (Eds.), Supporting learning flow through integrative technologies—proceedings of ICCE 2007 (pp. 313–320). Amsterdam (Netherlands): Ios Press.Google Scholar
  8. Goodman, B. A., Linton, F. N., Gaimari, R. D., Hitzeman, J. M., Ross, H. J., & Zarrella, G. (2005). Using dialogue features to predict trouble during collaborative learning. User Modeling and User-Adapted Interaction, 15(1–2), 85–134.CrossRefGoogle Scholar
  9. Greer, J., McCalla, G., Cooke, J., Collins, J., Kumar, V., Bishop, A., Vassileva, J., et al. (1998). The intelligent helpdesk: Supporting peer-help in a university course. In B. P. Goettl (Ed.), Intelligent tutoring systems - proceedings of ITS 1998 (pp. 494–503). Berlin-Heidelberg: Springer LNCS 1452.Google Scholar
  10. Harrer, A., Martínez-Monés, A., & Dimitracopoulou, A. (2009). Users’ data: Collaborative and social analysis. In N. Balacheff, S. Ludvigsen, T. de Jong, A. Lazonder, & S. Barnes (Eds.), Technology-enhanced learning. Principles and products (pp. 175–193). Netherlands: Springer.CrossRefGoogle Scholar
  11. Hoppe, H. U. (1994). Deductive error diagnosis and inductive error generalization for Intelligent Tutoring Systems. Journal of Artificial Intelligence in Education, 5(1), 27–49.Google Scholar
  12. Hoppe, H.U. (1995). Use of multiple student modeling to parametrize group learning. In J. Greer (ed.), Artificial intelligence in education - proceedings of AIED ‘95 (234–249). August 1995, Washington, DC (USA).Google Scholar
  13. Hoppe, H. U., & Plötzner, R. (1999). Can analytic models support learning in groups? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp. 147–168). Amsterdam: Elsevier.Google Scholar
  14. Hoppe, H. U., Lingnau, A., Machado, I., Paiva, A., Prada, R., & Tewissen, F. (2000a). Supporting collaborative activities in computer integrated classrooms - the NIMIS approach. In Proceedings of the Sixth International Workshop on Groupware - CRIWG 2000 (pp. 94–101). Los Alamos: IEEE Press.CrossRefGoogle Scholar
  15. Hoppe, H. U., Gaßner, K., & Tewissen, F. (2000b). Distributed visual language environments for cooperation and learning: Applications and intelligent support. Group Decision and Negotiation, 9(3), 205–220.CrossRefGoogle Scholar
  16. Lesgold, A., Katz, S., Greenberg, L., Hughes, E., & Eggan, G. (1992). Extensions of intelligent tutoring paradigms to support collaborative learning. In S. Dijkstra, H. Krammer, & J. van Merrienboer (Eds.), Instructional Models in Computer-Based Learning Environments (pp. 291–311). Berlin: Springer.Google Scholar
  17. Manske, S., Hecking, T., Bollen, L., Göhnert, T., Ramos, A., & Hoppe, H.U. (2014). A flexible framework for the authoring of reusable and portable learning analytics gadgets. In Proceedings of the 14th IEEE International Conference on Advanced Learning Technologies (254–258). Athens (Greece).Google Scholar
  18. McCalla, G. (2000). The fragmentation of culture, learning, teaching, and technology: implications for the artificial intelligence in education research agenda in 2010. International Journal of Artificial Intelligence in Education, 11, 177–196.Google Scholar
  19. Mühlenbrock, M. (2001). Action Based Collaboration Analysis for Group Learning. Amsterdam: Ios Press.MATHGoogle Scholar
  20. Mühlenbrock, M., & Hoppe, H. U. (1999). Computer supported interaction analysis of group problem solving. In C. Hoadley & J. Roschelle (Eds.), Proceedings of the Conference on Computer Supported Collaborative Learning - CSCL ‘99 (pp. 398–405). Palo Alto: Erbaum.Google Scholar
  21. Mühlenbrock, M., Tewissen, F., & Hoppe, H. U. (1998). A framework system for intelligent support in open distributed learning environments. International Journal of Artificial Intelligence in Education, 9, 256–274.Google Scholar
  22. Pinkwart, N., Hoppe, H. U., Bollen, L., & Fuhlrott, E. (2002). Group-oriented modelling tools with heterogeneous semantics. In S. A. Cerri, G. Gouardéres, & F. Paraguacu (Eds.), Intelligent Tutoring Systems - Proceedings of ITS 2002 (pp. 21–30). Berlin-Heidelberg: Springer LNCS 2363.Google Scholar
  23. Pinkwart, N., Hoppe, H. U., Milrad, M., & Perez, J. (2003). Educational scenarios for cooperative use of Personal Digital Assistants. Journal of Computer Assisted Learning, 19(3), 383–391.CrossRefGoogle Scholar
  24. Ritter, S., & Koedinger, K. R. (1996). An architecture for plug-in tutor agents. International Journal of Artificial Intelligence in Education, 7(3/4), 315–347.Google Scholar
  25. Soller, A., Martínez, A., Jermann, P., & Mühlenbrock, M. (2005). From mirroring to guiding: a review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261–290.Google Scholar
  26. Tewissen, F., Lingnau, A., & Hoppe, H. U. (2000). “Today’s Talking Typewriter” - supporting early literacy in a classroom environment. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent Tutoring Systems - Proceedings of ITS 2000 (pp. 252–261). Berlin-Heidelberg: Springer LNCS 1839.Google Scholar
  27. Vassileva, J. (1998). Goal-based autonomous social agents supporting adaptation and teaching in a distributed environment. Proceedings of the Third International Conference on Intelligent Tutoring Systems, San Antonio, Texas. pp. 564–573.Google Scholar
  28. Wasson, B. (1998). Identifying coordination agents for collaborative telelearning. International Journal of Artificial Intelligence in Education, 9, 275–299.Google Scholar
  29. Weinbrenner, S., Engler, J., Wichmann, A., Hoppe, U., et al. (2010). Monitoring and analysing students’ systematic behaviour - The SCY pedagogical agent framework. In M. Wolpers (Ed.), Sustaining TEL: From Innovation to Learning and Practice - Proceedings of EC-TEL 2010 (pp. 602–607). Barcelona: Springer LNCS.CrossRefGoogle Scholar
  30. Wyckoff, P., McLaughry, S. W., Lehman, T. J., & Ford, D. A. (1998). T Spaces. IBM Systems Journal, 37(3), 454–474.CrossRefGoogle Scholar

Copyright information

© International Artificial Intelligence in Education Society 2015

Authors and Affiliations

  1. 1.COLLIDE Research Group - Department of Computer Science and Applied Cognitive ScienceUniversity of Duisburg-EssenEssenGermany

Personalised recommendations