Assessing Learning in MOOCs Through Interactions Between Learners

  • Francis BrounsEmail author
  • Olga Firssova
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1014)


This paper presents a retrospective analysis of learning in a MOOC as reconstructed from the conversations that learners conducted in MOOC group forums while performing the course tasks. A mixed method approach was applied to analyze the quantity and the quality of these conversations. Two activity patterns were distinguished – in groups with higher activity levels, there were more individual contributions (posts) on more course themes and these contributions were broader spread throughout the course. In high activity groups there was also more interaction between participants, i.e., more questions, answers, explanations and elaborations. The presented study demonstrates how modeling interactions in group forums helps to elicit individual and emerging group knowledge construction and thus supports defining MOOC learning, informs MOOC design and provides insights on how assessing MOOC learning can be automated.


Assessing learning Learner interactions Knowledge building Text analysis MOOC learning Mixed methods 


  1. 1.
    Henderikx, M., Kreijns, K., Kalz, M.: To change or not to change? That’s the question… On MOOC-success, barriers and their implications. In: Delgado Kloos, C., Jermann, P., Pérez-Sanagustín, M., Seaton, Daniel T., White, S. (eds.) EMOOCs 2017. LNCS, vol. 10254, pp. 210–216. Springer, Cham (2017). Scholar
  2. 2.
    Henderikx, M.A., Kreijns, K., Kalz, M.: Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Educ. 38, 353–368 (2017). Scholar
  3. 3.
    Dillenbourg, P.: What do you mean by collaborative learning? In: Dillenbourg, P. (ed.) Collaborative-Learning: Cognitive and Computational Approaches, pp. 1–19. Elsevier, Oxford (1999)Google Scholar
  4. 4.
    Stahl, G.: Collaborative information environments to support knowledge construction by communities. AI Soc. 14, 71–97 (2000). Scholar
  5. 5.
    Stahl, G.: Group Cognition: Computer Support for Building Collaborative Knowledge. MIT Press, Cambridge (2006)CrossRefGoogle Scholar
  6. 6.
    Nijland, F.J.: Mirroring interaction: an exploratory study into student interaction in independent working. Tilburg University, The Netherlands (2011)Google Scholar
  7. 7.
    Laurillard, D.: Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. Routledge, London (2002)CrossRefGoogle Scholar
  8. 8.
    Veldhuis-Diermanse, A.E., Biemans, H.J.A., Mulder, M., Mahdizadeh, H.: Analysing learning processes and quality of knowledge construction in networked learning. J. Agric. Educ. Ext. 12, 41–57 (2006). Scholar
  9. 9.
    Schrire, S.: Knowledge building in asynchronous discussion groups: going beyond quantitative analysis. Comput. Educ. 46, 49–70 (2006). Scholar
  10. 10.
    Strijbos, J.-W., Martens, R.L., Prins, F.J., Jochems, W.M.G.: Content analysis: what are they talking about? Comput. Educ. 46, 29–48 (2006). Scholar
  11. 11.
    Weinberger, A., Fischer, F.: A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Comput. Educ. 46, 71–95 (2006). Scholar
  12. 12.
    De Wever, B., Schellens, T., Valcke, M., Van Keer, H.: Content analysis schemes to analyze transcripts of online asynchronous discussion groups: a review. Comput. Educ. 46, 6–28 (2006). Scholar
  13. 13.
    Kalz, M., Specht, M.: If MOOCS are the answer, did we ask the right questions? Implications for the design of large-scale online-courses. Maastricht School of Management (2013).
  14. 14.
    Berlanga, A.J., Kalz, M., Stoyanov, S., Van Rosmalen, P., Smithies, A., Braidman, I.: Using language technologies to diagnose learner’s conceptual development. In: Proceedings of the 9th IEEE International Conference on Advanced Learning Technologies (ICALT 2009), pp. 669–673. IEEE (2009)Google Scholar
  15. 15.
    Rubens, W., Kalz, M., Koper, R.: Improving the learning design of massive open online courses. Turk Online J. Educ. Technol. 13, 71–80 (2014)Google Scholar
  16. 16.
    Firssova, O., Brouns, F., Kalz, M.: Designing for open learning: design principles and scalability affordances in practice. In: L@S: Third Annual ACM Conference on Learning at Scale. ACM (2016).
  17. 17.
    Paavola, S., Hakkarainen, K.: The knowledge creation metaphor – an emergent epistemological approach to learning. Sci. Educ. 14, 535–557 (2005). Scholar
  18. 18.
    Scardamalia, M., Bereiter, C.: A brief history of knowledge building. Can. J. Learn. Technol. 36(1) (2010).
  19. 19.
    Hewitt, J., Scardamalia, M.: Design principles for distributed knowledge building processes. Educ. Psychol. Rev. 10, 75–96 (1998). Scholar
  20. 20.
    Cacciamani, S., Perrucci, V., Khanlari, A.: Conversational functions for knowledge building communities: a coding scheme for online interactions. Educ. Technol. Res. Dev. 66(6), 1529–1546 (2018). Scholar
  21. 21.
    Gunawardena, C.N., Lowe, C.A., Anderson, T.: Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. J. Educ. Comput. Res. 17, 397–431 (1997). Scholar
  22. 22.
    Laurillard, D.: The pedagogical challenges to collaborative technologies. J. Comput.-Support. Collab. Learn. 4(1), 5–20 (2009). Scholar
  23. 23.
    Garrison, D.R., Anderson, T., Archer, W.: Critical thinking, cognitive presence, and computer conferencing in distance education. Am. J. Dist. Educ. 15, 7–23 (2001). Scholar
  24. 24.
    Joksimović, S., Gašević, D., Kovanović, V., Riecke, B.E., Hatala, M.: Social presence in online discussions as a process predictor of academic performance. J. Comput. Assist. Learn. 31, 638–654 (2015). Scholar
  25. 25.
    Pena-Shaff, J.B., Nicholls, C.: Analyzing student interactions and meaning construction in computer bulletin board discussions. Comput. Educ. 42, 243–265 (2004). Scholar
  26. 26.
    Geisler, C.: Coding for language complexity: the interplay among methodological commitments, tools, and workflow in writing research. Writ. Commun. 35, 215–249 (2018). Scholar
  27. 27.
    Geisler, C.: Analyzing Streams of Language: Twelve Steps to the Systematic Coding of Text, Talk, and Other Verbal Data. Pearson Longman, New York (2004)Google Scholar
  28. 28.
    Geisler, C.: Current and emerging methods in the rhetorical analysis of texts - introduction: toward an integrated approach. J. Writ. Res. 7, 417–424 (2016). Scholar
  29. 29.
    Harry, B., Sturges, K.M., Klingner, J.K.: Mapping the process: an exemplar of process and challenge in grounded theory analysis. Educ. Res. 34, 3–13 (2005)CrossRefGoogle Scholar
  30. 30.
    Rourke, L., Anderson, T.: Validity in quantitative content analysis. Educ. Technol. Res. Dev. 52, 5–18 (2004). Scholar
  31. 31.
    Alario-Hoyos, C., Perez-Sanagustin, M., Delgado-Kloos, C., Parada G, H.A., Munoz-Organero, M.: Delving into participants’ profiles and use of social tools in MOOCs. IEEE Trans. Learn. Technol. 7, 260–266 (2014). Scholar
  32. 32.
    Huang, J., Dasgupta, A., Ghosh, A., Manning, J., Sanders, M.: Superposter behavior in MOOC forums. In: Proceedings of the First ACM Conference on Learning @ Scale Conference, pp. 117–126. ACM, Atlanta (2014).
  33. 33.
    Kester, L., Sloep, P., Brouns, F., Van Rosmalen, P., De Vries, F., De Croock, M.: Enhancing social interaction and spreading tutor responsibilities in bottom-up organized learning networks. In: IADIS International Conference Web Based Communities 2006, p. 472. IADIS (2006)Google Scholar
  34. 34.
    Sloep, P., Kester, L.: From lurker to active participant. In: Koper, R. (ed.) Learning Network Services for Professional Development, pp. 17–25. Springer, Berlin (2009). Scholar
  35. 35.
    Rosé, C., et al.: Analyzing collaborative learning processes automatically: exploiting the advances of computational linguistics in computer-supported collaborative learning. Int. J. Comput.-Support. Collab. Learn. 3, 237–271 (2008). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Welten InstituteOpen University of the NetherlandsHeerlenThe Netherlands

Personalised recommendations