Technology, Knowledge and Learning

, Volume 24, Issue 4, pp 545–565 | Cite as

Promoting Learners’ Voice Productions Using Chatbots as a Tool for Improving the Learning Process in a MOOC

  • Juanan Pereira
  • María Fernández-Raga
  • Sara Osuna-AcedoEmail author
  • Margarita Roura-Redondo
  • Oskar Almazán-López
  • Alejandro Buldón-Olalla
Original research


The globally widespread instant messaging (IM) mobile applications such as WhatsApp or Telegram were not originally educational tools, but they have become platforms for peer to peer assessment (P2P). The IM applications offer “chatbots” or “virtual assistant bots” that help students by providing them a multitude of services in the form of text or voice dialogs. A new method for integrating P2P assessment using voice recordings with the help of a chatbot is proposed. By using this system we can effectively improve both the typical learning and the P2P evaluation process of a massive open on-line course (MOOC). After a 2-month experiment, with 77 students that recorded 737 voice answers with a Telegram based chatbot, we describe in detail how to use a chatbot and the way to design voice-based challenges to perform a new kind of assignment in a MOOC, with 90% of the learners encouraging us to use chatbots in future courses.


MOOC Chatbot Peer assessment Mobile learning Voice-recording 



The research reported here was supported by ECO European Project, registered in the Competitiveness and Innovation Framework Programme (CIP-ICT-PSP.2013 Theme 2: Digital content, open data and creativity, Obj 2.3.a: Piloting and showcasing excellence in ICT for learning for all).


  1. Acosta, E. S., & Otero, J. J. E. (2014). Clasificación de medios de evaluación en los MOOC. Edutec Revista Electrónica de Tecnología Educativa. Scholar
  2. Admiraal, W., Huisman, B., & Van de Ven, M. (2014). Self-and peer assessment in massive open online courses. International Journal of Higher Education, 3(3), 119–128.CrossRefGoogle Scholar
  3. Ahn, J., Watson, P., Chang, M., Sundararajan, S., Ma, T., Mukhi, N., et al. (2017). Wizard’s apprentice: Cognitive suggestion support for wizard-of-Oz question answering. In E. André, R. Baker, X. Hu, M. M. T. Rodrigo, & B. du Boulay (Eds.), Artificial intelligence in education. Lecture notes in computer science (pp. 630–635). Berlin: Springer.CrossRefGoogle Scholar
  4. Al-lmari, L., Yang, J., & Pimlott, N. (2016). Peer-support writing group in a community family medicine teaching unit: Facilitating professional development. Canadian Family Physician Médecin de Famille Canadien, 62(12), 724–730.Google Scholar
  5. Benotti, L., Martinez, M. C., & Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. CrossRefGoogle Scholar
  6. Bii, P. K., Too, J. K., & Mukwa, C. W. (2018/00/00). Teacher attitude towards use of chatbots in routine teaching. Universal Journal of Educational Research, 6(7), 1586–1597. CrossRefGoogle Scholar
  7. Brixey, J., Hoegen, R., Lan, W., Rusow, J., Singla, K., Yin, X., Artstein, R., & Leuski, A. (2017). Shihbot: A facebook chatbot for sexual health information on hiv/aids. In Proceedings of the 18th annual SIGdial meeting on discourse and dialogue (pp. 370–373).
  8. Briz-Ponce, L., & Juanes-Méndez, J. A. (2015). Mobile devices and apps, characteristics and current potential on learning. Journal of Information Technology Research, 8(4), 26–37. Scholar
  9. Briz-Ponce, L., Pereira, A., Carvalho, L., Juanes-Méndez, J. A., & García-Peñalvo, F. J. (2016). Learning with mobile technologies—Students’ behavior. Computers in Human Behavior, 72, 612–620. Scholar
  10. Cerwall, P., & Lundvall, A. (2016). Ericsson mobility report on the pulse of the networked society. 23 May 2019.
  11. Chao, T., Chen, J., Star, J. R., & Dede, C. (2016). Using digital resources for motivation and engagement in learning mathematics: Reflections from teachers and students. Digital Experiences in Mathematics Education, 2(3), 253–277. Scholar
  12. Chen, M., & Decary, M. (2019). Embedding health literacy tools in patient EHR portals to facilitate productive patient engagement. Studies in Health Technology and Informatics, 257, 59–63.Google Scholar
  13. Chen, J. A., Metcalf, S. J., & Tutwiler, M. S. (2014). Motivation and beliefs about the nature of scientific knowledge within an immersive virtual ecosystems environment. Contemporary Educational Psychology, 39(2), 112–123. Scholar
  14. Conole, G. (2014). A new classification schema for MOOCs. The International Journal for Innovation and Quality in Learning, 2(3), 65–77.Google Scholar
  15. Dalipi, F., Imran, A. S., Idrizi, F., & Aliu, H. (2017). An analysis of learner experience with MOOCs in mobile and desktop learning environment. In J. I. Kantola, T. Barath, S. Nazir, & T. Andre (Eds.), Advances in human factors, business management, training and education (pp. 393–402). Cham: Springer. Scholar
  16. Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-Herráiz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380–392. Scholar
  17. Dubosson, F., Schaer, R., Savioz, R., & Schumacher, M. (2017). Going beyond the relapse peak on social network smoking cessation programmes: ChatBot opportunities. Swiss Medical Informatics, 33(00).Google Scholar
  18. Elmasri, D., & Maeder, A. (2016). A conversational agent for an online mental health intervention. In International conference on brain and health informatics (pp. 243–251). Springer. Scholar
  19. Eranki, K. L. N., & Moudgalya, K. M. (2016). Comparing the effectiveness of self-learning Java workshops with traditional classrooms. Educational Technology & Society, 19(4), 59–74.Google Scholar
  20. Fogg, B. J. (2002). Persuasive technology: Using computers to change what we think and do. Ubiquity 2002 (December). CrossRefGoogle Scholar
  21. Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. Scholar
  22. Galiamova, K., Pavlov, Y., Smirnova, E., Zakharov, M., & Zverev, A. (2018). Psychological adaptation mechanism of the higher education engineering students: Artificial conversational entity usage for help. In Proceedings of INTED2018 (pp. 3472–3476).Google Scholar
  23. Gallagher, S. E., & Savage, T. (2016). Comparing learner community behavior in multiple presentations of a massive open online course. Journal of Computing in Higher Education, 28(3), 358–369. Scholar
  24. González, C. (2010). What do university teachers think eLearning is good for in their teaching? Studies in Higher Education, 35(1), 61–78.CrossRefGoogle Scholar
  25. Graf, B., Krüger, M., Müller, F., Ruhland, A., & Zech, A. (2015). Nombot: Simplify food tracking. In Proceedings of the 14th international conference on mobile and ubiquitous multimedia. ACM, New York, NY, USA, MUM’15 (pp. 360–363).
  26. Güler, Ç. (2017). Use of WhatsApp in higher education: What’s Up with assessing peers anonymously? Journal of Educational Computing Research, 55(2), 272–289. Scholar
  27. Imran, A. S., & Kowalski, S. J. (2014). HIP—A technology-rich and interactive multimedia pedagogical platform. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies: Designing and developing novel learning experiences (pp. 151–160). Berlin: Springer. Scholar
  28. Ismail MH (2015) ELISA: E-learning integrated short announcement. In Colloquium in computer and mathematical sciences education (pp. 110–114).Google Scholar
  29. Kim, J., Glassman, E. L., Monroy-Hernández, A., & Morris, M. R. (2015). RIMES: Embedding interactive multimedia exercises in lecture videos. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, ACM, New York, NY, USA, CHI’15 (pp. 1535–1544).
  30. Kim, T., My, Y., Bae, J., Ba, M., Lee, I., & Kim, J. (2017). Escape from infinite freedom: Effects of constraining user freedom on the prevention of dropout in an online learning context. Computers in Human Behavior, 66, 217–231. Scholar
  31. Klopfenstein, L. C., Delpriori, S., Malatini, S., & Bogliolo, A. (2017). The rise of bots: A survey of conversational interfaces, patterns, and paradigms. In Proceedings of the 2017 conference on designing interactive systems, ACM, New York, NY, USA, DIS’17 (pp. 555–565).
  32. Kowatsch, T., Volland, D., Shih, I., Rüegger, D., Künzler, F., Barata, F., Filler, A., Büchter, D., Brogle, B., & Heldt, K. (2017). Design and evaluation of a mobile chat app for the open source behavioral health intervention platform mobilecoach. In International conference on design science research in information systems (pp. 485–489). Springer. Scholar
  33. Lin, L. M. T. M., Yong, G. B., Fiona, P. J. C., & Amin, S. B. M. (2011). Digital natives learning the periodic table via Microsoft network instant messaging. In I. C. Torres, L. G. Chova & A. L. Martinez (Eds.) 4th International conference of education, research and innovation (pp. 5459–5467).Google Scholar
  34. Lisetti, C., Amini, R., & Yasavur, U. (2015). Now all together: Overview of virtual health assistants emulating face-to-face health interview experience. KI-Künstliche Intelligenz, 29(2), 161–172. CrossRefGoogle Scholar
  35. Liyanagunawardena, T. R., Adams, A. A., & Williams, S. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227. Scholar
  36. Maringe, F., & Sing, N. (2014). Teaching large classes in an increasingly internationalising higher education environment: Pedagogical, quality and equity issues. Higher Education, 67(6), 761–782.CrossRefGoogle Scholar
  37. Meek, S. E., Blakemore, L., & Marks, L. (2017). Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review. Assessment & Evaluation in Higher Education, 42(6), 1000–1013. CrossRefGoogle Scholar
  38. Moos, D. C., & Marroquin, E. (2010). Multimedia, hypermedia, and hypertext: Motivation considered and reconsidered. Computers in Human Behavior, 26(3), 265–276. Scholar
  39. Osuna-Acedo, S., Frau-Meigs, D., Camarero-Cano, L., Bossu, A., Pedrosa, R., & Jansen, D. (2017). Intercreativity and interculturality in the virtual learning environments of the ECO MOOC project. In M. Jemni, Kinshuk, & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 161–187). Berlin: Springer. Scholar
  40. Osuna-Acedo, S., & Quintana, J. G. (2017). The European ECO project. Breaking down barriers to access knowledge. Educación XX1, 20(2), 3. Scholar
  41. Pengnate, S., & Sarathy, R. (2017). An experimental investigation of the influence of website emotional design features on trust in unfamiliar online vendors. Computers in Human Behavior, 67, 49–60. Scholar
  42. Pereira, J. (2016). Leveraging chatbots to improve self-guided learning through conversational quizzes. In Proceedings of the 4th international conference on technological ecosystems for enhancing multiculturality (pp. 911–918). ACM.
  43. Pereira, J. A., Sanz-Santamaría, S., Perurena, I., & Gutiérrez, J. (2012). Interactive speaking practice, assessment and exercise sharing with Babelium plug-in. In 1st Moodle research conference (MRC2012), Heraklion, Crete (pp. 107–114).Google Scholar
  44. Peterson, R. (2013). Why do students drop out of MOOCs? Minding the Campus, 1. Google Scholar
  45. Pimmer, C., Mateescu, M., & Gröhbiel, U. (2016). Mobile and ubiquitous learning in higher education settings. A systematic review of empirical studies. Computers in Human Behavior, 63, 490–501. Scholar
  46. Sadler, P. M., & Good, E. (2006). The impact of self- and peer-grading on student learning. Educational Assessment, 11(1), 1–31. Scholar
  47. Sanou, B. (2016). ICT facts and figures 2016. Technical report. International Telecommunication Union.Google Scholar
  48. Silva, S. (2016). The knowledge contained in oral tradition. E-Scrita-Revista Do Curso De Letras Da Uniabeu, 7(3), 335–349.Google Scholar
  49. So, S. (2016). Mobile instant messaging support for teaching and learning in higher education. The Internet and Higher Education, 31, 32–42. Scholar
  50. Spiteri, M., & Rundgren, S. N. C. (2018). Literature review on the factors affecting primary teachers’ use of digital technology. Technology, Knowledge and Learning, 1–14.Google Scholar
  51. Sridharan, B., Tai, J., & Boud, D. (2018). Does the use of summative peer assessment in collaborative group work inhibit good judgement? Higher Education. Scholar
  52. Stagg Peterson, S., & Dwyer, B. (2016). Research in Canada’s northern rural and indigenous communities: Supporting young children’s oral language and writing. The Reading Teacher, 70(3), 383–387. Scholar
  53. Suanpang, P., & Kalceff, W. (2004). Teamwork vs. individual student projects in an online course. Norfolk: Assoc Advancement Computing Education.Google Scholar
  54. Tarighat, S., & Khodabakhsh, S. (2016). Mobile-assisted language assessment: Assessing speaking. Computers in Human Behavior, 64, 409–413. Scholar
  55. Tenório, T., Bittencourt, I. I., Isotani, S., & Silva, A. P. (2016). Does peer assessment in on-line learning environments work? A systematic review of the literature. Computers in Human Behavior, 64, 94–107. Scholar
  56. Toshalis, E., & Nakkula, M. J. (2012). Motivation, engagement, and student voice. The Education Digest, 78(1), 29.Google Scholar
  57. Ustalov, D. (2015). Teleboyarin. Mechanized labor for telegram. In Proceedings of the AINL-ISMW (pp. 195–197).Google Scholar
  58. van Heerden, A., Ntinga, X., & Vilakazi, K. (2017). The potential of conversational agents to provide a rapid HIV counseling and testing services. In 2017 international conference on the frontiers and advances in data science (FADS) (pp. 80–85). IEEE.
  59. West, M., & Vosloo, S. (2013). UNESCO policy guidelines for mobile learning. Technical report. UNESCO.Google Scholar
  60. Xiong, Y., & Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges and future directions. International Review of Education, 64(2), 241–263.CrossRefGoogle Scholar
  61. Yousef, A. M. F., Chatti, M. A., Wosnitza, M., & Schroeder, U. (2015). A cluster analysis of MOOC stakeholder perspectives. Universities and Knowledge Society Journal, 12(1), 74. Scholar
  62. Zhang, J. (2016). Can MOOCs be interesting to students? An experimental investigation from regulatory focus perspective. Computers & Education, 95, 340–351. Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Computer Languages and SystemsUniversity of the Basque CountryLeioa, BilbaoSpain
  2. 2.Department of Applied Chemistry and PhysicsUniversity of LeonLeónSpain
  3. 3.Department of Didactic and Scholar Organization and D.D.E.E.National Distance Education University (UNED)MadridSpain
  4. 4.Department of Specific Didactic, Cardenal Cisneros University CollegeUniversity of AlcaláAlcalá De Henares, MadridSpain
  5. 5.Portland Public SchoolsPortlandUSA
  6. 6.GetafeSpain

Personalised recommendations