Skip to main content
SpringerLink
Log in

Quizbot: Exploring Formative Feedback with Conversational Interfaces

  • Conference paper
  • First Online:
Technology Enhanced Assessment (TEA 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1014))

Included in the following conference series:

Abstract

Conversational interfaces (also called chatbots) have recently disrupted the Internet and opened up endless opportunities for assessment and learning. Formative feedback that provides learners with practical instructions for improvement is one of the challenging tasks in self-assessment settings and self-directed learning. This becomes even more challenging if a user’s personal information such as learning history and previous achievements cannot be exploited for data protection reasons or are simply not available. This study seeks to explore the opportunities of providing formative feedback in chatbot-based self-assessment. Two main challenges were faced: the limitations of the messenger as an interface that restricts visual representation of the quiz questions, and zero information about the user to generate adaptive feedback. Two types of feedback were investigated regarding their formative effect: immediate feedback, which was given after answering a question, and cumulative feedback detailing strengths and weaknesses of the user in each of the topics covered along with the directives for improvement. A chatbot called SQL Quizbot was deployed on Facebook Messenger for the purposes of this study (Try out the prototype at https://www.messenger.com/t/2076690849324267). A survey conducted to disclose users’ perception of the feedback reveals that more than 80% of the users find immediate feedback helpful. Overall this study shows that chatbots have a great potential as an aiding tool for e-learning systems to include an interactive component into feedback in order to increase user motivation and retention.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://uk.businessinsider.com/the-messaging-app-report-2015-11?r=US&IR=T, last checked 22.11.2018.

  2. 2.

    https://chatfuel.com.

  3. 3.

    https://Firebase.google.com.

  4. 4.

    https://www.Integromat.com/.

References

  1. Muirhead, B., Juwah, C.: Interactivity in computer-mediated college and university education: a recent review of the literature. J. Educ. Technol. Soc. 7(1), 12–20 (2004)

    Google Scholar 

  2. Zheng, S., Rosson, M.B., Shih, P.C., Carroll, J.M.: Understanding student motivation, behaviors and perceptions in MOOCs. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & #38; Social Computing, CSCW 2015, pp. 1882–1895. ACM, New York (2015)

    Google Scholar 

  3. Weizenbaum, J.: ELIZA - a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 36–45 (1966)

    Article  Google Scholar 

  4. Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008)

    Article  Google Scholar 

  5. Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77(1), 81–112 (2007)

    Article  Google Scholar 

  6. Lipowsky, F.: Unterricht. In: Wild, E., Möller, J. (eds.) Pädagogische Psychologie. SLB, pp. 69–105. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-642-41291-2_4

    Chapter  Google Scholar 

  7. Wiliam, D.: Embedded Formative Assessment. Solution Tree Press, Bloomington (2011)

    Google Scholar 

  8. Höhn, S., Ras, E.: Designing formative and adaptive feedback using incremental user models. In: Chiu, D.K.W., Marenzi, I., Nanni, U., Spaniol, M., Temperini, M. (eds.) ICWL 2016. LNCS, vol. 10013, pp. 172–177. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47440-3_19

    Chapter  Google Scholar 

  9. Denton, P., Madden, J., Roberts, M., Rowe, P.: Students’ response to traditional and computer-assisted formative feedback: a comparative case study. Br. J. Educ. Technol. 39(3), 486–500 (2008)

    Article  Google Scholar 

  10. Black, P., Wiliam, D.: Developing the theory of formative assessment. Educ. Assess. Eval. Account. (Former.: J. Pers. Eval. Educ.) 21(1), 5 (2009)

    Article  Google Scholar 

  11. Wiggins, G.: Seven keys to effective feedback. Educ. Leadersh. 70(1), 10–16 (2012)

    Google Scholar 

  12. Espasa, A., Guasch, T., Mayordomo, R., Martínez-Melo, M., Carless, D.: A dialogic feedback index measuring key aspects of feedback processes in online learning environments. High. Educ. Res. Dev. 37(3), 499–513 (2018)

    Article  Google Scholar 

  13. Narciss, S.: Designing and evaluating tutoring feedback strategies for digital learning. Digit. Educ. Rev. 23, 7–26 (2013)

    Google Scholar 

  14. Butler, A.C., Roediger, H.L.: Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Mem. Cogn. 36(3), 604–616 (2008)

    Article  Google Scholar 

  15. Bälter, O., Enström, E., Klingenberg, B.: The effect of short formative diagnostic web quizzes with minimal feedback. Comput. Educ. 60(1), 234–242 (2013)

    Article  Google Scholar 

  16. De Klerk, S., Veldkamp, B.P., Eggen, T.: The psychometric evaluation of a summative multimedia-based performance assessment. In: Ras, E., Joosten-ten Brinke, D. (eds.) CAA 2015. CCIS, vol. 571, pp. 1–11. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27704-2_1

    Chapter  Google Scholar 

  17. Novacek, P.: Confidence-based assessments within an adult learning environment. In: International Association for Development of the Information Society, pp. 403–406 (2013)

    Google Scholar 

  18. Hench, T.L.: Using confidence as feedback in multi-sized learning environments. In: Kalz, M., Ras, E. (eds.) CAA 2014. CCIS, vol. 439, pp. 88–99. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08657-6_9

    Chapter  Google Scholar 

  19. Lundeberg, M.A., Fox, P.W., Punćcohaí, J.: Highly confident but wrong: gender differences and similarities in confidence judgments. J. Educ. Psychol. 86(1), 114 (1994)

    Article  Google Scholar 

  20. Jonsson, A.C., Allwood, C.M.: Stability and variability in the realism of confidence judgments over time, content domain, and gender. Pers. Individ. Differ. 34(4), 559–574 (2003)

    Article  Google Scholar 

  21. Burns, K.M., Burns, N.R., Ward, L.: Confidence–more a personality or ability trait? It depends on how it is measured: a comparison of young and older adults. Front. Psychol. 7, 518 (2016)

    Article  Google Scholar 

  22. West, R.F., Stanovich, K.E.: The domain specificity and generality of overconfidence: individual differences in performance estimation bias. Psychon. Bull. Rev. 4(3), 387–392 (1997)

    Article  Google Scholar 

  23. Gardner-Medwin, A.: Confidence assessment in the teaching of basic science. ALT-J 3(1), 80–85 (1995)

    Article  Google Scholar 

  24. Gardner-Medwin, A.: 12 confidence-based marking. In: Innovative Assessment in Higher Education, p. 141 (2006)

    Google Scholar 

  25. Ericsson, K.A., Krampe, R.T., Tesch-Römer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100(3), 363–406 (1993)

    Article  Google Scholar 

  26. Christodoulou, D., Wiliam, D.: Making Good Progress?: The Future of Assessment for Learning. Oxford University Press, Oxford (2017)

    Google Scholar 

  27. Roediger, H.L., Karpicke, J.D.: The power of testing memory: basic research and implications for educational practice. Perspect. Psychol. Sci. 1(3), 181–210 (2006). PMID: 26151629

    Article  Google Scholar 

  28. Petersen, K.A.: Implicit corrective feedback in computer-guided interaction: does mode matter? Ph.D. thesis, Georgetown University (2010)

    Google Scholar 

  29. Wilske, S.: Form and meaning in dialog-based computer-assisted language learning. Ph.D. thesis, University of Saarland (2014)

    Google Scholar 

  30. Kerly, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl.-Based Syst. 20(2), 177–185 (2007)

    Article  Google Scholar 

  31. Kane, D.A.: The role of chatbots in teaching and learning. In: E-Learning and the Academic Library: Essays on Innovative Initiatives, UC Irvine, pp. 1–26 (2016)

    Google Scholar 

  32. Soliman, M., Guetl, C.: Intelligent pedagogical agents in immersive virtual learning environments: a review. In: MIPRO 2010 Proceedings of the 33rd International Convention, pp. 827–832. IEEE (2010)

    Google Scholar 

  33. MacTear, M., Callejas, Z., Griol, D.: The Conversational Interface: Talking to Smart Devices. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32967-3

    Book  Google Scholar 

  34. DeSmedt, W.H.: Herr Kommissar: an ICALL conversation simulator for intermediate German. In: Holland, V.M., Sams, M.R., Kaplan, J.D. (eds.) Intelligent Language Tutors: Theory Shaping Technology. Routledge, New York (1995)

    Google Scholar 

  35. Lu, C.-H., Chiou, G.-F., Day, M.-Y., Ong, C.-S., Hsu, W.-L.: Using instant messaging to provide an intelligent learning environment. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 575–583. Springer, Heidelberg (2006). https://doi.org/10.1007/11774303_57

    Chapter  Google Scholar 

  36. Jia, J.: CSIEC: a computer assisted english learning chatbot based on textual knowledge and reasoning. Knowl.-Based Syst. 22(4), 249–255 (2009)

    Article  Google Scholar 

  37. Höhn, S.: A data-driven model of explanations for a chatbot that helps to practice conversation in a foreign language. In: Proceedings of SIGDIAL 2017 Conference. ACM (2017)

    Google Scholar 

  38. Lyster, R., Ranta, L.: Corrective feedback and learner uptake. Stud. Second Lang. Acquis. 19(01), 37–66 (1997)

    Article  Google Scholar 

  39. Lyster, R., Saito, K., Sato, M.: Oral corrective feedback in second language classrooms. Lang. Teach. 46, 1–40 (2013)

    Article  Google Scholar 

  40. Amaral, L.A., Meurers, D.: On using intelligent computer-assisted language learning in real-life foreign language teaching and learning. ReCALL 23(01), 4–24 (2011)

    Article  Google Scholar 

  41. Gross, S., Pinkwart, N.: Towards an integrative learning environment for Java programming. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies (ICALT), pp. 24–28, July 2015

    Google Scholar 

  42. Perikos, I., Grivokostopoulou, F., Hatzilygeroudis, I.: Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. Int. J. Artif. Intell. Educ. 27(3), 475–514 (2017)

    Article  Google Scholar 

  43. Ryan, T., Henderson, M.: Feeling feedback: students’ emotional responses to educator feedback. Assess. Eval. High. Educ. 43(6), 880–892 (2018)

    Article  Google Scholar 

  44. Palminteri, S., Khamassi, M., Joffily, M., Coricelli, G.: Contextual modulation of value signals in reward and punishment learning. Nat. Commun. 6, 8096 (2015). https://doi.org/10.1038/ncomms9096

    Article  Google Scholar 

  45. Ras, E., Baudet, A., Foulonneau, M.: A hybrid engineering process for semi-automatic item generation. In: Joosten-ten Brinke, D., Laanpere, M. (eds.) TEA 2016. CCIS, vol. 653, pp. 105–116. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57744-9_10

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Luxembourg, Esch-sur-Alzette, Luxembourg

    Bharathi Vijayakumar

  2. Artificial Companions and Chatbots Lab, University of Luxembourg, 6, Avenue de la Fonte, 4364, Esch-sur-Alzette, Luxembourg

    Sviatlana Höhn

  3. Interdisciplinary Lab for Intelligent and Adaptive Systems, University of Luxembourg, 6, Avenue de la Fonte, 4364, Esch-sur-Alzette, Luxembourg

    Christoph Schommer

Authors
  1. Bharathi Vijayakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sviatlana Höhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christoph Schommer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sviatlana Höhn .

Editor information

Editors and Affiliations

  1. Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

    Silvester Draaijer

  2. Welten Instituut, Open University of the Netherlands, Heerlen, The Netherlands

    Desirée Joosten-ten Brinke

  3. Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg

    Eric Ras

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vijayakumar, B., Höhn, S., Schommer, C. (2019). Quizbot: Exploring Formative Feedback with Conversational Interfaces. In: Draaijer, S., Joosten-ten Brinke, D., Ras, E. (eds) Technology Enhanced Assessment. TEA 2018. Communications in Computer and Information Science, vol 1014. Springer, Cham. https://doi.org/10.1007/978-3-030-25264-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-25264-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-25263-2

  • Online ISBN: 978-3-030-25264-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation