Pedagogical Evaluation of Automatically Generated Questions

  • Karen Mazidi
  • Rodney D. Nielsen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8474)


Automatic Question Generation from text is a critical component of educational technology applications such as Intelligent Tutoring Systems. We describe an automatic question generator that uses semantic-based templates. We evaluate the system along with two comparable systems for both linguistic quality and pedagogical value of generated questions and find that our system outperforms prior work.


question generation syntactic semantic pedagogy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, M., Shah, R., Mannem, P.: Automatic question generation using discourse cues. In: Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 1–9. Association for Computational Linguistics (2011)Google Scholar
  2. 2.
    Babko-Malaya, O.: Propbank annotation guidelines (2005), http://www.verbs
  3. 3.
    Beck, J.E., Mostow, J., Bey, J.: Can automated questions scaffold childrens reading comprehension? In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 478–490. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Boyer, K., Piwek, P. (eds.): Proc. QG2010: The Third Workshop on Question Generation, Pittsburgh, PA (2010)Google Scholar
  5. 5.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Machine Language Research 12, 2493–2537 (2011)zbMATHGoogle Scholar
  6. 6.
    Graesser, A.C., Rus, V., Cai, Z., Hu, X.: Question answering and generation. In: McCarthy, P.M., Boonthum, C. (eds.) Applied NLP. IGI Global, Hershey (2011)Google Scholar
  7. 7.
    Graesser, A.C., Rus, V., Cai, Z.: Question classification schemes. In: Proc. WS of the QGSTEC (2008)Google Scholar
  8. 8.
    Heilman, M., Smith, N.A.: Question generation via overgeneration transformations and ranking. No. CMU-LTI-09-013. Carnegie-Mellon University, Pittsburgh (2009)Google Scholar
  9. 9.
    Heilman, M., Smith, N.A.: Rating computer-generated questions with Mechanical Turk. In: Proc. of the NAACL HLT 2010 Workshop CSLDAMT, pp. 35–40. Association for Computational Linguistics (2010)Google Scholar
  10. 10.
    Lindberg, D., Popowich, F., Nesbit, J., Winne, P.: Generating natural language questions to support learning on-line. In: Proc. 14th European Workshop NLG, pp. 105–114. Association for Computational Linguistics (2013)Google Scholar
  11. 11.
    Mannem, P., Prasad, R., Joshi, A.: Question generation from paragraphs at UPenn. In: Proc. QG2010: The Third Workshop on Question Generation, Pittsburgh, PA, pp. 84–91 (2010)Google Scholar
  12. 12.
    Roediger, H.L., Pyc, M.A.: Inexpensive techniques to improve education. J. Applied Research in Memory and Cognition 1(4), 242–248 (2012)CrossRefGoogle Scholar
  13. 13.
    Wolfe, J.H.: Automatic question generation from text. ACM SIGCUE Outlook 10(SI), 104–112 (1976)CrossRefGoogle Scholar
  14. 14.
    Wyse, B., Piwek, P.: Generating questions from openlearn study units (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Karen Mazidi
    • 1
  • Rodney D. Nielsen
    • 1
  1. 1.HiLT Lab.University of North TexasDentonUSA

Personalised recommendations