Automatic Generation of Named Entity Distractors of Multiple Choice Questions Using Web Information
This paper presents a novel technique for automatic generation of distractors for multiple choice questions. Distractors are the wrong choices given along with the correct answer (key) to befuddle the examinee. Various techniques have been proposed in the literature for automatic distractor generation. But none of these approaches are suitable when the key is a named entity. And named entity key or distractors are dominating in many domains including sports and entertainment. Here, we propose a technique for generation of named entity distractors. For generating good named entity distractors, we first detect the class of the key and collect a set of attribute values, classified into generic and specific categories. Based on these attributes, we retrieve a set of candidate distractors from a few trusted Web sites like Wikipedia. Then, we find the similarity between the key and a candidate distractor. The close ones are chosen as the final set of distractors. A set of human evaluators assess the distractors by using a set of parameters. In our evaluation, we observe that the system-generated distractors are good in terms of relevance and close to the key.
KeywordsDistractors MCQ Question generation Named entity
This work is supported by the project grant (project file no.: YSS/2015/001948) provided by the Science and Engineering Research Board (SERB), Govt. of India.
- 1.Agarwal Manish and Mannem Prashanth. 2011. Automatic Gap-fill Question Generation from Text Books. Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 56–64.Google Scholar
- 2.Aldabe Itziar and Maritxalar Montse. 2010. Automatic Distractor Generation for Domain Specific Texts. IceTAL 2010, LNAI 6233, pp. 27–38.Google Scholar
- 3.Bhatia Arjun Singh, Kirti Manas and Saha Sujan Kumar. 2013. Automatic Generation of Multiple Choice Questions using Wikipedia. Proc. of Pattern Recognition and Machine Intelligence (PReMI -13), LNCS Vol. 8251, pp. 733–738.Google Scholar
- 4.Brown JC., Frishkoff GA and Eskenazi M. 2005. Automatic question generation for vocabulary assessment. Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pp. 819–826.Google Scholar
- 5.Coniam David. 1997. A Preliminary Inquiry into Using Corpus Word Frequency Data in the Automatic Generation of English Language Cloze Tests. CAL-ICO Journal, 14 (2):15–33.Google Scholar
- 6.Correia, R., Baptista, J., Mamede, N., Trancoso, I., and Eskenazi M. 2010. Automatic Generation of Cloze Question Distractors. In Second Language Studies: Acquisition, Learning, Education and Technology.Google Scholar
- 7.Lafferty John D., McCallum Andrew and Pereira Fernando C. N. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proc. of Eighteenth International Conference on Machine Learning, pp. 282–289.Google Scholar
- 8.Majumdar Mukta, Saha Sujan Kumar. 2015. A System for Generating Multiple Choice Questions: With a Novel Approach for Sentence Selection. Proceedings of the 2nd ACL Workshop on Natural Language Processing Techniques for Educational Applications (NLP-TEA), pages 64–72.Google Scholar
- 9.McKenna Colleen and Bull Joanna. 1999. Designing effective objective test questions: an introductory workshop. Technical Report: CAA Centre, Lough-borough University.Google Scholar
- 10.Mitkov R. and Ha L.A. 2003. Computer-aided generation of multiple-choice tests. Proceedings of the HLT/NAACL 2003 Workshop on Building educational applications using Natural Language Processing. pp. 17–22.Google Scholar
- 11.Mitkov, R., Ha, L.A., Varga, A. and Rello, L. 2009. Semantic similarity of distractors in multiple-choice tests: extrinsic evaluation. Proceedings of EACL 2009 Workshop on GEMS: GEometical Models of Natural Language Semantics, pp. 49–56.Google Scholar
- 12.Papasalouros A., Kanaris K and Kotis K. 2008. Automatic Generation of multiple-choice questions from domain ontologies. IADIS e-Learning.Google Scholar