Skip to main content
SpringerLink
Log in
Book cover

Ibero-American Conference on Artificial Intelligence

IBERAMIA 2018: Advances in Artificial Intelligence - IBERAMIA 2018 pp 380–391Cite as

Machine Learning Approach for Automatic Short Answer Grading: A Systematic Review

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11238))

Abstract

In this systematic review, we investigate the automatic short answer grading (ASAG) field, which focuses on assessing short natural language responses to questions in an automatic way. Short answers have been recognized as a tool to perform a deeper assessment of the student’s knowledge than, for example, multiple choice questions. Automatically scoring short responses can be used as an important resource to the educational field, where the student’s answers can be easily, fairly and quickly evaluated for feedback purposes in, for instance, massive open online courses, in which precision and agility are required. We conducted the research by including only works that employed machine learning methods in order to solve the problem. The final selection considering all criteria selected 44 papers reporting different ASAG systems. Those studies were analyzed by answering the proposed research questions, extracting: the nature of datasets, used natural language processing and machine learning techniques, features selected to create the models and the results obtained from their systems’ evaluation.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://web.eecs.umich.edu/~mihalcea/downloads.html.

  2. 2.

    www.uni-tuebingen.de/en/research/core-research/collaborative-research-centers/sfb-833/section-a-context/a4-meurers/software-resources-and-corpora.html.

  3. 3.

    www.kaggle.com/c/asap-sas.

  4. 4.

    www.cs.york.ac.uk/semeval-2013/task7/index.php%3Fid=data.html.

  5. 5.

    https://nlp.stanford.edu/software/lex-parser.shtml.

References

  1. Burrows, S., Gurevych, I., Stein, B.: The eras and trends of automatic short answer grading. Int. J. Artif. Intell. Educ. 25, 60–117 (2015)

    Article  Google Scholar 

  2. Dzikovska, M., et al.: SemEval-2013 task 7: the joint student response analysis and 8th recognizing textual entailment challenge. In: Seventh International Workshop on Semantic Evaluation, pp. 263–274 (2013)

    Google Scholar 

  3. Hasanah, U., Permanasari, A.E., Kusumawardani, S.S., Pribadi, F.S.: A review of an information extraction technique approach for automatic short answer grading. In: International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), pp. 192–196. IEEE (2016)

    Google Scholar 

  4. Kitchenham, B.: Procedures for performing systematic reviews. Keele, UK, Keele Univ. 33(2004), 1–26 (2004)

    Google Scholar 

  5. Kouylekov, M., Dini, L., Bosca, A., Trevisan, M.: Celi: EDITS and generic text pair classification. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), vol. 2, pp. 592–597 (2013)

    Google Scholar 

  6. Levy, O., Zesch, T., Dagan, I., Gurevych, I.: UKP-BIU: similarity and entailment metrics for student response analysis. In: Second Joint Conference on Lexical and Computational Semantics, Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), vol. 2, pp. 285–289 (2013)

    Google Scholar 

  7. Liu, O.L., Rios, J.A., Heilman, M., Gerard, L., Linn, M.C.: Validation of automated scoring of science assessments. J. Res. Sci. Teach. 53(2), 215–233 (2016). https://doi.org/10.1002/tea.21299

    Article  Google Scholar 

  8. Magooda, A., Zahran, M.A., Rashwan, M., Raafat, H., Fayek, M.B.: Vector based techniques for short answer grading. In: International Florida Artificial Intelligence Research Society Conference Ahmed, pp. 238–243 (2016)

    Google Scholar 

  9. Meurers, D., Ziai, R., Ott, N., Bailey, S.M.: Integrating parallel analysis modules to evaluate the meaning of answers to reading comprehension questions. Int. J. Continuing Eng. Educ. Life-Long Learn. 21(4), 355 (2011). https://doi.org/10.1504/IJCEELL.2011.042793

    Article  Google Scholar 

  10. Meurers, D., Ziai, R., Ott, N., Kopp, J.: Evaluating answers to reading comprehension questions in context: results for German and the role of information structure. In: Proceedings of the TextInfer 2011 Workshop on Textual Entailment, pp. 1–9 (2011)

    Google Scholar 

  11. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  12. Mohler, M., Bunescu, R., Mihalcea, R.: Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 752–762 (2011)

    Google Scholar 

  13. Passero, G., Haendchen Filho, A., Dazzi, R.: Avaliação do uso de métodos baseados em lsa e wordnet para correção de questões discursivas. In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), vol. 27, p. 1136 (2016)

    Google Scholar 

  14. Pérez-Marín, D., Pascual-Nieto, I., Rodríguez, P.: Computer-assisted assessment of free-text answers. Knowl. Eng. Rev. 24(04), 353–374 (2009)

    Article  Google Scholar 

  15. Roy, S., Narahari, Y., Deshmukh, O.D.: A perspective on computer assisted assessment techniques for short free-text answers. In: Ras, E., Joosten-ten Brinke, D. (eds.) CAA 2015. CCIS, vol. 571, pp. 96–109. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27704-2_10

    Chapter  Google Scholar 

  16. Santos, J.C.A.D., et al.: Avaliação automática de questões discursivas usando lsa. Universidade Federal do Pará (2016)

    Google Scholar 

  17. Sukkarieh, J.Z.: Using a MaxEnt classifier for the automatic content scoring of free-text responses. In: American Institute of Physics Conference Proceedings, pp. 41–48 (2010). https://doi.org/10.1063/1.3573647

  18. Valenti, S., Neri, F., Cucchiarelli, A.: An overview of current research on automated essay grading. J. Inf. Technol. Educ. 2, 319–330 (2003)

    Google Scholar 

  19. Vijaymeena, M., Kavitha, K.: A survey on similarity measures in text mining. Mach. Learn. Appl.: Int. J. 3(2), 19–28 (2016)

    Google Scholar 

  20. Ziai, R., Ott, N., Meurers, D.: Short answer assessment: establishing links between research strands. In: Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pp. 190–200 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, State University of Londrina, Londrina, PR, Brazil

    Lucas Busatta Galhardi & Jacques Duílio Brancher

Authors
  1. Lucas Busatta Galhardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacques Duílio Brancher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucas Busatta Galhardi .

Editor information

Editors and Affiliations

  1. Universidad Nacional del Sur, Bahía Blanca, Buenos Aires, Argentina

    Guillermo R. Simari

  2. University of Madeira, Funchal, Portugal

    Eduardo Fermé

  3. Universidad Nacional de Piura, Castilla-Piura, Peru

    Flabio Gutiérrez Segura

  4. Universidad Nacional de Trujillo, Trujillo, Peru

    José Antonio Rodríguez Melquiades

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Galhardi, L.B., Brancher, J.D. (2018). Machine Learning Approach for Automatic Short Answer Grading: A Systematic Review. In: Simari, G., Fermé, E., Gutiérrez Segura, F., Rodríguez Melquiades, J. (eds) Advances in Artificial Intelligence - IBERAMIA 2018. IBERAMIA 2018. Lecture Notes in Computer Science(), vol 11238. Springer, Cham. https://doi.org/10.1007/978-3-030-03928-8_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03928-8_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03927-1

  • Online ISBN: 978-3-030-03928-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation