Advertisement

Using Lexical Semantic Techniques to Classify Free-Responses

  • Jill Burstein
  • Susanne Wolff
  • Chi Lu
Part of the Text, Speech and Language Technology book series (TLTB, volume 10)

Abstract

The education community wants to include more performance-based assessments on standardized exams. The research described in i;his paper shows the use of lexical semantic techniques for automated scoring of short-answer and essay responses from performance-based test items. We use lexical semantic techniques in order to identify the meaningful content of free-text responses for small data sets. The research demonstrates applications of lexical semantic techniques for free-text responses of varying length and in different subject domains. Prototype designs, and the results of the different prototype applications are discussed.

Keywords

Test Item Test Question Lexical Semantic Police Item Superordinate Concept 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bergler, S. 1995. From Lexical Semantics to Text Analysis. In P. Saint-Dizier and E. Vie-gas (eds.), Computational Lexical Semantics. Cambridge, UK: Cambridge University Press.Google Scholar
  2. Burstein, J. C., K. Kukich, L. Braden-Harder, M. Chodorow, S. Hua, B. Kaplan, C Lu, J. Nolan, D. Rock and S. Wolff. 1998a. Computer Analysis of Essay Content for Automated Score Prediction: A prototype automated scoring system for GMAT Analytical Writing Assessment. (RR-98-15). Princeton, NJ: Educational Testing Service.Google Scholar
  3. Burstein, J. C., K. Kukich, S. Wolff, C. Lu, C. M. Chodorow, L. Braden-Harder and M. D. Harris. 1998b. Automated Scoring Using A Hybrid Feature Identification Technique. In the Proceedings of 36th the Annual Meeting of the Association of Computational Linguistics,Montréal, Canada.Google Scholar
  4. Burstein, J. C., K. Kukich, S. Wolff, C. Lu and M. Chodorow. 1998c. Enriching Automated Scoring Using Discourse Marking. In the Proceedings of the Workshop on Discourse Relations and Discourse Marking,36th Annual Meeting of the Association of Computational Linguistics, Montréal, Canada.Google Scholar
  5. Burstein, J. C., S. Wolff, C., Lu and R. M. Kaplan. 1997. An Automatic Scoring System for Advanced Placement Biology Essays. In the Proceedings of the Fifth Conference on Applied Natural Language Processing,Washington, DC.Google Scholar
  6. Burstein, J. C. and R. M. Kaplan. 1995a. On the Application of Context to Natural Language Processing Applied to the Analysis of Test Responses. In the Proceedings from the Workshop on Context in Natural Language Processing,IJCAI, Montréal, Canada.Google Scholar
  7. Burstein, J. C. and R. M. Kaplan. 1995b. GE-FRST Evaluation Report: How well does a Statistically-Based Natural Language Processing System Score Natural LanguageGoogle Scholar
  8. Constructed Responses? (RR-95-29). Princeton, NJ: Educational Testing Service. Church, K. and P. Hanks. 1990. Word Association Norms, Mutual Information and Lexicography. Computational Linguistics,16(1), pp. 22-29.Google Scholar
  9. Cruse, D. A. 1986. Lexical Semantics. Cambridge, UK• Cambridge University Press. Fellbaum, C. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press.Google Scholar
  10. Foltz, P. W., W. Kintsch and T. K. Landauer. 1998. The Measurement of Textual Co- herence with Latent Semantic Analyis. Discourse Processes, 25 (2amp3), pp. 285 - 307.CrossRefGoogle Scholar
  11. Gerstl, P. 1991. A Model for the Interaction of Lexical and Non-Lexical Knowledge in the Determination of Word Meaning. In J. Pustejovsky and S. Bergler (eds.), Lexical Semantics and Knowledge Representation. New York, NY: Springer-Verlag.Google Scholar
  12. Holland, M. V. 1994. Intelligent Tutors for Foreign Languages: How Parsers and Lexical Semantics can Help Learners and Assess Learning. In the Proceedings of the Educational Testing Service Conference on Natural Language Processing Techniques and Technology in Education and Assessment,Princeton, New Jersey.Google Scholar
  13. Kaplan, R. M. and R. E. Bennett. 1994. Using the Free-Response Scoring Tool To Automatically Score the Formulating-Hypothesis Item. (RR-94-08). Princeton, NJ: Educational Testing Service.Google Scholar
  14. Kud, J. M., G. G. Kripka and L. Rau. 1994. Methods Short Answer Responses for Categorizing. In the Proceedings of the Educational Testing Service Conference on Natural Language Processing Techniques and Technology in Education and Assessment,Princeton, New Jersey.Google Scholar
  15. Montemagni, S. and L. Vanderwende. 1993. Structural Patterns versus String Patterns for Extracting Semantic Information from Dictionaries. In K. Jensen, G. Heidorn and S. Richardson (eds.) Natural Language Processing: The PLNLP Approach. Boston, MA: Kluwer Academic Publishers.Google Scholar
  16. MSNLP (1996): http://research.microsoft.com/research/nlp Google Scholar
  17. Quirk, R., S. Greenbaum, G. Leech and J. Svartik 1985. A Comprehensive Grammar of the English Language. New York: Longman.Google Scholar
  18. Richardson, S. D., L. Vanderwende and W. Dolan. 1993. Combining Dictionary bases and Example-Based Methods for Natural Language Analysis. (MSR-TR-93-08), Redmond, WA, Microsoft Corporation.Google Scholar
  19. Sager, N. 1981. Natural Language Information Processing: A computer grammar of English and its applications. Reading, Massachusetts: Addison-Wesley.Google Scholar
  20. Smadja, F. 1993. Retrieving Collocations from Text: Xtract. Computational Linguistics 19 (1), pp. 143 - 177.Google Scholar
  21. Soderland, S., D. Fisher, J. Aseltine and W. Lehnert. 1994. CRYSTAL: Inducing a Con- ceptual Dictionary, available from http://ciir.cs.umass.edu/medical/ss-ijkai.hmtl. Google Scholar
  22. Tsutsumi, T. 1992. Word Sense Disambiguation by Examples. In K. Jensen, G. Heidorn and S. Richardson (eds.), Natural Language Processing: the PLNLP Approach. Boston, MA: Kluwer Academic Publishers.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1999

Authors and Affiliations

  • Jill Burstein
    • 1
  • Susanne Wolff
    • 2
  • Chi Lu
    • 2
  1. 1.Educational Testing Service - llRPrincetonUSA
  2. 2.Educational Testing Service - 17RPrincetonUSA

Personalised recommendations