Skip to main content
SpringerLink
Log in
Book cover

Soft Computing Applications in Business pp 201–230Cite as

Soft Computing in Intelligent Tutoring Systems and Educational Assessment

  • Chapter

Part of the book series: Studies in Fuzziness and Soft Computing ((STUDFUZZ,volume 230))

Abstract

The need for soft computing technologies to facilitate effective automated tutoring is pervasive – from machine learning techniques to predict content significance and generate appropriate questions, to interpretation of noisy spoken responses and statistical assessment of the response quality, through user modeling and determining how best to respond to the learner in order to optimize learning gains. This chapter focuses primarily on the domain-independent semantic analysis of learner responses, reviewing prior work in intelligent tutoring systems and educational assessment. We present a new framework for assessing the semantics of learner responses and the results of our initial implementation of a machine learning approach based on this framework.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Agichtein, E., Gravano, L.: Snowball: Extracting relations from large plain-text collections. In: Proc. of the 5th ACM ICDL (2000)

    Google Scholar 

  • Aleven, V., Popescu, O., Koedinger, K.R.: A tutorial dialogue system with knowledge-based understanding and classification of student explanations. In: IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems (2001)

    Google Scholar 

  • Anderson, J.R., Corbett, A.T., Koedinger, K., Pelletier, R.: Cognitive tutors: Lessons learned. J of Learning Sciences 4, 167–207 (1995)

    Article  Google Scholar 

  • Banerjee, S., Pedersen, T.: Extended gloss overlaps as a measure of semantic relatedness. In: IJCAI, pp. 805–810 (2003)

    Google Scholar 

  • Barzilay, R., Lee, L.: Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In: Proc. of HLT-NAACL, pp. 16–23 (2003)

    Google Scholar 

  • Barzilay, R., McKeown, K.: Extracting paraphrases from a parallel corpus. In: Proc. of the ACL/EACL, pp. 50–57 (2001)

    Google Scholar 

  • Bennett R.: Reinventing assessment: Speculations on the future of large-scale educational testing. Educational Testing Service, on August 16, 2005 (1998) http://www.ets.org/research/pic/bennett.html (Downloaded from August 16, 2005)

  • Bethard, S., Nielsen, R.D., Martin, J.H., Ward, W., Palmer, M.: Semantic integration in learning from text. In: Machine Reading AAAI Spring Symposium (2007)

    Google Scholar 

  • Bloom, B.S.: The 2 sigma problem: The search for methods of group instruction as effective as one-on-one tutoring. Educational Researcher 13, 4–16 (1984)

    Google Scholar 

  • Braz, R.S., Girju, R., Punyakanok, V., Roth, D., Sammons, M.: An inference model for semantic entailment in natural language. In: Proc. of the PASCAL Recognizing Textual Entailment challenge workshop (2005)

    Google Scholar 

  • Breiman, L.: Random forests. Journal of Machine Learning 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  • Budanitsky, A., Hirst, G.: Evaluating WordNet-based measures of lexical semantic relatedness. Computational Linguistics 32(1) (2006)

    Google Scholar 

  • Burger, J., Ferro, L.: Generating an entailment corpus from news headlines. In: Proc. of the ACL workshop on Empirical Modeling of Semantic Equivalence and Entailment, pp. 49–54 (2005)

    Google Scholar 

  • Callear, D., Jerrams-Smith, J., Soh, V.: CAA of short non-MCQ answers. In: Proc. of the 5th International CAA conference, Loughborough (2001)

    Google Scholar 

  • Chi, M.T.H.: Constructing self-explanations and scaffolded explanations in tutoring. Applied Cognitive Psychology 10, 33–49 (1996)

    Article  MathSciNet  Google Scholar 

  • Church, K.W., Hanks, P.: Word association norms, mutual information, and lexicography. In: Proceedings of the 27th ACL, Vancouver BC, pp. 76–83 (1989)

    Google Scholar 

  • Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 37–46 (1960)

    Article  Google Scholar 

  • Cohen, P.A., Kulik, J.A., Kulik, C.L.C.: Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research J 19, 237–248 (1982)

    Google Scholar 

  • Cole R., VanVuuren S., Pellom B., Hacioglu K., Ma J., Movellan J., Schwartz S., Wade-Stein D., Ward W., Yan J.: Perceptive animated interfaces: First steps toward a new paradigm for human computer interaction (2003)

    Google Scholar 

  • Cowie, J., Lehnert, W.G.: Information extraction. Communications of the ACM 39(1), 80–91 (1996)

    Article  Google Scholar 

  • Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognizing Textual Entailment Challenge. In: Proc. of the PASCAL RTE challenge workshop (2005)

    Google Scholar 

  • Duclaye, F., Yvon, F., Collin, O.: Using the web as a linguistic resource for learning reformulations automatically. LREC (2002)

    Google Scholar 

  • Franzke, M., Kintsch, E., Caccamise, D., Johnson, N., Dooley, S.: Summary Street: Computer support for comprehension and writing. Journal of Educational Computing Research 33, 53–80 (2005)

    Article  Google Scholar 

  • Gildea, D., Jurafsky, D.: Automatic labeling of semantic roles. Computational Linguistics 28(3), 245–288 (2002)

    Article  Google Scholar 

  • Glickman, O., Dagan, I.: Identifying lexical paraphrases from a single corpus: A case study for verbs. In: Proc. of RANLP (2003)

    Google Scholar 

  • Glickman, O., Dagan, I., Koppel, M.: Web based probabilistic textual entailment. In: Proc. of the PASCAL RTE challenge workshop (2005)

    Google Scholar 

  • Graesser, A.C., Hu, X., Susarla, S., Harter, D., Person, N.K., Louwerse, M., Olde, B.: The Tutoring Research Group AutoTutor: An intelligent tutor and conversational tutoring scaffold. In: Proc. of the 10th International Conference of Artificial Intelligence in Education, San Antonio, TX, pp. 47–49 (2001)

    Google Scholar 

  • Haghighi, A.D., Ng, A.Y., Manning, C.D.: Robust Textual Inference via Graph Matching. In: Proc. HLT-EMNLP (2005)

    Google Scholar 

  • Herrera, J., Peñas, A., Verdejo, F.: Textual entailment recognition based on dependency analysis and WordNet. In: Proc. of PASCAL Recognizing Textual Entailment challenge workshop (2005)

    Google Scholar 

  • Hickl, A., Bensley, J.: A discourse commitment-based framework for recognizing textual entailment. In: Proc. of the ACL-PASCAL workshop on textual entailment and paraphrasing (2007)

    Google Scholar 

  • Hickl, A., Bensley, J., Williams, J., Roberts, K., Rink, B., Shi, Y.: Recognizing textual entailment with LCC’s GROUNDHOG system. In: Proc. of the Second PASCAL Recognizing Textual Entailment challenge workshop (2006)

    Google Scholar 

  • Jiang, J.J., Conrath, D.W.: Semantic similarity based on corpus statistics and lexical taxonomy. In: Proc. of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pp. 19–33 (1997)

    Google Scholar 

  • Jordan, P.W., Makatchev, M., VanLehn, K.: Combining competing language understanding approaches in an intelligent tutoring system. In: Lester, J.C., Vicari, R.M., Paraguacu, F. (eds.) 7th Conference on Intelligent Tutoring Systems, pp. 346–357. Springer, Heidelberg (2004)

    Google Scholar 

  • Kintsch, W.: Comprehension: A paradigm for cognition. Cambridge University Press, Cambridge (1998)

    Google Scholar 

  • Kipper, K., Dang, H.T., Palmer, M.: Class-based construction of a verb lexicon. In: AAAI seventeenth National Conference on Artificial Intelligence (2000)

    Google Scholar 

  • Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. International J of AI in Education 8, 30–43 (1997)

    Google Scholar 

  • Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. J Psychological Review (1997)

    Google Scholar 

  • Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis. Discourse Processes 25, 259–284 (1998)

    Article  Google Scholar 

  • Leacock, C.: Scoring free-response automatically: A case study of a large-scale Assessment. Examens 1(3) (2004)

    Google Scholar 

  • Leacock, C., Chodorow, M.: C-rater: Automated scoring of short-answer questions. Computers and the Humanities 37, 4 (2003)

    Article  Google Scholar 

  • Lin, D.: Automatic retrieval and clustering of similar words. In: Proc. of the ACL, Montreal, pp. 898–904 (1998)

    Google Scholar 

  • Lin, D., Pantel, P.: DIRT – Discovery of inference rules from text. In: Proc. of KDD (2001a)

    Google Scholar 

  • Lin, D., Pantel, P.: Discovery of inference rules for question answering. Natural Language Engineering 7(4), 343–360 (2001b)

    Article  Google Scholar 

  • MacCartney, B., Grenager, T., de Marneffe, M., Cer, D., Manning, C.: Learning to recognize features of valid textual entailments. In: Proc. of HLT-NAACL (2006)

    Google Scholar 

  • Madden, N.A., Slavin, R.E.: Effective pullout programs for students at risk. In: Slavin, R.E., Karweit, N.L., Madden, N.A. (eds.) Effective programs for students at risk, Allyn and Bacon, Boston (1989)

    Google Scholar 

  • Makatchev, M., Jordan, P., VanLehn, K.: Abductive theorem proving for analyzing student explanations and guiding feedback in intelligent tutoring systems. Journal of Automated Reasoning special issue on automated reasoning and theorem proving in education 32(3), 187–226 (2004)

    Google Scholar 

  • Mathews, E.C., Jackson, G.T., Person, N.K., Graesser, A.C.: Discourse patterns in Why/AutoTutor. In: Proceedings of the 2003 AAAI Spring Symposia on Natural Language Generation, pp. 45–51. AAAI Press, Menlo Park, Palo Alto, CA (2003)

    Google Scholar 

  • Meichenbaum, D., Biemiller, A.: Nurturing independent learners: Helping students take charge of their learning. Brookline. Brookline, Cambridge (1998)

    Google Scholar 

  • Mitchell, T., Aldridge, N., Broomhead, P.: Computerized marking of short-answer free-text responses. In: 29th annual conference of the International Association for Educational Assessment, Manchester, UK (2003)

    Google Scholar 

  • Mitchell, T., Russell, T., Broomhead, P., Aldridge, N.: Towards robust computerized marking of free-text responses. In: Proc. of 6th International Computer Aided Assessment Conference, Loughborough (2002)

    Google Scholar 

  • Mostow, J., Aist, G., Burkhead, P., Corbett, A., Cuneo, A., Eitelman, S., Huang, C., Junker, B., Sklar, M.B., Tobin, B.: Evaluation of an automated reading tutor that listens: Comparison to human tutoring and classroom instruction. Journal of Educational Computing Research 29(1), 61–117 (2003)

    Article  Google Scholar 

  • NAEP (2005), http://nces.ed.gov/nationsreportcard/

  • Nielsen, R.D.: MOB-ESP and other improvements in probability estimation. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (2004)

    Google Scholar 

  • Nielsen, R.D., Pradhan, S.: Mixing weak learners in semantic parsing. In: Proc. of EMNLP, Barcelona, Spain (2004)

    Google Scholar 

  • Nielsen, R.D., Ward, W., Martin, J.H.: Toward dependency path based entailment. In: Proc. of the second PASCAL RTE challenge workshop (2006)

    Google Scholar 

  • Nielsen, R.D., Ward, W.: A corpus of fine-grained entailment relations. In: Proc. of the ACL workshop on Textual Entailment and Paraphrasing (2007)

    Google Scholar 

  • Nivre J., Kubler S.: Dependency parsing. Tutorial at COLING-ACL, Sydney (2006)

    Google Scholar 

  • Nivre, J., Hall, J., Nilsson, J., Chanev, A., Eryigit, G., Kübler, S., Marinov, S., Marsi, E.: MaltParser: A language-independent system for data-driven dependency parsing. Natural Language Engineering 13(2), 95–135 (2007)

    Google Scholar 

  • Pang, B., Knight, K., Marcu, D.: Syntax-based alignment of multiple translations: Extracting paraphrases and generating sentences. In: Proc. HLT/NAACL (2003)

    Google Scholar 

  • Pellom, B.: SONIC: The University of Colorado continuous speech recognizer. University of Colorado, tech report #TR-CSLR-2001-01, Boulder (2001)

    Google Scholar 

  • Pellom, B., Hacioglu, K.: Recent improvements in the CU SONIC ASR system for noisy speech: The SPINE task. In: Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong (2003)

    Google Scholar 

  • Peters, S., Bratt, E.O., Clark, B., Pon-Barry, H., Schultz, K.: Intelligent systems for training damage control assistants. In: Proc. of Interservice/Industry Training, Simulation and Education Conference (2004)

    Google Scholar 

  • Platt, J.: Probabilities for support vector machines. In: Smola, A., Bartlett, P., Scolkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers. MIT Press, Cambridge (2000)

    Google Scholar 

  • Pon-Barry, H., Clark, B., Schultz, K., Bratt, E.O., Peters, S.: Contextualizing learning in a reflective conversational tutor. In: Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies (2004)

    Google Scholar 

  • Pradhan, S., Ward, W., Hacioglu, K., Martin, J.H., Jurafsky, D.: Semantic role labeling using different syntactic views. In: Proceedings of ACL (2005)

    Google Scholar 

  • Pulman, S.G., Sukkarieh, J.Z.: Automatic short answer marking. In: Proc. of the 2nd Workshop on Building Educational Applications Using NLP, ACL (2005)

    Google Scholar 

  • Raina, R., Haghighi, A., Cox, C., Finkel, J., Michels, J., Toutanova, K., MacCartney, B., de Marneffe, M.C., Manning, C.D., Ng, A.Y.: Robust textual inference using diverse knowledge sources. In: Proc of the PASCAL RTE challenge workshop (2005)

    Google Scholar 

  • Ravichandran, D., Hovy, E.: Learning surface text patterns for a question answering system. In: Proc. of the 40th ACL conference, Philadelphia (2002)

    Google Scholar 

  • Roll, I., Baker, R.S., Aleven, V., McLaren, B.M., Koedinger, K.R.: Modeling students’ metacognitive errors in two intelligent tutoring systems. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) User Modeling, pp. 379–388 (2005)

    Google Scholar 

  • Rosé, C.P., Roque, A., Bhembe, D., VanLehn, K.: A hybrid text classification approach for analysis of student essays. In: Building Educational Applications Using Natural Language Processing, pp. 68–75 (2003a)

    Google Scholar 

  • Rosé, C.P., Gaydos, A., Hall, B.S., Roque, A., VanLehn, K.: Overcoming the knowledge engineering bottleneck for understanding student language input. In: Proceedings of AI in Education. IOS Press, Amsterdam (2003b)

    Google Scholar 

  • Shinyama, Y., Sekine, S.: Paraphrase acquisition for information extraction. In: Proc. of the Second International Workshop on Paraphrasing, Sapporo, Japan (2003)

    Google Scholar 

  • Shinyama, Y., Sekine, S., Sudo, K., Grishman, R.: Automatic paraphrase acquisition from news articles. In: Proc. of HLT, San Diego, CA (2002)

    Google Scholar 

  • Sudo, K., Sekine, S., Grishman, R.: Automatic pattern acquisition for japanese information extraction. In: Proc. of HLT, San Diego, CA (2001)

    Google Scholar 

  • Sukkarieh, J.Z., Pulman, S.G., Raikes, N.: Auto-marking: Using computational linguistics to score short, free text responses. In: Proc. of the 29th conference of the International Association for Educational Assessment, Manchester, UK (2003)

    Google Scholar 

  • Sukkarieh, J.Z., Pulman, S.G.: Information extraction and machine learning: Auto-marking short free text responses to science questions. In: Proc. of AIED (2005)

    Google Scholar 

  • Sweet, A.P., Snow, C.E. (eds.): Rethinking reading comprehension. Guilford Press, New York (2003)

    Google Scholar 

  • Tatu, M., Moldovan, D.: COGEX at RTE 3. In: Proc. of the ACL-PASCAL workshop on Textual Entailment and Paraphrasing (2007)

    Google Scholar 

  • Topping, K., Whitley, M.: Participant evaluation of parent-tutored and peer-tutored projects in reading. Educational Research 32(1), 14–32 (1990)

    Google Scholar 

  • Turney, P.D.: Mining the web for synonyms: PMI-IR versus LSA on TOEFL. In: Proc. of 12th European Conference on Machine Learning, pp. 491–502 (2001)

    Google Scholar 

  • Turney, P.D., Littman, M.L., Bigham, J., Shnayder, V.: Combining independent modules to solve multiple-choice synonym and analogy problems. In: Proceedings of RANLP, Borovets, Bulgaria, pp. 482–489 (2003)

    Google Scholar 

  • VanLehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The Andes physics tutoring system: Five years of evaluations. In: McCalla, G., Looi, C.K. (eds.) Proc. of the 12th International Conference on AI in Education. IOS Press, Amsterdam (2005)

    Google Scholar 

  • VanLehn, K., Siler, S., Murray, C., Yamauchi, T., Baggett, W.B.: Why do only some events cause learning during human tutoring. In: Cognition and Instruction, vol. 21(3), pp. 209–249. Lawrence Erlbaum Associates (2003)

    Google Scholar 

  • Voorhees, E.M., Harman, D.K.: Overview of the 6th text retrieval conference (TREC6). In: Proc. of 17th Text Retrieval Conference. NIST (1998)

    Google Scholar 

  • Ward, W.H.: The Phoenix system: Understanding spontaneous speech. In: Proc. of IEEE ICASSP (1991)

    Google Scholar 

  • Witten, I.H., Frank, E.: Data mining: Practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Spoken Language Research (2 Director),  

    Rodney D. Nielsen, Wayne Ward & James H. Martin

  2. Institute of Cognitive Science,  

    Rodney D. Nielsen, Wayne Ward & James H. Martin

  3. Department of Computer Science,  

    Rodney D. Nielsen, Wayne Ward & James H. Martin

  4. Department of Linguistics, University of Colorado, Boulder,  

    James H. Martin

Authors
  1. Rodney D. Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wayne Ward
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James H. Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Bhanu Prasad

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Nielsen, R.D., Ward, W., Martin, J.H. (2008). Soft Computing in Intelligent Tutoring Systems and Educational Assessment. In: Prasad, B. (eds) Soft Computing Applications in Business. Studies in Fuzziness and Soft Computing, vol 230. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79005-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-79005-1_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-79004-4

  • Online ISBN: 978-3-540-79005-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation