Skip to main content
SpringerLink
Log in
Book cover

State-of-the-Art and Future Directions of Smart Learning pp 231–241Cite as

Toward Non-intrusive Assessment in Dialogue-Based Intelligent Tutoring Systems

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Educational Technology ((LNET))

Abstract

This paper describes a study whose goal was to assess students’ prior knowledge level with respect to a target domain based solely on characteristics of the natural language interaction between students and a state-of-the-art conversational ITS. We report results on data collected from two conversational ITSs: a micro-adaptive-only ITS and a fully adaptive (micro- and macro-adaptive) ITS. Our models rely on both dialogue and session interaction features including time-on-task, student-generated content features (e.g., vocabulary size or domain-specific concept use), and pedagogy-related features (e.g., level of scaffolding measured as number of hints). Linear regression models were explored based on these features in order to predict students’ knowledge level, as measured with a multiple-choice pre-test, and yielded in the best cases an r = 0.949 and adjusted r-square = 0.878.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Chi, M. T. H., Siler, S. A., & Jeong, H. (2004). Can tutors monitor students’ understanding accurately? Cognition and Instruction, 22(3), 363–387.

    Google Scholar 

  2. Chi, M. T. H., Siler, S., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533.

    Article  Google Scholar 

  3. Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence-based approach to reform. Consortium for Policy Research in Education Report #RR-63. Philadelphia, PA: Consortium for Policy Research in Education.

    Google Scholar 

  4. Forbes-Riley, K., & Litman, D. J. (2006). Modelling user satisfaction and student learning in a spoken dialogue tutoring system with generic, tutoring, and user affect parameters. In Proceedings of Human Language Technology Conference.

    Google Scholar 

  5. Lintean, M., Rus, V., & Azevedo, R. (2011). Automatic detection of student mental models during prior knowledge activation in metatutor. International Journal of Artificial Intelligence in Education, 21(3), 169–190.

    Google Scholar 

  6. Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates.

    Google Scholar 

  7. Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers and Education, 68, 458–472.

    Article  Google Scholar 

  8. Rus, V., D’Mello, S., Hu, X., & Graesser, A. C. (2013). Recent advances in conversational intelligent tutoring systems. AI Magazine, 34(3). Lang. Syst. 15, 5 (Nov. 1993), 795–825.

    Google Scholar 

  9. Rus, V., Stefanescu, D., Baggett, W., Niraula, N., Franceschetti, D., & Graesser, A. C. (2014). Macro-adaptation in Conversational Intelligent Tutoring Matters, The 12th International Conference on Intelligent Tutoring Systems, June 5–9, Honolulu, Hawaii.

    Google Scholar 

  10. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

    Google Scholar 

  11. Shute, V. J., & Ventura, M. (2013). Measuring and Supporting Learning in Games: Stealth Assessment. Cambridge, MA: The MIT Press.

    Google Scholar 

  12. Taraban, R., & Rynearson, K. (1998). Computer-based comprehension research in a content area. Journal of Developmental Education, 21, 10–18.

    Google Scholar 

  13. VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rose, C. P. (2007). When are tutorial dialogues more effective than reading? Cognitive Science, 31, 3–62.

    Article  Google Scholar 

  14. Williams, C., & D’Mello, S. (2010). Predicting student knowledge level from domain-independent function and content words. In Intelligent Tutoring Systems (pp. 62–71). Berlin: Springer.

    Google Scholar 

  15. Woolf, B. (2008). Building Intelligent Interactive Tutors, Student-Centered Strategies for Revolutionizing E-Learning. Elsevier & Morgan Kaufmann Publishers.

    Google Scholar 

  16. Yoo, J. -B., & Kim, J. (2012). Predicting learner’s project performance with dialogue features in online Q&A discussions. In Intelligent Tutoring Systems (pp. 570–575).

    Google Scholar 

Download references

Acknowledgments

This research was supported by the Institute for Education Sciences (IESs) under award R305A100875 to Dr. Vasile Rus. All opinions and findings presented here are solely the authors’.

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute for Intelligent Systems, The University of Memphis, Memphis, TN, 38152, USA

    Vasile Rus & Dan Stefanescu

Authors
  1. Vasile Rus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Stefanescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasile Rus .

Editor information

Editors and Affiliations

  1. Beijing Normal University, Beijing, China

    Yanyan Li

  2. Computing and Information Systems, Athabasca University, Edmonton, Alberta, Canada

    Maiga Chang

  3. RWTH Aachen University, Aachen, Germany

    Milos Kravcik

  4. University of Craiova, Craiova, Romania

    Elvira Popescu

  5. Beijing Normal University, Beijing, China

    Ronghuai Huang

  6. Athabasca University, Edmonton, Alberta, Canada

    Kinshuk

  7. National Sun Yat-sen University, Taipei, Taiwan

    Nian-Shing Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media Singapore

About this paper

Cite this paper

Rus, V., Stefanescu, D. (2016). Toward Non-intrusive Assessment in Dialogue-Based Intelligent Tutoring Systems. In: Li, Y., et al. State-of-the-Art and Future Directions of Smart Learning. Lecture Notes in Educational Technology. Springer, Singapore. https://doi.org/10.1007/978-981-287-868-7_26

Download citation

  • DOI: https://doi.org/10.1007/978-981-287-868-7_26

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-287-866-3

  • Online ISBN: 978-981-287-868-7

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation