Identifying Effective Moves in Tutoring: On the Refinement of Dialogue Act Annotation Schemes

  • Alexandria Katarina Vail
  • Kristy Elizabeth Boyer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8474)


The rich natural language dialogue that is exchanged between tutors and students has inspired many successful lines of research on tutorial dialogue systems. Yet, today’s tutorial dialogue systems do not regularly achieve the same level of student learning gain as has been observed with expert human tutors. Implementing models directly informed by, and even machine-learned from, human-human tutorial dialogue is highly promising. With this goal in mind, this paper makes two contributions to tutorial dialogue systems research. First, it presents a dialogue act annotation scheme that is designed specifically to address a common weakness within dialogue act tag sets, namely, their dominance by a single large majority dialogue act class. Second, using this new fine-grained annotation scheme, the paper describes important correlations uncovered between tutor dialogue acts and student learning gain within a corpus of tutorial dialogue for introductory computer science. These findings can inform the design of future tutorial dialogue systems by suggesting ways in which systems can adapt at a fine-grained level to student actions.


Learning Gain Annotation Scheme Tutorial Dialogue Evaluative Question Dialogue Event 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    VanLehn, K., et al.: When Are Tutorial Dialogues More Effective Than Reading? Cog. Sci. 31(1), 3–62 (2007)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Bloom, B.S.: The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educ. Res. 13(6), 4–16 (1984)CrossRefGoogle Scholar
  3. 3.
    Chi, M.T., et al.: Learning from human tutoring. Cog. Sci. 25(4), 471–533 (2001)CrossRefGoogle Scholar
  4. 4.
    Lepper, M.R., et al.: Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. Computers as Cognitive Tools 1993, 75–105 (1999)Google Scholar
  5. 5.
    Graesser, A.C., et al.: Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cog. Psy. 9(6), 495–522 (1995)CrossRefGoogle Scholar
  6. 6.
    Chi, M., VanLehn, K., Litman, D.: Do Micro-Level Tutorial Decisions Matter: Applying Reinforcement Learning to Induce Pedagogical Tutorial Tactics. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 224–234. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Dzikovska, M.O., Steinhauser, N.B., Moore, J.D., Campbell, G.E., Harrison, K.M., Taylor, L.S.: Content, social, and metacognitive statements: An empirical study comparing human-human and human-computer tutorial dialogue. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 93–108. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Kumar, R., Ai, H., Beuth, J.L., Rosé, C.P.: Socially Capable Conversational Tutors Can Be Effective in Collaborative Learning Situations. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 156–164. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    D’Mello, S.K., et al.: A Motivationally Supportive Affect-Sensitive AutoTutor. New Perspectives on Affect and Learning Tech. 3, 113–126 (2011)CrossRefGoogle Scholar
  10. 10.
    Chen, L., et al.: Exploring Effective Dialogue Act Sequences in One-on-one Computer Science Tutoring Dialogues. In: Tetreault, J., et al. (eds.) Proc. 6th BEA Work., Portland, USA, pp. 65–75. Assoc. for Comp. Ling (2011)Google Scholar
  11. 11.
    Stellan, Ohlsson, o.: Beyond the Code-and-count Analysis of Tutoring Dialogues. In: R, Luckin, o. (eds.) Proc. 13th Int. Conf. AIED, Los Angeles, USA, vol. 158, pp. 349–356. IOS (2007)Google Scholar
  12. 12.
    Forbes-Riley, K., Litman, D.J.: Adapting to Student Uncertainty Improves Tutoring Dialogues. In: Vania, Dimitrova, o. (eds.) Proc. 14th Int. Conf. AIED, Brighton, United Kingdom, pp. 33–40. IOS (2009)Google Scholar
  13. 13.
    Cohen, P.A., et al.: Educational Outcomes of Tutoring: A Meta-analysis of Findings. Am. Educ. Res. J. 19(2), 237–248 (1982)CrossRefGoogle Scholar
  14. 14.
    D’Mello, S.K., et al.: Mining Collaborative Patterns in Tutorial Dialogues. J. EDM 2(1), 1–37 (2010)Google Scholar
  15. 15.
    Chu-Carroll, J.: A Statistical Model for Discourse Act Recognition in Dialogue Interactions. In: Chu-Carroll, J., Green, N. (eds.) AAAI Spring Symp.: Applying Machine Learning to Discourse Processing, Pan Alto, USA, vol. 1996, pp. 12–17. AAAI Press (1998)Google Scholar
  16. 16.
    Litman, D.J., Forbes-Riley, K.: Correlations between dialogue acts and learning in spoken tutoring dialogues. Nat. Lang. Eng. 12(2), 161–176 (2006)CrossRefGoogle Scholar
  17. 17.
    Mitchell, C.M., et al.: Recognizing Effective and Student-Adaptive Tutor Moves in Task-Oriented Tutorial Dialogue. In: Youngblood, M.G., McCarthy, P.M. (eds.) Proc. 25th Int. FLAIRS Conf., Marco Island, Florida, pp. 450–455. AAAI Press (2009)Google Scholar
  18. 18.
    Person, N.K., et al.: The Dialog Advancer Network: A Conversation Manager for AutoTutor. In: Gauthier, G., et al. (eds.) Proc. ITS Work. Modeling Human Teaching Tactics and Strategies, Montreal, Canada, pp. 86–92. Springer (2000)Google Scholar
  19. 19.
    Core, M.G., Allen, J.F.: Coding Dialogs with the DAMSL Annotation Scheme. In: Proc. 1997 AAAI Fall Symp.: Communicative Action in Humans and Machines, Providence, USA, pp. 28–35. AAAI (1997)Google Scholar
  20. 20.
    Landis, J.R., Koch, G.G.: The Measurement of Observer Agreement for Categorical Data Data for Categorical of Observer Agreement The Measurement. Biometrics 33(1), 159–174 (1977)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alexandria Katarina Vail
    • 1
  • Kristy Elizabeth Boyer
    • 1
  1. 1.Department of Computer ScienceNorth Carolina State UniversityRaleighUSA

Personalised recommendations