Evaluation of Guided-Planning and Assisted-Coding with Task Relevant Dynamic Hinting

  • Wei Jin
  • Albert Corbett
  • Will Lloyd
  • Lewis Baumstark
  • Christine Rolka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8474)


We describe a programming tutor framework that consists of two configurable components, a guided-planning component and an assisted-coding component that offers task relevant automatically-generated hints on demand to students. We evaluate the effectiveness of the new integrated planning and coding environment by comparing it to three other tutor conditions: planning-only, coding-only, and planning-only interleaved with planning-coding. We conclude that the integrated planning and coding tutor environment is more effective than tutored planning-only activities and that students make more efficient use of tutor feedback in the integrated environment than in the coding only environment.


Intelligent tutoring systems automatic hint generation programming tutors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aleven, V., McLaren, B.M., Sewall, J., Koedinger, K.R.: Example-Tracing Tutors: A New Paradigm for Intelligent Tutoring Systems. International Journal of Artificial Intelligence in Education 19, 105–154 (2009)Google Scholar
  2. 2.
    Anderson, J.R., Conrad, F.G., Corbett, A.T.: Skill Acquisition and the LISP Tutor. Cognitive Science 13(4), 467–505 (1989)CrossRefGoogle Scholar
  3. 3.
    Barnes, T., Stamper, J.: Automatic hint generation for logic proof tutoring using historical data. Journal Educational Technology & Society 13(1), 3–12 (2010); Special issue on Intelligent Tutoring SystemsGoogle Scholar
  4. 4.
    Bloom, B.S.: The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13, 4–16 (1984)CrossRefGoogle Scholar
  5. 5.
    Bonar, J., Cunningham, R.: BRIDGE: An intelligent tutor for thinking about programming. In: Self, J. (ed.) Artificial Intelligence and Human Learning, ch. 24, pp. 391–409. Chapman and Hall (1988)Google Scholar
  6. 6.
    Coheen, J., Chen, L.Y.: Migrating out of computer science. Computing Research News 15(2) (2003)Google Scholar
  7. 7.
    Corbett, A.T., Anderson, J.R.: Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In: ACM Conference on Human Factors in Computing Systems, pp. 245–252 (2001)Google Scholar
  8. 8.
    Huang, J., Piech, C., Nguyen, A., Guibas, L.: Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC Stanford University. In: 1st Workshop on Massive Open Online Courses at the 16th Annual Conference on Artificial Intelligence in Education (2013)Google Scholar
  9. 9.
    Jin, W., Barnes, T., Stamper, J., Eagle, M.J., Johnson, M.W., Lehmann, L.: Program Representation for Automatic Hint Generation for a Data-Driven Novice Programming Tutor. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 304–309. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Jin, W., Corbett, A.: Effectiveness of Cognitive Apprenticeship Learning (CAL) and Cognitive Tutors (CT) for Problem Solving Using Fundamental Programming Concepts. In: 42nd ACM SIGCSE Technical Symposium on Computer Science Education, pp. 305–310 (2011)Google Scholar
  11. 11.
    Lane, H.C., VanLehn, K.: Teaching program planning skills to novices with natural language tutoring. Computer Science Education 15(3), 183–201 (2005)CrossRefGoogle Scholar
  12. 12.
    McCracken, M., et al.: A multi-national multi-institutional study of assessment of programming skills of first-year CS students. SIGCSE Bulletin 34(11) (March 2002)Google Scholar
  13. 13.
    Mitrovic, A., Ohlsson, S.: Evaluation of a Constraint-Based Tutor for a Database Language. International Journal of AI in Education 10(3-4), 238–256 (1999)Google Scholar
  14. 14.
    Pirolli, P.: A cognitive model and computer tutor for programming recursion. Human Computer Interaction 2, 319–355 (1986)CrossRefGoogle Scholar
  15. 15.
    Rivers, K., Koedinger, K.R.: Automatic Generation of Programming Feedback: A Data-Driven Approach. In: The Workshops at the 16th International Conference on Artificial Intelligence in Education, pp. 50–59 (2013)Google Scholar
  16. 16.
    Stamper, J., Barnes, T., Croy, M.: Enhancing the automatic generation of hints with expert seeding. Intl Journal of AI in Education 21(1), 153–167 (2011)Google Scholar
  17. 17.
    VanLehn, K.: The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist 46(4), 197–221 (2011)CrossRefGoogle Scholar
  18. 18.
    Weber, G., Brusilovsky, P.: ELM-ART: An adaptive versatile system for Web-based instruction. International Journal of Artificial Intelligence in Education 12(4), 351–384 (2001); Special Issue on Adaptive and Intelligent Web-based Educational SystemsGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Wei Jin
    • 1
  • Albert Corbett
    • 2
  • Will Lloyd
    • 1
  • Lewis Baumstark
    • 1
  • Christine Rolka
    • 1
  1. 1.University of West GeorgiaCarrolltonUSA
  2. 2.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations