Skip to main content
SpringerLink
Log in

User-adaptive explanatory program visualization: evaluation and insights from eye movements

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

User-adaptive visualization and explanatory visualization have been suggested to increase educational effectiveness of program visualization. This paper presents an attempt to assess the value of these two approaches. The results of a controlled experiment indicate that explanatory visualization allows students to substantially increase the understanding of a new programming topic. Furthermore, an educational application that features explanatory visualization and employs a user model to track users’ progress allows students to interact with a larger amount of material than an application which does not follow users’ activity. However, no support for the difference in short-term knowledge gain between the two applications is found. Nevertheless, students admit that they prefer the version that estimates and visualizes their progress and adapts the learning content to their level of understanding. They also use the application’s estimation to pace their work. The differences in eye movement patterns between the applications employing adaptive and non-adaptive explanatory visualizations are investigated as well. Gaze-based measures show that adaptive visualization captivates attention more than its non-personalized counterpart and is more interesting to students. Natural language explanations also accumulate a big portion of students’ attention. Furthermore, the results indicate that working memory span can mediate the perception of adaptation. It is possible that user-adaptation in an educational context provides a different service to people with different mental processing capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bednarik, R., Myller, N., Sutinen, E., Tukianinen, M.T.: Effects of experience on gaze behavior during program animation. In: Proceedings of the 17th Annual Psychology of Programming Interest Group Workshop (PPIG), pp. 49–61 (2005)

  • Blumenktants, M., Starovisky, H., Shamir, A.: Narrative algorithm animation. Proceedings of ACM Symposium on Software Visualization, pp. 17–26 (2006)

  • Boud D.: Enhancing Learning Through Self Assessment. Routledge, New York, NY (1995)

    Google Scholar 

  • Boyle C., Encarnacion A.O.: Metadoc: an adaptive hypertext reading system. User Model. User-Adapt. Interact. 4(1), 1–19 (1994)

    Article  Google Scholar 

  • Brusilovsky, P.: Program visualization as a debugging tool for novices. In: Proceedings of the 5th International Conference on Human-Computer Interaction (INTERCHI; Adjunct Proceedings), pp. 29–30 (1993)

  • Brusilovsky, P.: Explanatory visualization in an educational programming environment: connecting examples with general knowledge. In: Proceedings of the 4th International Conference on Human-Computer Interaction, pp. 202–212 (1994)

  • Brusilovsky, P., Loboda, T.D.: WADEIn II: a case for adaptive explanatory visualization. In: Proceedings of the 10th Conference on Innovation Technology in Computer Science Education (ITiCSE), pp. 48–52 (2006)

  • Brusilovsky, P., Spring, M.: Adaptive, engaging, and explanatory visualization in a C programming course. In: Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1264–1271 (2004)

  • Brusilovsky, P., Su, H.-D.: Adaptive visualization component of a distributed Web-based adaptive educational system. In: Proceedings of the 6th International Conference on Intelligent Tutoring Systems (ITS), pp. 229–238 (2002)

  • Byrne M.D., Catarambone R., Stasko J.T.: Evaluating animations as student aids in learning computer algorithms. Comput. Educ. 33(5), 253–278 (1999)

    Article  Google Scholar 

  • Case R., Kurland M.D., Goldberg J.: Operational efficiency and the growth of short-term memory span. J. Exp. Child Psychol. 33, 386–404 (1982)

    Article  Google Scholar 

  • Conati C., Merten C.: Eye-tracking for user modeling in exploratory learning environments: an empirical evaluation. Knowl.-Based Syst. 20(6), 557–574 (2007)

    Article  Google Scholar 

  • Daily L.Z., Lovett M.C., Reder L.M.: Modeling individual differences in working memory performance: A source activation account. Cogn. Sci. Multidiscipl. J. 25(3), 315–353 (2001)

    Article  Google Scholar 

  • Dancik, G., Kumar, A.N.: A tutor for counter-controlled loop concepts and its evaluation. In: Proceedings of 2003 Frontiers in Education Conference, Session T3C (2003)

  • Daneman M., Carpenter P.A.: Individual differences in working memory and reading. J. Verbal Learn. Verbal Behav. 19(4), 450–466 (1980)

    Article  Google Scholar 

  • Dmitrienko A., Molenbergs G., Chuang-Stein C., Offen W.: Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Publishing, Cary (2005)

    Google Scholar 

  • Graf, W., Krueger, H.: Ergonomic evaluation of user interfaces by means of eye movement data. In: Proceedings of the 3rd World Conference on Educational Multimedia, Hypermedia and Telecommunications Conference on Human-Computer Interaction (HCI), pp. 659–665 (1989)

  • Henderson J.M., Pierce G.L.: Eye movements during scene viewing: Evidence for mixed control of fixation durations. Psychon. Bull. Rev. 15(3), 566–573 (2008)

    Article  Google Scholar 

  • Hundhausen C.D., Douglas S.A., Stasko J.T.: A meta-study of algorithm visualization effectiveness. J. Vis. Lang. Comput. 13(3), 259–290 (2002)

    Article  Google Scholar 

  • Hooge I.Th.C., Vlaskamp B.N.S., Over E.A.B.: Saccadic search: on duration of a fixation. In: Gompel, R.P.G., Fischer, M.H., Murray, W.S., Hill, R.L. (eds) Eye Movements: A Window on Mind and Brain, pp. 581–596. Elsevier, Amsterdam, The Netherlands (2003)

    Google Scholar 

  • Jarc, D.J., Feldman, M.B., Heller, R.S.: Assessing the benefits of interactive prediction using Web-based algorithm animation courseware. In: Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, pp. 377–381 (2000)

  • Just M.A., Carpenter P.A.: Eye fixations and cognitive processes. Cogn. Psychol. 8, 441–480 (1976)

    Article  Google Scholar 

  • Kenward M.G., Roger J.H.: Small sample inference for fixed effects from restricted maximum likelihood. Biometrics 53, 983–997 (1997)

    Article  MATH  Google Scholar 

  • Kerren, A., Stasko, J.: Algorithm animation—introduction. In: Diehl, S. (ed.) Software Visualization State of the Art Survey, pp. 1–15. Springer, (2002)

  • Kerren, A., Mueldner, T., Shakshuki, E.: Novel algorithm explanation techniques for improving algorithm teaching. In: Proceedings of the ACM Symposium on Software Visualization (SoftVis), pp. 175–176 (2006)

  • Krebs, M., Lauer, T., Ottmann, T., Trahasch, S.: Student-built algorithm visualizations for assessment: flexible generation, feedback and grading. In: Proceedings of the 9th Conference on Innovation Technology in Computer Science Education (ITiCSE), pp. 281–285 (2005)

  • Kobsa A., Koenemann J., Pohl W.: Personalised hypermedia presentation techniques for improving online customer relationships. Knowl. Eng. Rev. 16(2), 111–155 (2001)

    Article  MATH  Google Scholar 

  • Kumar, A.N.: Model-based generation of demand feedback in a programming tutor. In: Proceedings of the 11th International Conference on Artificial Intelligence in Education (AIED), pp. 425–432 (2003)

  • Kumar, A.N.: Results from the evaluation of the effectiveness of an online tutor on expression evaluation. In: Proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education, pp. 216–220 (2005)

  • Lahtinen, E., Ahoniemi, T.: Annotations for defining interactive instructions to interpreter based program visualization tools. In: Rössling G. (ed.) Electronic Notes in Theoretical Computer Science, Vol. 178, pp. 121–128 (2007)

  • Littell R.C., Milliken G.A., Stroup W.W., Wolfinger R.D., Schabenberger O.: SAS for Mixed Models. 2nd edn. SAS Publishing, Cary (2006)

    Google Scholar 

  • Loboda, T.D., Brusilovsky, P.: Adaptation in the context of explanatory visualization. In: Proceedings of the 3rd European Conference on Technology Enhanced Education (ECTEL), pp. 250–261 (2008)

  • Loftus G.R.: Eye fixations on text and scenes. In: Rayner, K. (eds) Eye Movements in Reading: Perceptual and Language Processes, pp. 359–376. Academic Press, New York, NY (1983)

    Google Scholar 

  • Manor B.R., Gordon E.: Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks. J. Neurosci. Methods 128(1–2), 85–93 (2003)

    Article  Google Scholar 

  • McCullagh P., Nelder J.A.: Generalized Linear Models. CRC Press, Boca Raton, FL (1989)

    MATH  Google Scholar 

  • Moreno, A., Myller, N., Sutinen, E., Ari, M.B.: Visualizing programs with Jeliot 3. In: Proceedings of the Working Conference on Advanced Visual Interfaces (AVI), pp. 373–376 (2004)

  • Naps, T.L., Eagan, J.R., Norton, L.L.: JHAVE—an environment to actively engage students in Web-based algorithm visualizations. In: Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, pp. 109–113 (2000)

  • Naps T.L., Rössling G., Almstrum V., Dann W., Fleischer R., Hundhausen C., Korhonen A., Malmi L., McNally M., Rodger S., Velázquez-Iturbide J.Á.: Exploring the role of visualization and engagement in computer science education. ACM SIGCSE Bull. 35, 131–152 (2002)

    Article  Google Scholar 

  • Naps T.L., Rössling G., Anderson J., Cooper S., Dann W., Fleischer R., Koldehofe B., Korhonen A., Kuittinen M., Leska C., McNally M., Malmi L., Rantakokko J., Ross R.J.: Evaluating the educational impact of visualization. ACM SIGCSE Bull. 35(4), 124–136 (2003)

    Article  Google Scholar 

  • Nevalainen, S., Sajaniemi, J.: An experiment on short-term effects of animated versus static visualization of operations on program perception. In: Proceedings of the 2nd International Workshop on Computing Education Research, pp. 7–16 (2006)

  • O’Regan J.K.: Optimal viewing position in words and the strategy-tactics theory of eye movements in reading. In: Rayner, K. (eds) Eye Movements and Visual Cognition: Scene Perception and Reading., pp. 333–354. Springer-Verlag, New York, NY (1992)

    Google Scholar 

  • Paas F., Renkl A., Sweller J.: Cognitive load theory and instructional design: Recent developments. Educ. Psychol. 38, 1–4 (2003)

    Article  Google Scholar 

  • Price B.: A principled taxonomy of software visualization. J. Vis. Lang. Comput. 4(3), 211–266 (1993)

    Article  Google Scholar 

  • R Development Core Team: R: A Language and Environment for Statistical (2009)

  • Rayner K.: Eye movement in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372–422 (1998)

    Article  Google Scholar 

  • SAS Institute Inc.: SAS 9.2 help and documentation (2008)

  • Salthouse T.A., Ellis C.L.: Determinants of eye-fixation duration. Am. J. Psychol. 93(2), 207–234 (1980)

    Article  Google Scholar 

  • Stasko, J., Badre, A., Lewis, C.: Do algorithm animations assist learning? An empirical study and analysis. In: Proceedings of the 5th International Conference on Human-Computer Interaction (INTERCHI), pp. 61–66 (1993)

  • Sweller J.: Cognitive load during problem-solving: Effects on learning. Cogn. Sci. 12, 257–285 (1988)

    Article  Google Scholar 

  • Turner M.L., Engle R.W.: Is working memory capacity task dependent?. J. Mem. Lang. 28, 127–154 (1989)

    Article  Google Scholar 

  • Velichkovsky B.M., Rothert A., Kopf M., Dornhoefer S.M., Joos M.: Towards an express diagnostics for level of processing and hazard perception. Transp. Res. F 5(2), 145–156 (2002)

    Google Scholar 

  • Velichkovsky, B.M., Joos, M., Helmert, J.R., Pannasch, S.: Two visual systems and their eye movements: evidence from static and dynamic scene perception. In: Proceedings of the 27th Conference of the Cognitive Science Society, pp. 2283–2288 (2005)

  • Weber G., Brusilovsky P.: ELM-ART: an adaptive versatile system for Web-based instruction. Int. J. Artif. Intell. Educ. 12(4), 351–384 (2001)

    Google Scholar 

  • Yamamoto, Y., Hirose, H.: Result of applying study support system that flow chart diagram displays by synchronizing with source program code to education. In: Proceedings of the 10th World Conference on E-Learning (E-LEARN), pp. 1186–1192 (2005)

Download references

Author information

Authors and Affiliations

  1. School of Information Sciences, University of Pittsburgh, Pittsburgh, PA, 15260, USA

    Tomasz D. Loboda & Peter Brusilovsky

Authors
  1. Tomasz D. Loboda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Brusilovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz D. Loboda.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Loboda, T.D., Brusilovsky, P. User-adaptive explanatory program visualization: evaluation and insights from eye movements. User Model User-Adap Inter 20, 191–226 (2010). https://doi.org/10.1007/s11257-010-9077-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-010-9077-1

Keywords

Search

Navigation