Skip to main content
SpringerLink
Log in

Effects of a Teacher Dashboard for an Intelligent Tutoring System on Teacher Knowledge, Lesson Planning, Lessons and Student Learning

  • Conference paper
  • First Online:
Book cover Data Driven Approaches in Digital Education (EC-TEL 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10474))

Included in the following conference series:

Abstract

Intelligent Tutoring Systems (ITSs) help students learn but often are not designed to support teachers and their practices. A dashboard with analytics about students’ learning processes might help in this regard. However, little research has investigated how dashboards influence teacher practices in the classroom and whether they can help improve student learning. In this paper, we explore how Luna, a dashboard prototype designed for an ITS and used with real data, affects teachers and students. Results from a quasi-experimental classroom study with 5 middle school teachers and 17 classes show that Luna influences what teachers know about their students’ learning in the ITS and that the teachers’ updated knowledge affects the lesson plan they prepare, which in turn guides what they cover in a class session. Results did not confirm that Luna increased student learning. In summary, even though teachers generally know their classes well, a dashboard with analytics from an ITS can still enhance their knowledge about their students and support their classroom practices. The teachers tended to focus primarily on dashboard information about the challenges their students were experiencing. To the best of our knowledge, this is the first study that demonstrates that a dashboard for an ITS can affect teacher knowledge, decision-making and actions in the classroom.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aleven, V., McLaren, B.M., Sewall, J., van Velsen, M., et al.: Example-tracing tutors: intelligent tutor development for non-programmers. Int. J. Artif. Intell. Educ. 26, 224–269 (2016)

    Article  Google Scholar 

  2. Aleven, V., Xhakaj, F., Holstein, K., McLaren, B.M.: Developing a teacher dashboard for use with intelligent tutoring systems. In: The Proceedings of the 4th International Workshop on Teaching Analytics, IWTA 2016 at the 11th European Conference On Technology Enhanced Learning, EC-TEL 2016, 13–16 September 2016, Lyon, France (2016)

    Google Scholar 

  3. Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)

    Article  Google Scholar 

  4. Arroyo, I., Woolf, B.P., Burleson, W., Muldner, K., Rai, D., Tai, M.: A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. Int. J. Artif. Intell. Educ. 24(4), 387–426 (2014)

    Article  Google Scholar 

  5. Hanington, B., Martin, B.: Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Rockport Publishers, Beverly (2012)

    Google Scholar 

  6. Holstein, K., Xhakaj, F., Aleven, V., McLaren, B.M.: Luna: A Dashboard for Teachers Using Intelligent Tutoring Systems. In: Proceedings of the 4th International Workshop on Teaching Analytics, IWTA 2016 at the 11th European Conference On Technology Enhanced Learning, EC-TEL 2016, 13–16 September 2016, Lyon, France (2016)

    Google Scholar 

  7. Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., Goldstein, D.S.: Estimating the Effect of Web-Based Homework. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 824–827. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39112-5_122

    Chapter  Google Scholar 

  8. Kulik, J.A., Fletcher, J.D.: Effectiveness of Intelligent Tutoring Systems: a meta-analytic review. Rev. Educ. Res. 86(1), 42–78 (2016)

    Article  Google Scholar 

  9. Long, Y., Aleven, V.: Mastery-oriented shared student/system control over problem selection in a linear equation tutor. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 90–100. Springer, Cham (2016). doi:10.1007/978-3-319-39583-8_9

    Google Scholar 

  10. Lovett, M., Meyer, O., Thille, C.: The open learning initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. J. Interact. Media Edu. 1, 1–16 (2008). doi:10.5334/2008-14

    Google Scholar 

  11. Ma, W., Adesope, O.O., Nesbit, J.C., Liu, Q.: Intelligent Tutoring Systems and learning outcomes: A meta-analysis. J. Educ. Psychol. 106(4), 901 (2014)

    Article  Google Scholar 

  12. Martinez-Maldonado, R., Pardo, A., Mirriahi, N., Yacef, K., Kay, J., Clayphan, A.: The LATUX workflow: designing and deploying awareness tools in technology-enabled learning settings. In: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, LAK 2015, pp. 1–10 (2015)

    Google Scholar 

  13. Martinez Maldonado, R., Kay, J., Yacef, K., Schwendimann, B.: An interactive teacher’s dashboard for monitoring groups in a multi-tabletop learning environment. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 482–492. Springer, Heidelberg (2012). doi:10.1007/978-3-642-30950-2_62

    Chapter  Google Scholar 

  14. Mavrikis, M., Gutierrez-Santos, S., Poulovassilis, A.: Design and evaluation of teacher assistance tools for exploratory learning environments. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, LAK 2016, pp. 168–172. ACM, New York (2016)

    Google Scholar 

  15. Mazza, R., Dimitrova, V.: CourseVis: a graphical student monitoring tool for supporting instructors in web-based distance courses. Int. J. Hum. Compu. Stud. 65(2), 125–139 (2007)

    Article  Google Scholar 

  16. McLaren, B.M., Scheuer, O., Miksatko, J.: Supporting collaborative learning and eDiscussions using artificial intelligence techniques. Int. J. Artif. Intell. Educ. 20(1), 1–46 (2010)

    Google Scholar 

  17. Raudenbush, S.W., Bryk, A.S.: Hierarchical Linear Models: Applications and Data Analysis Methods. Sage Publications, Newbury Park (2002)

    Google Scholar 

  18. Schwendimann, B.A., Rodríguez-Triana, M.J., Vozniuk, A., Prieto, L.P., Boroujeni, M.S., Holzer, A., Gillet, D., Dillenbourg, P.: Understanding learning at a glance: An overview of learning dashboard studies. In: Gasevic, D., Lynch, G., Dawson, S., Drachsler, H., Rose, C.P. (eds.), Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, LAK 2016, pp. 532–533. ACM, New York (2016)

    Google Scholar 

  19. Steenbergen-Hu, S., Cooper, H.: A meta-analysis of the effectiveness of Intelligent Tutoring Systems on college students’ academic learning. J. Educ. Psychol. 106(2), 331–347 (2014)

    Article  Google Scholar 

  20. van Leeuwen, A., Janssen, J., Erkens, G., Brekelmans, M.: Supporting teachers in guiding collaborating students: effects of learning analytics in CSCL. Comput. Educ. 79, 28–39 (2014)

    Article  Google Scholar 

  21. VanLehn, K.: The behavior of tutoring systems. Int. J. Artif. Intell. Educ. 16(3), 227–265 (2006)

    Google Scholar 

  22. Verbert, K., Govaerts, S., Duval, E., Santos, J.L., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Pers. Ubiquit. Comput. 18(6), 1499–1514 (2014)

    Google Scholar 

  23. Waalkens, M., Aleven, V., Taatgen, N.: Does supporting multiple student strategies lead to greater learning and motivation? Investigating a source of complexity in the architecture of Intelligent Tutoring Systems. Comput. Educ. 60, 159–171 (2013)

    Article  Google Scholar 

  24. Woolf, B.P.: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing E-learning. Morgan Kauffman, Burlington (2009)

    Google Scholar 

  25. Xhakaj, F., Aleven, V., McLaren, B.M.: How teachers use data to help students learn: contextual inquiry for the design of a dashboard. In: Verbert, K., Sharples, M., Klobučar, T. (eds.) EC-TEL 2016. LNCS, vol. 9891, pp. 340–354. Springer, Cham (2016). doi:10.1007/978-3-319-45153-4_26

    Chapter  Google Scholar 

Download references

Acknowledgments

We thank all the teachers, schools and students who took part in our study, Gail Kusbit, Kenneth Holstein, the coders and graders for the project. This work is supported by NSF Award # 1530726.

Author information

Authors and Affiliations

  1. Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA, USA

    Françeska Xhakaj, Vincent Aleven & Bruce M. McLaren

Authors
  1. Françeska Xhakaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent Aleven
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruce M. McLaren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Françeska Xhakaj , Vincent Aleven or Bruce M. McLaren .

Editor information

Editors and Affiliations

  1. Jean Moulin Lyon 3 University, Lyon, France

    Élise Lavoué

  2. German Institute for International Educational Research, Goethe University, Frankfurt, Germany

    Hendrik Drachsler

  3. KU Leuven, Leuven, Belgium

    Katrien Verbert

  4. University of Toulouse, Toulouse, France

    Julien Broisin

  5. Pontificia Universidad, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Xhakaj, F., Aleven, V., McLaren, B.M. (2017). Effects of a Teacher Dashboard for an Intelligent Tutoring System on Teacher Knowledge, Lesson Planning, Lessons and Student Learning. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds) Data Driven Approaches in Digital Education. EC-TEL 2017. Lecture Notes in Computer Science(), vol 10474. Springer, Cham. https://doi.org/10.1007/978-3-319-66610-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66610-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66609-9

  • Online ISBN: 978-3-319-66610-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation