Engagement Analytics: A Microlevel Approach to Measure and Visualize Student Engagement

  • Isuru Balasooriya
  • Enric Mor
  • M. Elena Rodríguez
Chapter
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 11)

Abstract

Learner disengagement is a persisting issue in the Science Technology Engineering and Mathematics (STEM) subjects. Student engagement is dynamically constituted by the behavioural, cognitive and emotional dimensions of engagement in a learning environment. Although strongly linked with academic achievement, much of the details of engagement becomes lost in a retrospective measurement. Timely and microlevel data on the other hand has the ability to enrich the traditional learning analytics dataset. From a pilot study carried out at Universitat Oberta de Catalunya, where we have designed a self-reported data capture module that collects microlevel engagement data, initial results suggest the validity of the proposed approach and data. In this paper we emphasize how our approach enables better understanding of the student learning process and their characteristics such as cognitive patterns, emotional states and behaviours that leads to academic success and also enable richer feedback from teachers and informed decision making by the institution.

Keywords

Learning analytics Student engagement Self-reported student data Virtual learning environments Programming education 

References

  1. Ainley, M. (2012). Students interest and engagement in classroom activities’ in Handbook of research on student engagement (pp. 283–302). US: Springer.CrossRefGoogle Scholar
  2. Alexander, K. L., Entwisle, D. R., & Horsey, C. S. (1997). From first grade forward: Early foundations of high school dropout. Sociology of Education, 87–107.Google Scholar
  3. Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44(5), 427–445.CrossRefGoogle Scholar
  4. Archambault, I. (2009). Adolescent behavioral, affective, and cognitive engagement in school: Relation to dropout. Journal of School Health, 79, 408–415.CrossRefGoogle Scholar
  5. Arnold, K. E., Pistilli, M. D. (2012). Course signals at purdue: using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 267–270).Google Scholar
  6. Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement. In ASCILITE-Australian Society for Computers in Learning in Tertiary Education Annual Conference (pp. 75–86).Google Scholar
  7. Berglund, A., Eckerdal, A. (2015). Learning practice and theory in programming education: Students’ lived experience, learning and teaching in computing and engineering (LaTiCE), 2015 International Conference (pp. 180–186).Google Scholar
  8. Boekaerts, M. (2016). Engagement as an inherent aspect of the learning process. Learning and Instruction, 43, 76–83.CrossRefGoogle Scholar
  9. Breiner, J. M., Harkness, S. S., Johnson, C. C., & Koehler, C. M. (2012). What is STEM? A discussion about conceptions of STEM in education and partnerships. School Science and Mathematics, 112(1), 3–11.CrossRefGoogle Scholar
  10. Carter, V. R. (2013). Defining Characteristics of an Integrated STEM Curriculum in K–12 Education, University of Arkansas.Google Scholar
  11. Coates, H. (2005). The value of student engagement for higher education quality assurance. Quality in Higher Education, 11(1), 25–36.CrossRefGoogle Scholar
  12. Connell, J., & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A motivational analysis of self-system process. In M. R. Gunnar & L. A. Sroufe (Eds.), Self process in development: Minnesota Symposium on Child Psychology, 2 (pp. 167–216). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  13. Crick, R. D., Broadfoot, P., & Claxton, G. (2004). Developing an effective lifelong learning inventory: The ELLI project. Assessment in Education: Principles, Policy & Practice, 11(3), 247–272.CrossRefGoogle Scholar
  14. Feidakis, M., Daradoumis, T., Caballé, S., & Conesa, J. (2012). Design of an emotion aware e-learning system. International Journal of Knowledge and Learning, 8(3–4), 219–238.CrossRefGoogle Scholar
  15. Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59(2), 117–142.CrossRefGoogle Scholar
  16. Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. Handbook of research on student engagement (pp. 763–782). US: Springer.CrossRefGoogle Scholar
  17. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109.CrossRefGoogle Scholar
  18. Froyd, J. E. (2008). White paper on promising practices in undergraduate STEM education, The National Academies Board on Science Education.Google Scholar
  19. Garcia, T., & Pintrich, P. (1996). Assessing students’ motivation and learning strategies in the classroom context: The motivation and strategies in learning questionnaire. In M. Birenbaum & F. J. Dochy (Eds.), Alternatives in assessment of achievements, learning processes, and prior knowledge (pp. 319–339). New York: Kluwer Academic/Plenum Press.CrossRefGoogle Scholar
  20. Glanville, J. L., & Wildhagen, T. (2007). The measurement of school engagement: Assessing dimensionality and measurement invariance across race and ethnicity. Educational and Psychological Measurement, 67(6), 1019–1041.MathSciNetCrossRefGoogle Scholar
  21. Goetz, T., Bieg, M., Lüdtke, O., Pekrun, R., & Hall, N. C. (2013). Do girls really experience more anxiety in mathematics? Psychological Science, 24(10), 2079–2087.CrossRefGoogle Scholar
  22. Greene, B. A., Miller, R. B., Crowson, H. M., Duke, B. L., & Akey, K. L. (2004). Predicting high school students’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary Educational Psychology, 29, 462–482.CrossRefGoogle Scholar
  23. Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers & Education, 90, 36–53.CrossRefGoogle Scholar
  24. Hughes, J. N., Luo, W., Kwok, O. M., & Loyd, L. K. (2008). Teacher-student support, effortful engagement, and achievement: A 3-year longitudinal study. Journal of Educational Psychology, 100(1), 1–14.CrossRefGoogle Scholar
  25. Jimerson, S. R., Campos, E., & Greif, J. L. (2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8(1), 7–27.CrossRefGoogle Scholar
  26. Kelly, S. (2008). Race, social class, and student engagement in middle school English classrooms. Social Science Research, 37(2), 434–448.CrossRefGoogle Scholar
  27. Kong, Q. P., Wong, N. Y., & Lam, C. C. (2003). Student engagement in mathematics: Development of instrument and validation of construct. Mathematics Education Research Journal, 15(1), 4–21.CrossRefGoogle Scholar
  28. Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2007). Piecing together the student success puzzle: Research, propositions, and recommendations. ASHE Higher Education Report, 32(5), 1–182.CrossRefGoogle Scholar
  29. Kuh, G. D. (2009). The national survey of student engagement: Conceptual and empirical foundations. New directions for institutional research, 141, 5–20.CrossRefGoogle Scholar
  30. Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z. and Wolff, A. (2015). OU Analyse: analysing at-risk students at The Open University, Learning Analytics Review, 1–16.Google Scholar
  31. Ladd, G. W., & Dinella, L. M. (2009). Continuity and change in early school engagement: Predictive of children’s achievement trajectories from first to eighth grade? Journal of Educational Psychology, 101(1), 190–206.CrossRefGoogle Scholar
  32. Larson, R., & Csikszentmihalyi, M. (2014). The experience sampling method in flow and the foundations of positive psychology (pp. 21–34). Dordrecht: Springer.Google Scholar
  33. Liu, D. Y. T., Froissard, J. C., Richards, D., & Atif, A. (2015). An enhanced learning analytics plugin for Moodle: student engagement and personalised intervention, Globally Connected, Digitally Enabled. Proceedings Ascilite, pp. 180–189.Google Scholar
  34. Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal, 37(1), 153–184.CrossRefGoogle Scholar
  35. Newmann, F., Wehlage, G. G., & Lamborn, S. D. (1992). The significance and sources of student engagement. In F. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 11–39). New York: Teachers College Press.Google Scholar
  36. Olejnik, S., & Nist, S. L. (1992). Identifying latent variables measured by the Learning and Study Strategies Inventory (LASSI). The Journal of experimental education, 60(2), 151–159.CrossRefGoogle Scholar
  37. Pardo, A., Han, F., & Ellis, R. A. (2017). Combining university student self-regulated learning indicators and engagement with online learning events to predict academic performance. IEEE Transactions on Learning Technologies, 10(1), 82–92.CrossRefGoogle Scholar
  38. Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco: Jossey-Bass.Google Scholar
  39. Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40.CrossRefGoogle Scholar
  40. Pintrich, P. R., Smith, D. A., García, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.CrossRefGoogle Scholar
  41. Prensky, M. (2008) Programming is the New Literacy, Available at: http://www.edutopia.org/literacy-computer-programming (Accessed: 18 June 2017)
  42. Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137–172.CrossRefGoogle Scholar
  43. Russell, V. J., Ainley, M., & Frydenberg, E. (2005). Student motivation and engagement. Schooling Issues Digest, 2, 1–11.Google Scholar
  44. Ryu, S., & Lombardi, D. (2015). Coding classroom interactions for collective and individual engagement. Educational Psychologist, 50, 70–83.CrossRefGoogle Scholar
  45. Salmela-Aro, K., Moeller, J., Schneider, B., Spicer, J., & Lavonen, J. (2016). Integrating the light and dark sides of student engagement using person-oriented and situation-specific approaches. Learning and Instruction, 43, 61–70.CrossRefGoogle Scholar
  46. Shernoff, D. J. (2013). Optimal learning environments to promote student engagement. New York, NY: Springer.CrossRefGoogle Scholar
  47. Shernoff, D. J., & Schmidt, J. A. (2008). Further evidence of an engagement–achievement paradox among US high school students. Journal of Youth and Adolescence, 37(5), 564–580.CrossRefGoogle Scholar
  48. Shernoff, D. J., Kelly, S., Tonks, S. M., Anderson, B., Cavanagh, R. F., Sinha, S., et al. (2016). Student engagement as a function of environmental complexity in high school classrooms. Learning and Instruction, 43, 52–60.CrossRefGoogle Scholar
  49. Shum, S. B., & Crick, R. D. (2012). Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. (pp. 92–101).Google Scholar
  50. Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13.CrossRefGoogle Scholar
  51. Sinclair, J., Butler, M., Morgan, M., & Kalvala, S. (2015). Measures of student engagement in computer science. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education. (pp. 242–247).Google Scholar
  52. Sirin, S. R., & Rogers-Sirin, L. (2004). Exploring school engagement of middle-class African American adolescents. Youth & Society, 35(3), 323–340.CrossRefGoogle Scholar
  53. Skinner, E.A., Kindermann, T.A., Connell, J.P., Wellborn, J.G. (2009). Engagement and disaffection as organizational constructs in the dynamics of motivational development. Handbook of Motivation at School. (pp. 223–245).Google Scholar
  54. Skinner, E. A., & Pitzer, J. R. (2012). Developmental dynamics of student engagement, coping, and everyday resilience. Handbook of research on student engagement (pp. 21–44). US: Springer.CrossRefGoogle Scholar
  55. Smith, K. A., Douglas, T. C., & Cox, M. F. (2009). Supportive teaching and learning strategies in STEM education. New Directions for Teaching and Learning, 117, 19–32.CrossRefGoogle Scholar
  56. Susman, G.I., Evered, R.D. (1978). An assessment of the scientific merits of action research. Administrative Science Quarterly. 582–603.Google Scholar
  57. Thomas, L., Ratcliffe, M., Woodbury, J., & Jarman, E. (2002). Learning styles and performance in the introductory programming sequence. ACM SIGCSE Bulletin, 34(1), 33–37.CrossRefGoogle Scholar
  58. Voelkl, K. E. (1997). Identification with school. American Journal of Education, 294–318.Google Scholar
  59. Voelkl, K. E. (2012). School Identification. Handbook of research on student engagement (pp. 193–218). US: Springer.CrossRefGoogle Scholar
  60. Wehlage, G.G., Smith, G.A. (1992). Building new programs for students at risk. Student engagement and achievement in American secondary schools, pp. 92–118.Google Scholar
  61. Wiedenbeck, S., Labelle, D., Kain, V.N. (2004). Factors affecting course outcomes in introductory programming. 16th Annual Workshop of the Psychology of Programming Interest Group, pp. 97–109.Google Scholar
  62. Wigfield, A., Guthrie, J. T., Perencevich, K. C., Taboada, A., Klauda, S. L., McRae, A., et al. (2008). Role of reading engagement in mediating effects of reading comprehension instruction on reading outcomes. Psychology in the Schools, 45(5), 432–445.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Isuru Balasooriya
    • 1
  • Enric Mor
    • 1
  • M. Elena Rodríguez
    • 1
  1. 1.Faculty of Computer Science, Multimedia and TelecommunicationsUniversitat Oberta de CatalunyaBarcelonaSpain

Personalised recommendations