Advertisement

Learning Analytics: Using Data-Informed Decision-Making to Improve Teaching and Learning

  • Alyssa Friend WiseEmail author
Chapter

Abstract

This chapter describes three characteristics of Learning Analytics work that distinguish it from prior educational research to give readers a concise overview of what makes learning analytics a unique and especially promising technology to improve teaching and learning. Data used in learning analytics relate to the process of learning, can come from a variety of sources (in both virtual and physical learning environments), and are characterized by their large quantity and relatively small grain size. Analysis approaches aim at detecting underlying patterns and relationships in the data and include prediction, structure discovery, temporal, language-based and visual methods. Pedagogical uses are what position learning analytics as more than simply a new set of methods but an impactful technology to drive data-informed decision-making through tailoring educational experiences, informing student self-direction, and supporting instructor planning and orchestration. The chapter concludes with an overview of the systemic and societal issues surrounding learning analytics use that frame how and to what extent they are able to affect education.

Keywords

Learning analytics Decision-making 

References

  1. ACM US Public Policy Council. (2017). Statement on algorithmic transparency and accountability. Washington, DC: ACM.Google Scholar
  2. Ahn, J. (2013). What can we learn from Facebook activity?: Using social learning analytics to observe new media literacy skills. In Proceedings of the third international conference on learning analytics & knowledge (pp. 135–144). Leuven: ACM.CrossRefGoogle Scholar
  3. Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers & Education, 58(1), 470–489.CrossRefGoogle Scholar
  4. Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 33(1), 1–10.Google Scholar
  5. Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (2nd ed., pp. 253–274). Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  6. Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.Google Scholar
  7. Baker, R. S., Hershkovitz, A., Rossi, L. M., Goldstein, A. B., & Gowda, S. M. (2013). Predicting robust learning with the visual form of the moment-by-moment learning curve. Journal of the Learning Sciences, 22(4), 639–666.CrossRefGoogle Scholar
  8. Baltrušaitis, T., Robinson, P., & Morency, L. P. (2016). Openface: An open source facial behavior analysis toolkit. In Proceedings of 2016 IEEE winter conference on applications of computer vision (pp. 1–10). Lake Placid: IEEE.Google Scholar
  9. Bergner, Y. (2017). Measurement and its uses in learning analytics. In Handbook of learning analytics (1st ed., pp. 35–48). Edmonton: SoLAR.CrossRefGoogle Scholar
  10. Brooks, C., Greer, J., & Gutwin, C. (2014). The data-assisted approach to building intelligent technology-enhanced learning environments. In J. A. Larusson & B. White (Eds.), Learning analytics (pp. 123–156). New York: Springer.Google Scholar
  11. Chen, B., & Resendes, M. (2014). Uncovering what matters: Analyzing transitional relations among contribution types in knowledge-building discourse. In Proceedings of the fourth international conference on learning analytics & knowledge (pp. 226–230). Indianapolis: ACM.CrossRefGoogle Scholar
  12. Chen, B., & Zhang, J. (2016). Analytics for knowledge creation: Towards epistemic agency and design-mode thinking. Journal of Learning Analytics, 3(2), 139–163.CrossRefGoogle Scholar
  13. Chen, B., Wise, A. F., Knight, S., & Cheng, B. H. (2016). Putting temporal analytics into practice: The 5th international workshop on temporality in learning data. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 488–489). Edinburgh: ACM.CrossRefGoogle Scholar
  14. Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.CrossRefGoogle Scholar
  15. Csikszentmihalyi, M., & Larson, R. (2014). Validity and reliability of the experience-sampling method. In Flow and the foundations of positive psychology (pp. 35–54). New York: Springer.Google Scholar
  16. Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press.Google Scholar
  17. Cui, Y., Jin, W. Q., & Wise, A. F. (2017). Humans and machines together: Improving characterization of large scale online discussions through dynamic interrelated post and thread categorization (DIPTiC). In Proceedings of learning at scale 2017 (pp. 217–219). Cambridge, MA: ACM.Google Scholar
  18. D’Angelo, C. M., Roschelle, J., & Bratt, H. (2015). Using students’ speech to characterize group collaboration quality. In Proceedings of the international conference on computer supported collaborative learning. Gothenburg: ISLS.Google Scholar
  19. Dawson, S. (2010). ‘Seeing’ the learning community: An exploration of the development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), 736–752.CrossRefGoogle Scholar
  20. Denley, T. (2013). Degree compass: A course recommendation system. Educause Review Online. https://er.educause.edu/articles/2013/9/degree-compass-a-course-recommendation-system
  21. Dowell, N., Skrypnyk, O., Joksimović, S., Graesser, A. C., Dawson, S., Gašević, D., Vries, P. D., Hennis, T., & Kovanović, V. (2015). Modeling learners’ social centrality and performance through language and discourse. In Proceedings of the 8th international conference on educational data mining (pp. 250–257). New York: ACM.Google Scholar
  22. Drachsler, H., Verbert, K., Santos, O. C., & Manouselis, N. (2015). Panorama of recommender systems to support learning. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender systems handbook (pp. 421–451). New York: Springer.CrossRefGoogle Scholar
  23. Duval, E., & Verbert, K. (2012). Learning analytics. E-Learning and Education, 1(8). https://eleed.campussource.de/archive/8/3336
  24. Ertmer, P. A. (1999). Addressing first-and second-order barriers to change: Strategies for technology integration. Educational Technology Research and Development, 47(4), 47–61.CrossRefGoogle Scholar
  25. Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T., & Vuorikari, R. (2016). Research evidence on the use of learning analytics – Implications for education policy. In R. Vuorikari & J. Castaño Muñoz (Eds.), Joint research centre science for policy report; EUR 28294 EN;  https://doi.org/10.2791/955210.
  26. Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2), 89–97.CrossRefGoogle Scholar
  27. Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.CrossRefGoogle Scholar
  28. Hecking, T., Chounta, I. A., & Hoppe, H. U. (2016). Investigating social and semantic user roles in MOOC discussion forums. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 198–207). New York: ACM.CrossRefGoogle Scholar
  29. Hero, A. O., & Rajaratnam, B. (2016). Foundational principles for large-scale inference: Illustrations through correlation mining. Proceedings of the IEEE, 104(1), 93–110.CrossRefGoogle Scholar
  30. Huberth, M., Chen, P., Tritz, J., & McKay, T. A. (2015). Computer-tailored student support in introductory physics. PLoS One, 10(9), e0137001.CrossRefGoogle Scholar
  31. Jayaprakash, S. M., Moody, E. W., Lauría, E. J., Regan, J. R., & Baron, J. D. (2014). Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), 6–47.CrossRefGoogle Scholar
  32. Jeong, H., Biswas, G., Johnson, J., & Howard, L. (2010). Analysis of productive learning behaviors in a structured inquiry cycle using hidden markov models. In Proceedings of the third international conference on educational data mining (pp. 81–90). Pittsburgh: EDM.Google Scholar
  33. Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education edition. Austin: The New Media Consortium.Google Scholar
  34. Joksimović, S., Kovanović, V., Jovanović, J., Zouaq, A., Gašević, D., & Hatala, M. (2015). What do cMOOC participants talk about in social media? A topic analysis of discourse in a cMOOC. In Proceedings of the fifth international conference on learning analytics & knowledge (pp. 156–165). Poughkeepsie: ACM.Google Scholar
  35. Joksimović, S., Manataki, A., Gašević, D., Dawson, S., Kovanović, V., & De Kereki, I. F. (2016). Translating network position into performance: Importance of centrality in different network configurations. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 314–323). Edinburgh: ACM.Google Scholar
  36. Klerkx, J., Verbert, K., & Duval, E. (2017). Learning analytics dashboards. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 143–150). Edmonton: SoLAR.CrossRefGoogle Scholar
  37. Knight, S., & Littleton, K. (2015). Discourse-centric learning analytics: Mapping the terrain. Journal of Learning Analytics, 2(1), 185–209.CrossRefGoogle Scholar
  38. Knight, S., Wise, A. F., Chen, B., & Cheng, B. H. (2015). It’s about time: 4th international workshop on temporal analyses of learning data. In Proceedings of the fifth international conference on learning analytics & knowledge (pp. 388–389). Poughkeepise: ACM.Google Scholar
  39. Kolb, D. A. (1984). Experiential education: Experience as the source of learning and learning science. Englewood Cliffs: Prentice Hall.Google Scholar
  40. Kovanović, V., Joksimović, S., Gašević, D., Hatala, M., & Siemens, G. (2017). Content analytics: The definition, scope, and an overview of published research. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 77–92). Edmonton: SoLAR.CrossRefGoogle Scholar
  41. Landauer, T. K., MacNamara, D. S., Dennis, S., & Kintsch, W. (Eds.). (2011). Handbook of latent semantic analysis. New York: Routledge.Google Scholar
  42. Laney, D. (2001). 3D data management: Controlling data volume, velocity and variety. META Group Research Note, 6, 70.Google Scholar
  43. Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.CrossRefGoogle Scholar
  44. McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57–86.CrossRefGoogle Scholar
  45. McNamara, D., Allen, L., Crossley, S., Dascalu, M., & Perret, C. (2017). Natural language processing and learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 93–104). Edmonton: SoLAR.CrossRefGoogle Scholar
  46. Merceron, A., & Yacef, K. (2008). Interestingness measures for association rules in educational data. In Proceedings for the first international conference on educational data mining 2008 (pp. 57–66). Montreal: International Working Group on Educational Data Mining.Google Scholar
  47. Mu, J., Stegmann, K., Mayfield, E., Rosé, C., & Fischer, F. (2012). The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions. International Journal of Computer-Supported Collaborative Learning, 7(2), 285–305.CrossRefGoogle Scholar
  48. Nwana, H. S. (1990). Intelligent tutoring systems: An overview. Artificial Intelligence Review, 4(4), 251–277.CrossRefGoogle Scholar
  49. Ochoa, X., & Worsley, M. (2016). Augmenting learning analytics with multimodal sensory data. Journal of Learning Analytics, 3(2), 213–219.CrossRefGoogle Scholar
  50. Papamitsiou, Z., & Economides, A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.Google Scholar
  51. Parameswaran, A., Venetis, P., & Garcia-Molina, H. (2011). Recommendation systems with complex constraints: A course recommendation perspective. ACM Transactions on Information Systems (TOIS), 29(4), 20.CrossRefGoogle Scholar
  52. Pecaric, M., Boutis, K., Beckstead, J., & Pusic, M. (2017). A big data and learning analytics approach to process-level feedback in cognitive simulations. Academic Medicine, 92(2), 175–184.CrossRefGoogle Scholar
  53. Poon, L. K., Kong, S. C., Wong, M. Y., & Yau, T. S. (2017). Mining sequential patterns of students’ access on learning management system. In International conference on data mining and big data (pp. 191–198). Fukuoka: Springer.CrossRefGoogle Scholar
  54. Poquet, L., & Dawson, S. (2016). Untangling MOOC learner networks. In Proceedings of the sixth international conference on learning analytics and knowledge (pp. 208–212). Edinburgh: ACM.Google Scholar
  55. Prinsloo, P., & Slade, S. (2013). An evaluation of policy frameworks for addressing ethical considerations in learning analytics. In Proceedings of the third international conference on learning analytics and knowledge (pp. 240–244). Indianapolis: ACM.CrossRefGoogle Scholar
  56. Rabbany, R., Takaffoli, M., & Zaïane, O. R. (2011). Analyzing participation of students in online courses using social network analysis techniques. In M. Pechenizkiy, T. Calders, C. Conati, S. Ventura, C. Romero, & J. Stamper (Eds.), Proceedings of the 4th international conference on educational data mining (pp. 21–30). EDM.Google Scholar
  57. Ritsos, P. D., & Roberts, J. C. (2014). Towards more visual analytics in learning analytics. In M. Phol & J. C. Roberts (Eds.), Proceedings of the EuroVis workshop on visual analytics (pp. 61–65). Swansea: Eurographics Association.Google Scholar
  58. Roll, I., MacFadyen, L. P., Ni, P., Cimet, M., Shiozaki, L., Paulin, D., & Harris, S. (2016). Questions, not answers: Boosting student participation in MOOC forums. In Proceedings of learning with MOOC IIIs (pp. 23–25). Philadelphia: LWMOOC.Google Scholar
  59. Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.). (2010). Handbook of educational data mining. New York: CRC Press.Google Scholar
  60. Rosé, C. (2017). Discourse analytics. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 105–114). Edmonton: SoLAR.CrossRefGoogle Scholar
  61. Rosé, C., Wang, Y. C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 237–271.CrossRefGoogle Scholar
  62. Rummel, N., Walker, E., & Aleven, V. (2016). Different futures of adaptive collaborative learning support. International Journal of Artificial Intelligence in Education, 26(2), 784–795.CrossRefGoogle Scholar
  63. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.Google Scholar
  64. Sclater, N. (2014). Code of practice for learning analytics: A literature review of the ethical and legal issues. Bristol: JISC.Google Scholar
  65. Sclater, N. (2017). Learning analytics explained. New York: Routledge.CrossRefGoogle Scholar
  66. Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., & Fernández-Manjón, B. (2014). Application of learning analytics in educational videogames. Entertainment Computing, 5(4), 313–322.CrossRefGoogle Scholar
  67. Shneiderman, B. (2014). The big picture for big data: Visualization. Science, 343(6172), 730–730.CrossRefGoogle Scholar
  68. Shum, S. B., Knight, S., McNamara, D., Allen, L., Bektik, D., & Crossley, S. (2016). Critical perspectives on writing analytics. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 481–483). Edinburgh: ACM.CrossRefGoogle Scholar
  69. Siemens, G., Gašević, D., Haythornthwaite, C., Dawson, S., Buckingham Shum, S., Ferguson, R., Duval, E., Verbert, K., & Baker, R. S. (2011). Open learning analytics: An integrated & modularized platform. [Concept paper]. Society for Learning Analytics Research.Google Scholar
  70. Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014). Your click decides your fate: Inferring information processing and attrition behavior from MOOC video clickstream interactions. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) workshop on modeling large scale social interaction in massively open online courses (pp. 3–14). Doha: ACL.Google Scholar
  71. Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529.CrossRefGoogle Scholar
  72. Slade, S., & Prinsloo, P. (2014). Student perspectives on the use of their data: Between intrusion, surveillance, and care. In Challenges for research into open & distance learning: Doing things better – Doing better things (pp. 291–300). Oxford: European Distance and E-Learning Network.Google Scholar
  73. Suthers, D. D., & Desiato, C. (2012). Exposing chat features through analysis of uptake between contributions. In Proceedings of the 45th Hawaii international conference on system science (pp. 3368–3377). Maui: IEEE.Google Scholar
  74. Suthers, D., Wise, A. F., Schneider, B., Shaffer, D. W., Hoppe, H. U., & Siemens, G. (2015). Learning analytics of and in mediational processes of collaborative learning. In O. Lindwall, P. Häkkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Proceedings of the eleventh international conference on computer supported collaborative learning (Vol. I, pp. 26–30). Gothenburg: ISLS.Google Scholar
  75. Svihla, V., Wester, M. J., & Linn, M. C. (2015). Distributed revisiting: An analytic for retention of coherent science learning. Journal of Learning Analytics, 2(2), 75–101.CrossRefGoogle Scholar
  76. van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics, 2(2), 138–162.CrossRefGoogle Scholar
  77. Velazquez, E., Ratté, S., & de Jong, F. (2016). Analyzing students’ knowledge building skills by comparing their written production to syllabus. In Proceedings of the international conference on interactive collaborative learning (pp. 345–352). Belfast: Springer.Google Scholar
  78. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509.CrossRefGoogle Scholar
  79. Vytasek, J., Wise, A. F., & Woloshen, S. (2017). Topic models to support instructors in MOOC forums. In Proceedings of the seventh international conference on learning analytics & knowledge (pp. 610–611). Vancouver: ACM.CrossRefGoogle Scholar
  80. Whitelock, D., Twiner, A., Richardson, J. T., Field, D., & Pulman, S. (2015). OpenEssayist: A supply and demand learning analytics tool for drafting academic essays. In Proceedings of the fifth international conference on learning analytics & knowledge (pp. 208–212). Poughkeepsie: ACM.Google Scholar
  81. Winne, P. H. (2010). Improving measurements of self-regulated learning. Educational Psychologist, 45(4), 267–276.CrossRefGoogle Scholar
  82. Winne, P. H. (2017). Leveraging big data to help each learner upgrade learning and accelerate learning science. Teachers College Record, 119(3), 1–24.Google Scholar
  83. Wise, A. F., & Chiu, M. M. (2011). Analyzing temporal patterns of knowledge construction in a role-based online discussion. International Journal of Computer-Supported Collaborative Learning, 6(3), 445–470.CrossRefGoogle Scholar
  84. Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.CrossRefGoogle Scholar
  85. Wise, A. F., & Vytasek, J. M. (2017). Learning analytics implementation design. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (1st ed., pp. 151–160). Edmonton: SoLAR.CrossRefGoogle Scholar
  86. Wise, A. F., Speer, J., Marbouti, F., & Hsiao, Y. (2013). Broadening the notion of participation in online discussions: Examining patterns in learners’ online listening behaviors. Instructional Science, 41(2), 323–343.CrossRefGoogle Scholar
  87. Wise, A. F., Zhao, Y., & Hausknecht, S. N. (2014). Learning analytics for online discussions: Embedded and extracted approaches. Journal of Learning Analytics, 1(2), 48–71.CrossRefGoogle Scholar
  88. Wise, A. F., Vytasek, J. M., Hausknecht, S. N., & Zhao, Y. (2016). Developing learning analytics design knowledge in the “middle space”: The student tuning model and align design framework for learning analytics use. Online Learning, 20(2), 1–28.CrossRefGoogle Scholar
  89. Wise, A. F., Cui, Y., & Jin, W. Q. (2017). Honing in on social learning networks in MOOC forums: Examining critical network definition decisions. In Proceedings of the seventh international conference on learning analytics & knowledge (pp. 383–392). Vancouver: ACM.CrossRefGoogle Scholar
  90. Yang, D., Sinha, T., Adamson, D., & Rose, C. P. (2013). Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses. In Proceedings of the 2013 NIPS workshop on data-driven education. Lake Tahoe: NIPS Foundation.Google Scholar
  91. Zheng, A. (2015). Evaluating machine learning models. Boston: O’Reilly Media.Google Scholar
  92. Zhu, M., Bergner, Y., Zhang, Y., Baker, R. S. J. D., Wang, Y., Paquette, L., & Barnes, T. (2016). Longitudinal engagement, performance, and social connectivity: A MOOC case study using exponential random graph models. In Proceedings of the sixth international conference on learning analytics and knowledge (pp. 223–230). Edinburgh: ACM.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  1. 1.Learning Analytics Research Network (LEARN)New York UniversityNYUSA

Personalised recommendations