Skip to main content
SpringerLink
Log in
Book cover

European Conference on Technology Enhanced Learning

EC-TEL 2017: Data Driven Approaches in Digital Education pp 209–223Cite as

Opportunities and Challenges in Using Learning Analytics in Learning Design

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10474))

Abstract

Educational institutions are designing, creating and evaluating courses to optimize learning outcomes for highly diverse student populations. Yet, most of the delivery is still monitored retrospectively with summative evaluation forms. Therefore, improvements to the course design are only implemented at the very end of a course, thus missing to benefit the current cohort. Teachers find it difficult to interpret and plan interventions just-in-time. In this context, Learning Analytics (LA) data streams gathered from ‘authentic’ student learning activities, may provide new opportunities to receive valuable information on the students’ learning behaviors and could be utilized to adjust the learning design already “on the fly” during runtime. We presume that Learning Analytics applied within Learning Design (LD) and presented in a learning dashboard provide opportunities that can lead to more personalized learning experiences, if implemented thoughtfully.

In this paper, we describe opportunities and challenges for using LA in LD. We identify three key opportunities for using LA in LD: (O1) using on demand indicators for evidence based decisions on learning design; (O2) intervening during the run-time of a course; and, (O3) increasing student learning outcomes and satisfaction. In order to benefit from these opportunities, several challenges have to be overcome. Following a thorough literature review, we mapped the identified opportunities and challenges in a conceptual model that considers the interaction of LA in LD.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Altbach, P., Reisberg, L., Rumbley, L.: Tracking a global academic revolution. Change 42(2), 30–39 (2010)

    Article  Google Scholar 

  2. Field, J.: Lifelong learning and the multigenerational workforce. In: Burke, R.J., Cooper, C.L., Antoniou, A.-S.G. (eds.) The multi-generational and Aging Workforce: Challenges and Opportunities, pp. 311–325. Edward Elgar Publishing, Cheltenham (2015)

    Chapter  Google Scholar 

  3. Volles, N.: Lifelong learning in the EU: changing conceptualisations, actors, and policies. Stud. High. Educ. 41(2), 343–363 (2014)

    Article  Google Scholar 

  4. Nonis, S., Hudson, G.: Academic performance of college students: influence of time spent studying and working. J. Edu. Bus. 81(3), 151–159 (2006)

    Article  Google Scholar 

  5. Tuononen, T., Parpala, A., Mattsson, M., Lindblom-Ylänne, S.: Work experience in relation to study pace and thesis grade: investigating the mediating role of student learning. High. Educ. 72(1), 41–58 (2016)

    Article  Google Scholar 

  6. Ministerie van Onderwijs, Cultuur & Wetenschap: De waarde(n) van weten: strategische agenda hoger onderwijs en onderzoek 2015–2025. Den Haag (2015)

    Google Scholar 

  7. Flavell, J.: Metacognition and cognitive monitoring: a new area of cognitive–developmental inquiry. Am. Psychol. 34(10), 906–911 (1979)

    Article  Google Scholar 

  8. Donald, C., Blake, A., Girault, I., Datt, A., Ramsay, E.: Approaches to learning design: past the head and the hands to the HEART of the matter. Distance Edu. 30(2), 179–199 (2009)

    Article  Google Scholar 

  9. Peterson, C.: Bringing ADDIE to life: instructional design at its best. J. Edu. Multimed. Hypermedia 12(3), 227–241 (2003)

    Google Scholar 

  10. Baker, R.: A framework for design and evaluation of internet-based distance learning courses: Phase one–Framework justification design and evaluation. Online J. Distance Learn. Admin. 6(2), 43–51 (2003)

    Google Scholar 

  11. Bloom, B., Engelhart, M., Furst, E., Hill, W., Krathwohl, D.: Taxonomy of Educational Objectives: The Classification Of Educational Goals. Handbook I: Cognitive Domain. David McKay Company, New York (1956)

    Google Scholar 

  12. Tyler, R.: Basic Principles of Curriculum and Instruction. University of Chicago Press, Chicago (1949)

    Google Scholar 

  13. Bundsgaard, J., Hansen, T.: Evaluation of learning materials: a holistic framework. J. Learn. Des. 4(4), 31–45 (2011)

    Google Scholar 

  14. Falconer, I., Beetham, H., Oliver, R., Littlejohn, A.: Mod4L final report: representing learning designs. Final report for the JISC-funded MOD4L project. Glasgow (2007)

    Google Scholar 

  15. Leacock, T., Nesbit, J.: A framework for evaluating the quality of multimedia learning resources. Edu. Technol. Soc. 10, 44–59 (2007)

    Google Scholar 

  16. Falconer, I.: Mediating between practitioner and developer communities: the learning activity design in education experience. Alt-J 15(2), 155–170 (2007)

    Article  MathSciNet  Google Scholar 

  17. Masie, E.: Big Learning Data. ASTD Press, Alexandria (2014)

    Google Scholar 

  18. Avella, J., Kebritchi, M., Nunn, S., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learn. J. 20(2), 13–29 (2016)

    Google Scholar 

  19. Siemens, G.: Learning analytics: the emergence of a discipline. Am. Behav. Sci. 57(10), 1380–1400 (2013)

    Article  Google Scholar 

  20. Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Shum, S., Ferguson, R., Duval, E., Verbert, K., Baker, R.: Open learning analytics: an integrated and modularized platform. Educause Rev. 42(4), 53–54 (2007)

    Google Scholar 

  21. Cooper, A.: A framework of characteristics for analytics. CETIS Anal. Ser. 1(7), 1–17 (2012)

    MathSciNet  Google Scholar 

  22. Scheffel, M., Drachsler, H., Stoyanov, S., Specht, M.: Quality indicators for learning analytics. Edu. Technol. Soc. 17(4), 124–140 (2014)

    Google Scholar 

  23. Greller, W., Drachsler, H.: Translating learning into numbers: a generic framework for learning analytics. Edu. Technol. Soc. 15(3), 42–57 (2012)

    Google Scholar 

  24. Ferguson, R.: Learning analytics: drivers, developments and challenges. Int. J. Technol. Enhanced Learn. 4(5/6), 304–317 (2012)

    Article  Google Scholar 

  25. Bichsel, J.: Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations. Educause Center for Applied Research, Louisville (2012)

    Google Scholar 

  26. Colvin, C., Rogers, T., Wade, A., Dawson, S., Gasevic, D., Shum, S., Nelson, K., Alexander, S., Lockyer, L., Kennedy, G., Corrin, L., Fisher, J.: Student retention and learning analytics: a snapshot of Australian practices and a framework for advancement. Australian Office for Learning and Teaching, Sydney (2015)

    Google Scholar 

  27. Rienties, B., Toetenel, L.: The impact of 151 learning designs on student satisfaction and performance: social learning (analytics) matters. In: Proceedings of LAK 2016 6th International Conference on Analytics and Knowledge, pp. 339–343 (2016)

    Google Scholar 

  28. Clow, D., Cross, S., Ferguson, R.: Evidence hub review. LACE Project, Milton Keynes (2014)

    Google Scholar 

  29. Papamitsiou, Z., Economides, A.: Learning analytics and educational data mining in practice: a systematic literature review of empirical evidence. Edu. Technol. Soc. 17(4), 49–64 (2014)

    Google Scholar 

  30. Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., Murphy, S.: Analytics4Action evaluation framework: a review of evidence-based learning analytics interventions at the Open University UK. J. Interact. Med. Edu. 1, 2 (2016)

    Google Scholar 

  31. Few, S.: Information dashboard design: the effective visual communication of data. O’Reilly Media, Sebastopol (2006)

    Google Scholar 

  32. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.: Learning analytics dashboard applications. Am. Behav. Sci. 57(10), 1500–1509 (2013)

    Article  Google Scholar 

  33. Mor, Y., Ferguson, R., Wasson, B.: Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. Br. J. Edu. Technol. 46(2), 221–229 (2015)

    Article  Google Scholar 

  34. Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 309–318. ACM, New York (2017)

    Google Scholar 

  35. Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Trans. Learn. Technol. 10(1), 30–41 (2017)

    Article  Google Scholar 

  36. Butler, D., Winne, P.: Feedback and self-regulated learning: a theoretical synthesis. Rev. Edu. Res. 65(3), 245–281 (1995)

    Article  Google Scholar 

  37. Winne, P.: Inherent details in self-regulated learning. Edu. Psychol. 30(4), 173–187 (1995)

    Article  Google Scholar 

  38. Zimmerman, B.: Self-regulation involves more than metacognition: a social cognitive perspective. Edu. Psychol. 30(4), 217–221 (1995)

    Article  Google Scholar 

  39. Winne, P.: How software technologies can improve research on learning and bolster school reform. Edu. Psychol. 41(1), 5–17 (2006)

    Article  MathSciNet  Google Scholar 

  40. Lockyer, L., Heathcote, E., Dawson, S.: Informing pedagogical action: aligning learning analytics with learning design. Am. Behav. Sci. 57(10), 1439–1459 (2013)

    Article  Google Scholar 

  41. Bakharia, A., Dawson, S.: SNAPP: a bird’s-eye view of temporal participant interaction. In: Proceedings of LAK 2011 1st International Conference on Analytics and Knowledge, pp. 168–173 (2011)

    Google Scholar 

  42. Persico, D., Pozzi, F.: Informing learning design with learning analytics to improve teacher inquiry. Br. J. Edu. Technol. 46(2), 230–248 (2015)

    Article  Google Scholar 

  43. Wise, A., Shaffer, D.: Why theory matters more than ever in the age of big data. J. Learn. Anal. 2(2), 5–13 (2015)

    Article  Google Scholar 

  44. Rodríguez-Triana, M., Martínez-Monés, A., Asensio-Pérez, J., Dimitriadis, Y.: Scripting and monitoring meet each other: aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. Br. J. Edu. Technol. 46(2), 330–343 (2015)

    Article  Google Scholar 

  45. Bakharia, A., Corrin, L., Barba, P. De, Kennedy, G., Gasevic, D., Mulder, R., Williams, D., Dawson, S., Lockyer, L.: A conceptual framework linking learning design with learning analytics. In: Proceedings of LAK 2016 6th International Conference on Analytics and Knowledge, pp. 329–338 (2016)

    Google Scholar 

  46. Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Pers. Ubiquit. Comput. 18(6), 1499–1514 (2014)

    Google Scholar 

  47. Park, Y., Jo, I.: Development of the learning analytics dashboard to support students’ learning performance. J. Univ. Comput. Sci. 21(1), 110–133 (2015)

    Google Scholar 

  48. FitzGerald, E., Kucirkova, N., Jones, A., Cross, S., Ferguson, R., Herodotou, C., Hillaire, G., Scanlon, E.: Dimensions of personalisation in technology-enhanced learning: a framework and implications for design. Br. J. Edu. Technol. (2017, in press)

    Google Scholar 

  49. Jivet, I., Scheffel, M., Drachsler, H., Specht, M.: Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In: Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017) (in press)

    Google Scholar 

  50. Scheffel, M., Drachsler, H., Toisoul, C., Ternier, S., Specht, M.: The proof of the pudding: examining validity and reliability of the evaluation framework for learning analytics. In: Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017) (in press)

    Google Scholar 

  51. Koper, R., Olivier, B.: Representing the learning design of units of learning. Edu. Technol. Soc. 7(3), 97–111 (2004)

    Google Scholar 

  52. Prinsloo, P., Slade, S.: An elephant in the learning analytics room: the obligation to act. In: LAK 2017 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 46–55 (2017)

    Google Scholar 

  53. Trigwell, K., Prosser, M.: Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes. High. Educ. 22(3), 251–266 (1991)

    Article  Google Scholar 

  54. De La Fuente, J., Sander, P., Martínez-Vicente, J.M., Vera, M., Garzón, A., Fadda, S.: Combined effect of levels in personal self-regulation and regulatory teaching on meta-cognitive, on meta-motivational, and on academic achievement variables in undergraduate students. Front. Psychol. 8, 232 (2017)

    Article  Google Scholar 

  55. Gašević, D., Jovanović, J., Pardo, A., Dawson, S.: Detecting learning strategies with analytics: links with self-reported measures and academic performance. J. Learn. Anal. 4(1), 113–128 (2017)

    Google Scholar 

  56. Pardo, A., Martinez-Maldonado, R., Buckingham Shum, S., Schulte, J., McIntyre, S., Gašević, D., Gao, J., Siemens, G.: Connecting data with student support actions in a course: a hands-on tutorial. In: LAK 2017 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 522–523 (2017)

    Google Scholar 

  57. Pintrich, P., Smith, D., Garcia, T., Mckeachie, W.: Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educ. Psychol. Measur. 53(3), 801–813 (1993)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the SURF Foundation & NRO for supporting the efforts of Marcel Schmitz, Evelien van Limbeek and Hendrik Drachsler under the REFLECTOR project grant.

Author information

Authors and Affiliations

  1. Zuyd University of Applied Sciences, Nieuw Eyckholt 300, 6419 AT, Heerlen, The Netherlands

    Marcel Schmitz & Evelien van Limbeek

  2. Vienna University of Education, Vienna, Austria

    Wolfgang Greller

  3. Open University, Valkenburgerweg 177, 6419 AT, Heerlen, The Netherlands

    Peter Sloep & Hendrik Drachsler

  4. Goethe University, Frankfurt, Germany

    Hendrik Drachsler

  5. German Institute for International Educational Research (DIPF), Frankfurt, Germany

    Hendrik Drachsler

Authors
  1. Marcel Schmitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evelien van Limbeek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfgang Greller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Sloep
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hendrik Drachsler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Marcel Schmitz , Evelien van Limbeek , Wolfgang Greller , Peter Sloep or Hendrik Drachsler .

Editor information

Editors and Affiliations

  1. Jean Moulin Lyon 3 University, Lyon, France

    Élise Lavoué

  2. German Institute for International Educational Research, Goethe University, Frankfurt, Germany

    Hendrik Drachsler

  3. KU Leuven, Leuven, Belgium

    Katrien Verbert

  4. University of Toulouse, Toulouse, France

    Julien Broisin

  5. Pontificia Universidad, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Schmitz, M., van Limbeek, E., Greller, W., Sloep, P., Drachsler, H. (2017). Opportunities and Challenges in Using Learning Analytics in Learning Design. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds) Data Driven Approaches in Digital Education. EC-TEL 2017. Lecture Notes in Computer Science(), vol 10474. Springer, Cham. https://doi.org/10.1007/978-3-319-66610-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66610-5_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66609-9

  • Online ISBN: 978-3-319-66610-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation