Measuring the Impact of E-Learning Platforms on Information Security Awareness

  • Tobias FertigEmail author
  • Andreas E. Schütz
  • Kristin Weber
  • Nicholas H. Müller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11590)


Humans play a central role in information security. The behavior of workers at their workplace affects the confidentiality, integrity, and availability of sensitive corporate information. In addition, attackers exploit the “human factor” as a weak point with techniques such as phishing, malware, and social engineering. Exploiting the lack of awareness is often an easy task with minimal risk. To make employees aware of their important role, companies typically carry out security awareness campaigns. Our university created an e-Learning Platform (eLP) to support our awareness campaigns. In order to determine the success, the effectiveness and the impact of such an awareness campaign, suitable measurement methods are needed. A common approach to measure the success of eLPs is to run surveys and questionnaires with the learners. Since the manual evaluation of those surveys and questionnaires is a time-consuming task, we are researching how a possible automation can be achieved. Moreover, the effectiveness is often evaluated through quizzes or knowledge tests. Since knowledge by itself does not improve the behavior of people, the compliant-behavior has to be measured, too. We derived metrics for success and effectiveness but recognized that success can hardly be measured automatically. To reduce the manual effort we decided to only measure the effectiveness automatically. Therefore, we are measuring the behavior and determine if the security-compliance has increased.


Information security awareness Measuring e-Learning Platforms Success Effectiveness Automated measuring 


  1. 1.
    Aparicio, M., Bacao, F., Oliveira, T.: Cultural impacts on e-learning systems’ success. Internet High. Educ. 31, 58–70 (2016). Scholar
  2. 2.
    Bada, M., Sasse, A.M., Nurse, J.R.: Cyber security awareness campaigns: why do they fail to change behaviour? In: Global Cyber Security Capacity Centre: Draft Working Paper, pp. 188–131 (2014)Google Scholar
  3. 3.
    Bell, M., Farrier, S.: Measuring success in e-learning-a multi-dimensional approach. Electron. J. e-Learn. 6(2), 99–110 (2008). Scholar
  4. 4.
    DeLone, W.H., McLean, E.R.: Information systems success: the quest for the dependent variable. Inf. Syst. Res. 3(1), 60–95 (1992). Scholar
  5. 5.
    DeLone, W.H., McLean, E.R.: The DeLone and McLean model of information systems success: a ten-year update. J. Manage. Inf. Syst. 19(4), 9–30 (2003). Scholar
  6. 6.
    Fleming, J., Becker, K., Newton, C.: Factors for successful e-learning: does age matter? Educ. + Training 59(1), 76–89 (2017). Scholar
  7. 7.
    Freeze, R.D., Alshare, K.A., Lane, P.L., Wen, H.J.: IS success model in e-learning context based on students’ perceptions. J. Inf. Syst. Educ. 21, 13 (2014)Google Scholar
  8. 8.
    Hagen, J., Ole Johnsen, S., Albrechtsen, E.: The long-term effects of information security-learning on organizational learning. Inf. Manage. Comput. Secur. 19(3), 140–154 (2011). Scholar
  9. 9.
    Harich, T.W.: IT-sicherheit im Unternehmen. mitp Professional, mitp-Verlags, Frechen, [Germany], 1. auflage edn. (2015)Google Scholar
  10. 10.
    Hassanzadeh, A., Kanaani, F., Elahi, S.: A model for measuring e-learning systems success in universities. Expert Syst. Appl. 39(12), 10959–10966 (2012). Scholar
  11. 11.
    Helisch, M., Pokoyski, D.: Security Awareness: Neue Wege zur erfolgreichen Mitarbeiter-Sensibilisierung. Vieweg+Teubner Verlag/GWV Fachverlage GmbH Wiesbaden, Wiesbaden (2009). Scholar
  12. 12.
    ISACA: State of Cybersecurity 2017. Part 2: Current Trends in Threat Landscape. Technical report, Information Systems Audit and Control Association, ISACA, 3701 Algonquin Road, Suite 1010 Rolling Meadows, IL 60008 USA (2017).
  13. 13.
    Kasprzyk, D., Montaño, D.E.: Application of an integrated behavioral model to understand HIV prevention behavior of high-risk men in rural Zimbabwe. In: Ajzen, I., Albarracin, D. (eds.) Prediction and Change of Health Behavior: Applying the Reasoned Action Approach, pp. 145–168. Psychology Press, London (2007)Google Scholar
  14. 14.
    Kirkpatrick, D.L.: Evaluating Training Programs: The Four Levels. Berrett-Koehler, Oakland (1994)Google Scholar
  15. 15.
    Kütz, M.: Kennzahlen in der IT: Werkzeuge für Controlling und Management. dpunkt-Verlag (2007). Google-Books-ID: bkbXGAAACAAJGoogle Scholar
  16. 16.
    Lin, H.F.: Measuring online learning systems success: applying the updated DeLone and McLean model. CyberPsychol. Behav. 10(6), 817–820 (2007). Scholar
  17. 17.
    Manisi, P., Jantjies, M., Kimani, L.: A conceptual integrated model for measuring the success of elearning in developing countries: literature review. In: 2018 IST-Africa Week Conference (IST-Africa), pp. 1–9, May 2018Google Scholar
  18. 18.
    Mitnick, K.D., Simon, W.L.: The Art of Deception: Controlling the Human Element of Security. Wiley, New York (2002)Google Scholar
  19. 19.
    Montaño, D.E., Kasprzyk, D.: Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In: Glanz, K., Rimer, B.K., Viswanath, K. (eds.) Health Behavior, pp. 95–124. APA PsycNet, Washington, DC (2015)Google Scholar
  20. 20.
    Montaño, D.E., Kasprzyk, D.: Theory of reasoned action, theory of planned behavior, and the integrated behavior model. In: Glanz, K., Rimer, B.K., Viswanath, K. (eds.) Health Behavior and Health Education, pp. 67–96. APA PsycNet, Washington, DC (2008)Google Scholar
  21. 21.
    Noesgaard, S.S., Ørngreen, R.: The effectiveness of e-learning: an explorative and integrative review of the definitions, methodologies and factors that promote e-learning effectiveness. Electron. J. e-Learn. 13(4), 278–290 (2015). Scholar
  22. 22.
    Schütz, A.E.: Information security awareness: it’s time to change minds! In: Proceedings of International Conference on Applied Informatics Imagination, Creativity, Design, Development - ICDD 2018. Sibiu, Romania (2018)Google Scholar
  23. 23.
    Sun, P.C., Tsai, R.J., Finger, G., Chen, Y.Y., Yeh, D.: What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput. Educ. 50(4), 1183–1202 (2008). Scholar
  24. 24.
    Tate, M., Sedera, D., McLean, E., Burton-Jones, A.: Information systems success research: the “20-year update?" Panel report from PACIS, 2011. Commun. Assoc. Inf. Syst. 34(1) (2014).
  25. 25.
    Verplanken, B., Aarts, H.: Habit, attitude, and planned behaviour: is habit an empty construct or an interesting case of goal-directed automaticity? Eur. Rev. Soc. Psychol. 10(1), 101–134 (1999). Scholar
  26. 26.
    Wang, Y.S., Wang, H.Y., Shee, D.Y.: Measuring e-learning systems success in an organizational context: Scale development and validation. Comput. Hum. Behav. 23(4), 1792–1808 (2007). Scholar
  27. 27.
    Weber, K., Schütz, A.E.: ISIS12-Hack: Mitarbeitersensibilisierenstatt informieren. In: Drews, P., Funk, B., Niemeyer, P., Xie, L. (eds.) Multikonferenz Wirtschsinformatik 2018, vol. IV, pp. 1737–1748, Lüneburg, Germany (2018)Google Scholar
  28. 28.
    Wolf, M.: Von security awareness zum secure behaviour. Hakin9 Extra 5, 18–19 (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tobias Fertig
    • 1
    Email author
  • Andreas E. Schütz
    • 1
  • Kristin Weber
    • 1
  • Nicholas H. Müller
    • 1
  1. 1.Faculty of Computer Science and Business Information SystemsUniversity of Applied Sciences Würzburg-SchweinfurtWürzburgGermany

Personalised recommendations