Evaluating the Acceptance of e-Learning Systems via Subjective and Objective Data Analysis

  • Imed BouchrikaEmail author
  • Nouzha Harrati
  • Zohra Mahfouf
  • Noureddine Gasmallah
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 11)


The adoption of e-learning technology by the academic community, has been a long source of research from multiple disciplines including education, psychology and computer science. As more and more academic institutions have opted to use online technology for their course delivery and pedagogical activities, there has been a surge of interest in evaluating the acceptance of the academic community to adopt and accept the use of e-learning management systems. This is due to the increasing concerns that despite the wide use and deployment of e-learning technologies, the intended impact on education is not achieved. We review the conducted studies on the use of objective procedures for evaluating e-learning systems in tandem with subjective data analysis. The evaluation process consists of understanding further the factors related to the acceptance and adoption of online educational systems by instructors and students in order to devise strategies for improving the teaching and research quality.


e-learning Usability evaluation Learning management system Subjective evaluation Objective evaluation e-learning adoption 


  1. Ajzen, I., & Fishbein, M. (1975). Belief, attitude, intention and behavior: An introduction to theory and research.Google Scholar
  2. Albert, W., & Tullis, T. ( 2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics. Newnes.Google Scholar
  3. Asarbakhsh, M., & Sandars, J. (2013). E-learning: The essential usability perspective. Clin. Teach., 10(1), 47–50.CrossRefGoogle Scholar
  4. Atterer, R., & Schmidt, A. (2007). Tracking the interaction of users with AJAX applications for usability testing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1347–1350). ACM.Google Scholar
  5. Barlow, K., & Lane, J. (2007). Like technology from an advanced alien culture: Google apps for education at ASU. In Proceedings of the 35th Annual ACM SIGUCCS Fall Conference (pp. 8–10). ACM.Google Scholar
  6. Bringula, R. P. (2013). Influence of faculty-and web portal design-related factors on web portal usability: A hierarchical regression analysis. Computers & Education, 68, 187–198.CrossRefGoogle Scholar
  7. Brooke, J. (1996). SUS—A quick and dirty usability scale. In Usability evaluation in industry (Vol. 189, no. 194, pp. 4–7).Google Scholar
  8. Burton-Jones, A., & Grange, C. (2012). From use to effective use: A representation theory perspective. Information Systems Research, 24(3), 632–658.CrossRefGoogle Scholar
  9. Cassino, R., & Tucci, M. (2011). Developing usable web interfaces with the aid of automatic verification of their formal specification. Journal of Visual Languages & Computing, 22(2), 140–149.CrossRefGoogle Scholar
  10. Cassino, R., Tucci, M., Vitiello, G., & Francese, R. (2015). Empirical validation of an automatic usability evaluation method. Journal of Visual Languages & Computing, 28, 1–22.CrossRefGoogle Scholar
  11. Chua, B. B., & Dyson, L. E. (2004). Applying the ISO 9126 model to the evaluation of an e-learning system. In Proceedings of ASCILITE (pp. 5–8).Google Scholar
  12. D’Ambra, J., Wilson, C. S., & Akter, S. (2013). Application of the task-technology fit model to structure and evaluate the adoption of e-books by academics. Journal of the Association for Information Science and Technology, 64(1), 48–64.CrossRefGoogle Scholar
  13. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.CrossRefGoogle Scholar
  14. de Santana, V. F., & Baranauskas, M. C. C. (2015). Welfit: A remote evaluation tool for identifying web usage patterns through client-side logging. International Journal of Human-Computer Studies, 76, 40–49.CrossRefGoogle Scholar
  15. de Vasconcelos, L. G., & Baldochi Jr., L. A. (2012). Towards an automatic evaluation of web applications. In Proceedings of the 27th Annual ACM Symposium on Applied Computing (pp. 709–716). ACM.Google Scholar
  16. Dimoka, A., Banker, R. D., Benbasat, I., Davis, F. D., Dennis, A. R., Gefen, D., et al. (2012). On the use of neurophysiological tools in is research: Developing a research agenda for neurois. MIS Quarterly, 36(3), 679–702.Google Scholar
  17. Dishaw, M. T., & Strong, D. M. (1999). Extending the technology acceptance model with task-technology fit constructs. Information & Management, 36(1), 9–21.CrossRefGoogle Scholar
  18. Eckhardt, A., Maier, C., & Buettner, R. (2012). The influence of pressure to perform and experience on changing perceptions and user performance: A multi-method experimental analysis. In ICIS 2012 Proceedings.Google Scholar
  19. Escobar-Rodriguez, T., & Monge-Lozano, P. (2012). The acceptance of moodle technology by business administration students. Computers & Education, 58(4), 1085–1093.CrossRefGoogle Scholar
  20. Garrison, D. R. (2011). E-learning in the 21st century: A framework for research and practice. Taylor & Francis.Google Scholar
  21. Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 213–236.Google Scholar
  22. Harrati, N., Bouchrika, I., & Mahfouf, Z. (2016). Investigating the uptake of educational systems by academics using the technology to performance chain model. Library Hi Tech, 35(4).Google Scholar
  23. Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2015). Automating the evaluation of usability remotely for web applications via a model-based approach. In 2015 First International Conference on New Technologies of Information and Communication (NTIC) (pp. 1–6). IEEE.Google Scholar
  24. Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61, 463–471.CrossRefGoogle Scholar
  25. Henriksson, A., Yi, Y., Frost, B., & Middleton, M. (2007). Evaluation instrument for e-government websites. Electronic Government, an International Journal, 4(2), 204–226.CrossRefGoogle Scholar
  26. Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102.CrossRefGoogle Scholar
  27. Hrtoňová, N., Kohout, J., Rohlíková, L., & Zounek, J. (2015). Factors influencing acceptance of e-learning by teachers in the Czech Republic. Computers in Human Behavior, 51, 873–879.CrossRefGoogle Scholar
  28. Hsu, C.-L., & Lin, J. C.-C. (2008). Acceptance of blog usage: The roles of technology acceptance, social influence and knowledge sharing motivation. Information & Management, 45(1), 65–74.MathSciNetCrossRefGoogle Scholar
  29. Ismailova, R. (2017). Web site accessibility, usability and security: A survey of government web sites in Kyrgyz Republic. Universal Access in the Information Society, 16(1), 257–264.CrossRefGoogle Scholar
  30. Ivory, M. Y., & Hearst, M. A. (2001). The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys, 33(4), 470–516.CrossRefGoogle Scholar
  31. Joo, S., & Choi, N. (2015). Factors affecting undergraduates selection of online library resources in academic tasks: Usefulness, ease-of-use, resource quality, and individual differences. Library Hi Tech, 33(2), 272–291.CrossRefGoogle Scholar
  32. Laurillard, D., Oliver, M., Wasson, B., & Hoppe, U. (2009). Implementing technology-enhanced learning. In Technology-enhanced learning (pp. 289–306). Springer.Google Scholar
  33. Liapis, A., Katsanos, C., Sotiropoulos, D., Xenos, M., & Karousos, N. (2015). Recognizing emotions in human computer interaction: Studying stress using skin conductance. In Human-Computer Interaction–INTERACT 2015 (pp. 255–262). Springer.Google Scholar
  34. Liaw, S.-S., Huang, H.-M., & Chen, G.-D. (2007). Surveying instructor and learner attitudes toward e-learning. Computers & Education, 49(4), 1066–1080.CrossRefGoogle Scholar
  35. Marco, F. A., Penichet, V. M. R., & Gallud, J. A. (2013). Collaborative e-learning through drag and share in synchronous shared workspaces. Journal of UCS, 19(7), 894–911.Google Scholar
  36. Mayhew, D. J. (1999). The usability engineering lifecycle. In CHI’99 Extended Abstracts on Human Factors in Computing Systems (pp. 147–148). ACM.Google Scholar
  37. McGill, T. J., & Hobbs, V. (2008). How students and instructors using a virtual learning environment perceive the fit between technology and task. Journal of Computer Assisted Learning, 24(3), 191–202.CrossRefGoogle Scholar
  38. McGill, T. J., & Klobas, J. E. (2009). A task-technology fit view of learning management system impact. Computers & Education, 52(2), 496–508.CrossRefGoogle Scholar
  39. McGill, T. J., Klobas, J. E., & Renzi, S. (2014). Critical success factors for the continuation of e-learning initiatives. The Internet and Higher Education, 22, 24–36.CrossRefGoogle Scholar
  40. McGill, T., Klobas, J., & Renzi, S. (2011). Lms use and instructor performance: The role of task-technology fit. International Journal on E-Learning, 10(1), 43–62.CrossRefGoogle Scholar
  41. Mott, J. (2010). Envisioning the post-LMS era: The open learning network. Educause Quarterly, 33(1), 1–9.MathSciNetGoogle Scholar
  42. Navimipour, N. J., & Zareie, B. (2015). A model for assessing the impact of e-learning systems on employees satisfaction. Computers in Human Behavior, 53, 475–485.CrossRefGoogle Scholar
  43. Orfanou, K., Tselios, N., & Katsanos, C. (2015). Perceived usability evaluation of learning management systems: Empirical evaluation of the system usability scale. The International Review of Research in Open and Distributed Learning, 16(2).Google Scholar
  44. Paganelli, L., & Paternò, F. (2002). Intelligent analysis of user interactions with web applications. In International Conference on Intelligent User Interfaces (pp. 111–118).Google Scholar
  45. Paternò, F., Santoro, C., & Spano, L. D. (2012). Improving support for visual task modelling. In Human-centered software engineering (pp. 299–306). Springer.Google Scholar
  46. Pentland, B. T. (1989). Use and productivity in personal computing: An empirical test. In Proceedings of the Tenth International Conference on Information Systems, MA, Boston (pp. 211–222).Google Scholar
  47. Persico, D., Manca, S., & Pozzi, F. (2014). Adapting the technology acceptance model to evaluate the innovative potential of e-learning systems. Computers in Human Behavior, 30, 614–622.CrossRefGoogle Scholar
  48. Phillips, L. A., Calantone, R., & Lee, M.-T. (1994). International technology adoption: Behavior structure, demand certainty and culture. Journal of Business & Industrial Marketing, 9(2), 16–28.CrossRefGoogle Scholar
  49. Phillips, R., McNaught, C., & Kennedy, G. (2012). Evaluating e-learning: Guiding research and practice. Routledge.Google Scholar
  50. Quade, M., Lehmann, G., Engelbrecht, K.-P., Roscher, D., & Albayrak, S. (2013). Automated usability evaluation of model-based adaptive user interfaces for users with special and specific needs by simulating user interaction. In User modeling and adaptation for daily routines (pp. 219–247). Springer.Google Scholar
  51. Raven, A., Leeds, E. M., & Park, C. (2010). Digital video presentation and student performance: A task technology fit perspective. International Journal of Information and Communication Technology Education, 6(1), 17.CrossRefGoogle Scholar
  52. Renaut, C., Batier, C., Flory, L., & Heyde, M. (2006). Improving web site usability for a better e-learning experience. In Current developments in technology-assisted education (pp. 891–895). Badajoz, Spain: FORMATEX.Google Scholar
  53. Roca, J. C., & Gagné, M. (2008). Understanding e-learning continuance intention in the workplace: A self-determination theory perspective. Computers in Human Behavior, 24(4), 1585–1604.CrossRefGoogle Scholar
  54. Rogers, E. M. (2010). Diffusion of innovations. Simon and Schuster.Google Scholar
  55. Rogers Everett, M. (1995). Diffusion of innovations 12. New York.Google Scholar
  56. Sauro, J., & Lewis, J. R. (2009). Correlations among prototypical usability metrics: Evidence for the construct of usability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1609–1618). ACM.Google Scholar
  57. Simões, A. P., & de Moraes, A. (2012). The ergonomic evaluation of a virtual learning environment usability. Work-Journal of Prevention Assessment and Rehabilitation, 41, 1140.Google Scholar
  58. Straub, E. T. (2009). Understanding technology adoption: Theory and future directions for informal learning. Review of Educational Research, 79(2), 625–649.CrossRefGoogle Scholar
  59. Šumak, B., HeričKo, M., & PušNik, M. (2011). A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types. Computers in Human Behavior, 27(6), 2067–2077.CrossRefGoogle Scholar
  60. Szajna, B. (1996). Empirical evaluation of the revised technology acceptance model. Management Science, 42(1), 85–92.CrossRefGoogle Scholar
  61. Taylor, S., & Todd, P. A. (1995). Understanding information technology usage: A test of competing models. Information Systems Research, 6(2), 144–176.CrossRefGoogle Scholar
  62. Tiedtke, T., Märtin, C., & Gerth, N. (2002). AWUSA—A tool for automated website usability analysis. In Workshop on Interactive Systems. Design, Specification, and Verification. Rostock, Germany, June (pp. 12–14).Google Scholar
  63. Tsai, P. C.-F., Yen, Y.-F., Huang, L.-C., & Huang, C. (2007). A study on motivating employees learning commitment in the post-downsizing era: Job satisfaction perspective. Journal of World Business, 42(2), 157–169.CrossRefGoogle Scholar
  64. Vaquero, L. M., Rodero-Merino, L., Caceres, J., & Lindner, M. (2008). A break in the clouds: Towards a cloud definition. ACM SIGCOMM Computer Communication Review, 39(1), 50–55.CrossRefGoogle Scholar
  65. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204.CrossRefGoogle Scholar
  66. Venkatesh, V., & Morris, M. G. (2000). Why don’t men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behavior. MIS Quarterly, 115–139.Google Scholar
  67. Welsh, E. T., Wanberg, C. R., Brown, K. G., & Simmering, M. J. (2003). E-learning: Emerging uses, empirical results and future directions. International Journal of Training and Development, 7(4), 245–258.CrossRefGoogle Scholar
  68. Yi, Y. J., You, S., & Bae, B. J. (2016). The influence of smartphones on academic performance: The development of the technology-to-performance chain model. Library Hi Tech, 34(3), 480–499.Google Scholar
  69. Zhang, D., & Nunamaker, J. F. (2003). Powering e-learning in the new millennium: An overview of e-learning and enabling technology. Information Systems Frontiers, 5(2), 207–218.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Imed Bouchrika
    • 1
    Email author
  • Nouzha Harrati
    • 1
  • Zohra Mahfouf
    • 1
  • Noureddine Gasmallah
    • 1
  1. 1.Faculty of Science & TechnologyUniversity of Souk AhrasSouk AhrasAlgeria

Personalised recommendations