Skip to main content
SpringerLink
Log in

Developing a model to assess the success of e-learning systems: evidence from a manufacturing company in transitional economy

  • Original Article
  • Published:
Information Systems and e-Business Management Aims and scope Submit manuscript

Abstract

This paper examines the success of an e-learning system in a company from the perspective of employees by using a multimethod approach. For this purpose Moodle learning management system was used. The success of e-learning as an information system was evaluated using four constructs of the updated DeLone and McLean IS success model—system quality, use, user satisfaction and net benefits, and adding one more construct—user performance. In this research a combination of observation and survey as two different research methods was used, which allowed the new measure to be incorporated into the model. Empirical assessment was carried out by exploratory factor analysis, confirmatory factor analysis and structural equation modeling. The research model was found to be valid and reliable. The results provide an expanded understanding of the constructs that measure the success of an e-learning system, helping to more deeply understand the key success dimensions and their interrelationships. The implications of our work were discussed. The DeLone and McLean IS success model applied equally well. However, the use of observation as a method of data collection revealed the weaknesses of the original model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Ahire SL, Golhar DY, Waller MA (1996) Development and validation of TQM implementation constructs. Decis Sci 27:23–56. doi:10.1111/j.1540-5915.1996.tb00842.x

    Article  Google Scholar 

  • Armstrong CP, Sambamurthy V (1999) Information technology assimilation in firms: the influence of senior leadership and IT infrastructures. Inf Syst Res 10:304–327

    Article  Google Scholar 

  • Asparouhov T, Muthén B (2009) Exploratory structural equation modeling. Struct Equ Model A Multidiscip J 16:397–438. doi:10.1080/10705510903008204

    Article  Google Scholar 

  • Avouris NM, Dimitracopoulou A, Daskalaki S, Tselios NK (2001) Evaluation of distance-learning environments: impact of usability on student performance. Int J Educ Telecommun 7:355–378

    Google Scholar 

  • Balaban I, Mu E, Divjak B (2013) Development of an electronic Portfolio system success model: an information systems approach. Comput Educ 60:396–411

    Article  Google Scholar 

  • Bartlett KR (2005) Survey research in organizations. In: Swanson RA, Holton EF (eds) Res. Organ. Found. Methods Inq. Berrett-Koehler, San Francisco, CA, pp 97–113

  • Benbasat I, Goldstein DK, Mead M (1987) The case research strategy in studies of information systems. MIS Q 11:369–386. doi:10.2307/248684

    Article  Google Scholar 

  • Bentler PM, Bonett DG (1980) Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull 88:588–606

    Article  Google Scholar 

  • Birnbaum MH (2004) Human research and data collection via the internet. Annu Rev Psychol 55:803–832

    Article  Google Scholar 

  • Chen H-J (2012) Clarifying the empirical connection of new entrants’ e-learning systems use to their job adaptation and their use patterns under the collective–individual training environment. Comput Educ 58:321–337

    Article  Google Scholar 

  • Creswell JW, Plano Clark VL (2011) Designing and conducting mixed methods research, 2nd ed. SAGE Publications, Thousand Oaks, CA

  • Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13:319–340

    Article  Google Scholar 

  • Davis D, Golicic S, Boerstler C (2011) Benefits and challenges of conducting multiple methods research in marketing. J Acad Mark Sci 39:467–479. doi:10.1007/s11747-010-0204-7

    Article  Google Scholar 

  • Delic M, Radlovacki V, Kamberovic B et al (2013) Examining relationships between quality management and organisational performance in transitional economies. Total Qual Manag Bus Excell. doi:10.1080/14783363.2013.799331

    Google Scholar 

  • DeLone W, McLean E (1992) Information systems success: the quest for the dependent variable. Inf Syst Res 3:60–95

    Article  Google Scholar 

  • DeLone W, McLean E (2003) The DeLone and McLean model of information systems success : a ten-year update. J Manag Inf Syst 19:9–31

    Google Scholar 

  • DeRouin RE, Fritzsche BA, Salas E (2005) E-Learning in Organizations. J. Manage 31:920–940

    Google Scholar 

  • Dillman DA (2007) Mail and internet surveys: the tailored design method, 2nd edn. John Wiley & Sons Inc, Hoboken

    Google Scholar 

  • Eom S, Ashill NJ, Arbaugh JB, Stapleton JL (2012) The role of information technology in e-learning systems success. Hum Syst Manag 31:147–163. doi:10.3233/HSM-2012-0767

    Google Scholar 

  • Escobar-Rodriguez T, Monge-Lozano P (2012) The acceptance of Moodle technology by business administration students. Comput Educ 58:1085–1093. doi:10.1016/j.compedu.2011.11.012

  • Garcia-Smith D, Effken JA (2013) Development and initial evaluation of the clinical information systems success model (CISSM). Int J Med Inform 82:539–552

    Article  Google Scholar 

  • Hair JF, Black WC, Babin BJ, Anderson RE (2009) Multivariate data analysis, 7th ed. Prentice Hall, Upper Saddle River, NJ

  • Hassanzadeh A, Kanaani F, Elahi S (2012) A model for measuring e-learning systems success in universities. Expert Syst Appl 39:10959–10966

    Article  Google Scholar 

  • Holsapple CW, Lee-Post A (2006) Defining, assessing, and promoting e-learning success: an information systems perspective. Decis Sci J Innov Educ 4:67–85

    Article  Google Scholar 

  • Hou C-K (2012) Examining the effect of user satisfaction on system usage and individual performance with business intelligence systems: an empirical study of Taiwan’s electronics industry. Int J Inf Manage 32:560–573

    Article  Google Scholar 

  • Hu L, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J 6:1–55. doi:10.1080/10705519909540118

    Article  Google Scholar 

  • Kaplan B, Duchon D (1988) Combining qualitative and quantitative methods in information systems research: a case study. MIS Q 12:571–586

    Article  Google Scholar 

  • Kirkpatrick DL, Kirkpatrick JD (2006) Evaluating training programs, 3rd edn. Berrett-Koehler Publishers, San Francisco

    Google Scholar 

  • Kong SC (2011) An evaluation study of the use of a cognitive tool in a one-to-one classroom for promoting classroom-based dialogic interaction. Comput Educ 57:1851–1864. doi:10.1016/j.compedu.2011.04.008

  • Lee J-K, Lee W-K (2008) The relationship of e-Learner’s self-regulatory efficacy and perception of e-Learning environmental quality. Comput Human Behav 24:32–47. doi:10.1016/j.chb.2006.12.001

  • Lee M, Cheung C, Chen Z (2005) Acceptance of Internet-based learning medium: the role of extrinsic and intrinsic motivation. Inf Manag 42:1095–1104. doi:10.1016/j.im.2003.10.007

  • Lin H-F (2007) Measuring online learning systems success: applying the updated DeLone and McLean model. Cyberpsychol Behav 10:817–820

    Article  Google Scholar 

  • Lopez-Fernandez O, Rodriguez-Illera JL (2009) Investigating university students’ adaptation to a digital learner course portfolio. Comput Educ 52:608–616. doi:10.1016/j.compedu.2008.11.003

  • Markus ML, Tanis C (2000) The enterprise system experience—from adoption to success. In: Zmud R (ed) Fram. Domains IT Manag. Proj. Futur. Through Past. Pinnaflex Educational Resources inc., Cincinnati, pp 173–207

    Google Scholar 

  • Martín-Blas T, Serrano-Fernández A (2009) The role of new technologies in the learning process: moodle as a teaching tool in Physics. Comput Educ 52:35–44

    Article  Google Scholar 

  • Mason RO (1978) Measuring information output: A communication systems approach. Inf Manag 1:219–234. doi:10.1016/0378-7206(78)90028-9

  • McGill T, Hobbs V, Klobas J (2003) User developed applications and information systems success. Inf Resour Manag J 16:24–45

    Article  Google Scholar 

  • Means B, Toyama Y, Murphy R, Bakia M, Jones K (2010) Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. Washington, DC: US Department of Education, Office of Planning, Evaluation, and Policy Development

  • Mingers J (2001) Combining IS research methods: towards a pluralist methodology. Inf Syst Res 12:240

    Article  Google Scholar 

  • Mingers J (2003) The paucity of multimethod research: a review of the information systems literature. Inf Syst J 13:233–249

    Article  Google Scholar 

  • Nunnally J, Bernstein H (1994) Psychometric theory. McGraw-Hill, New York

    Google Scholar 

  • Ozkan S, Koseler R (2009) Multi-dimensional students’ evaluation of e-learning systems in the higher education context: an empirical investigation. Comput Educ 53:1285–1296

    Article  Google Scholar 

  • Petter S, McLean ER (2009) A meta-analytic assessment of the DeLone and McLean IS success model: an examination of IS success at the individual level. Inf Manag 46:159–166

    Article  Google Scholar 

  • Petter S, DeLone W, McLean E (2008) Measuring information systems success: models, dimensions, measures, and interrelationships. Eur J Inf Syst 17:236–263. doi:10.1057/ejis.2008.15

    Article  Google Scholar 

  • Pitt LF, Watson RT, Kavan CB (1995) Service quality: a measure of information systems effectiveness. MIS Q 19:173–187

    Article  Google Scholar 

  • Rice W (2006) Moodle e-learning course development. Packt Publishing Ltd., Birmingham

    Google Scholar 

  • Roca JC, Chiu C-M, Martínez FJ (2006) Understanding e-learning continuance intention: an extension of the technology acceptance model. Int J Hum Comput Stud 64:683–696. doi:10.1016/j.ijhcs.2006.01.003

  • Romero C, Ventura S, García E (2008) Data mining in course management systems: Moodle case study and tutorial. Comput Educ 51:368–384. doi:10.1016/j.compedu.2007.05.016

  • Schmitt TA (2011) Current methodological considerations in exploratory and confirmatory factor analysis. J Psychoeduc Assess 29:304–321. doi:10.1177/0734282911406653

    Article  Google Scholar 

  • Seddon PB (1997) A respecification and extension of the DeLone and McLean model of IS success. Inf Syst Res 8:240–253

    Article  Google Scholar 

  • Seddon PB, Kiew M-Y (1996) A partial test and development of Delone and Mclean’s model of IS success. Australas J Inf Syst 4:90–109. doi:10.3127/ajis.v4i1.379

    Google Scholar 

  • Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois Press, Urbana

    Google Scholar 

  • Urbach N, Smolnik S, Riempp G (2009) The state of research on information systems success. Bus Inf Syst Eng 1:315–325. doi:10.1007/s12599-009-0059-y

  • Urbach N, Smolnik S, Riempp G (2010) An empirical investigation of employee portal success. J Strateg Inf Syst 19:184–206

    Article  Google Scholar 

  • Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci 46:186

    Article  Google Scholar 

  • Venkatesh V, Morris MG, Davis GB, Davis FD (2003) User acceptance of information technology: toward a unified view. MIS Q 27:425–478

    Google Scholar 

  • Venkatesh V, Brown SA, Bala H (2013) Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. MIS Q 37:21–54

    Google Scholar 

  • Wang Y-S (2003) Assessment of learner satisfaction with asynchronous electronic learning systems. Inf Manag 41:75–86. doi:10.1016/S0378-7206(03)00028-4

  • Wang Y-S, Liao Y-W (2008) Assessing eGovernment systems success: a validation of the DeLone and McLean model of information systems success. Gov Inf Q 25:717–733

    Article  Google Scholar 

  • Wang Y-S, Wang H-Y, Shee DY (2007) Measuring e-learning systems success in an organizational context: scale development and validation. Comput Hum Behav 23:1792–1808

    Article  Google Scholar 

  • Wu J-H, Wang Y-M (2006) Measuring KMS success: a respecification of the DeLone and McLean’s model. Inf Manag 43:728–739

    Article  Google Scholar 

  • Zhang P (2008) Motivational affordances: fundamental reasons for ICT design and use. Commun ACM 51:145–147. doi:10.1145/1400214.1400244

    Article  Google Scholar 

  • Zhang D, Nunamaker J (2003) Powering e-learning in the new millennium: an overview of e-learning and enabling technology. Inf Syst Front 5:207–218. doi:10.1023/A:1022609809036

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovica 6, 21000, Novi Sad, Serbia

    Ugljesa Marjanovic, Milan Delić & Bojan Lalic

Authors
  1. Ugljesa Marjanovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Milan Delić
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bojan Lalic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ugljesa Marjanovic.

Appendix: Survey items used in this study

Appendix: Survey items used in this study

System quality

SQ1:

The eGovernment system is user friendly

SQ2:

The eGovernment system is easy to use

User satisfaction

US1:

You are satisfied with the content of the e-learning system

US2:

You are satisfied with the instructions on how to use the e-learning system

US3:

You are satisfied with the e-learning environment

US4:

Overall, you are satisfied with this e-learning system

Net benefits

NB1:

The e-learning system helped me to improve my knowledge

NB2:

The e-learning system helped me to improve my attitude towards the organization

Note This appendix presents only survey items. Observation items are not presented here.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marjanovic, U., Delić, M. & Lalic, B. Developing a model to assess the success of e-learning systems: evidence from a manufacturing company in transitional economy. Inf Syst E-Bus Manage 14, 253–272 (2016). https://doi.org/10.1007/s10257-015-0282-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10257-015-0282-7

Keywords

Search

Navigation