Meta-analyses of Large-Scale Datasets: A Tool for Assessing the Impact of Information and Communication Technology in Education

  • Yuen-Kuang Cliff LiaoEmail author
  • Wan-Ching Lai
Reference work entry
Part of the Springer International Handbooks of Education book series (SIHE)


This chapter focuses on the types of research questions that might be answered by using meta-analysis in the area of information communication technology (ICT) in education and the strengths and weaknesses of such meta-analyses. The chapter is divided into four parts: (a) brief description of the meta-analysis research method, (b) synthesis of meta-analysis studies of the impact of ICT in education from 1988 to 2017, particularly focusing on 2007–2017, (c) identifying what kinds of evidence outcomes can be achieved through meta-analysis, and (d) discussion of the strengths and weaknesses of meta-analysis in ICT in education and what are the most reliable methods to obtain accurate evidence about the impact of ICT in education.


Meta-analysis Computer-based instruction Distance education Game-based learning Mobile learning 


  1. Abraham, L. B. (2008). Computer-mediated glosses in second language reading comprehension and vocabulary learning: A meta-analysis. Computer Assisted Language Learning, 21(3), 199–226.CrossRefGoogle Scholar
  2. Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.Google Scholar
  3. Bangert-Drowns, R. (1986). Review of developments in meta-analytic method. Psychological Bulletin, 93, 388–399.CrossRefGoogle Scholar
  4. Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87–122.CrossRefGoogle Scholar
  5. Borokhovski, E., Bernard, R. M., Tamim, R. M., Schmid, R. F., & Sokolovskaya, A. (2016). Technology-supported student interaction in post-secondary education: A meta-analysis of designed versus contextual treatments. Computers & Education, 96, 15–28.CrossRefGoogle Scholar
  6. Brydges, R., Manzone, J., Shanks, D., Hatala, R., Hamstra, S. J., Zendejas, B., & Cook, D. A. (2015). Self-regulated learning in simulation-based training: A systematic review and meta-analysis. Medical Education, 49(4), 368–378.CrossRefGoogle Scholar
  7. Camnalbur, M., & Erdogan, Y. (2008). A meta analysis on the effectiveness of computer-assisted instruction: Turkey sample. Kuram Ve Uygulamada Egitim Bilimleri, 8(2), 497.Google Scholar
  8. Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30.CrossRefGoogle Scholar
  9. Chinnery, G. M. (2006). Emerging technologies going to the mall: Mobile assisted language learning. Language, Learning and Technology, 10(1), 9–16.Google Scholar
  10. Chiu, Y. H. (2013). Computer-assisted second language vocabulary instruction: A meta-analysis. British Journal of Educational Technology, 44(2), E52–E56.CrossRefGoogle Scholar
  11. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459.CrossRefGoogle Scholar
  12. Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.CrossRefGoogle Scholar
  13. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale: Erlbaum.Google Scholar
  14. Cook, D. A. (2014). How much evidence does it take? A cumulative meta-analysis of outcomes of simulation-based education. Medical Education, 48(8), 750–760.CrossRefGoogle Scholar
  15. D’Angelo, C., Rutstein, D., Harris, C., Bernard, R., Borokhovski, E., & Haertel, G. (2014). Simulations for STEM learning: Systematic review and meta-analysis. Menlo Park: SRI International.Google Scholar
  16. Demir, S., & Basol, G. (2014). Effectiveness of computer-assisted mathematics education (CAME) over academic achievement: A meta-analysis study. Educational Sciences: Theory and Practice, 14(5), 2026–2035.Google Scholar
  17. Dýnçer, S. (2015). Effects of computer-assisted learning on students’ achievements in Turkey: A meta-analysis. Journal of Turkish Science Education, 12(1), 98.Google Scholar
  18. Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulations and Games, 33(4), 441–467.CrossRefGoogle Scholar
  19. Gegenfurtner, A., Quesada-Pallarès, C., & Knogler, M. (2014). Digital simulation-based training: A meta-analysis. British Journal of Educational Technology, 45(6), 1097–1114.CrossRefGoogle Scholar
  20. Grgurović, M., Chapelle, C. A., & Shelley, M. C. (2013). A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL, 25(02), 165–198.CrossRefGoogle Scholar
  21. Jahng, N., Krug, D., & Zhang, Z. (2007). Student achievement in online distance education compared to face-to-face education. European Journal of Open, Distance and E-Learning, 10(1). Retrieved April 7, 2018, from
  22. Kim, J. K., & Kim, D. J. (2013). Meta-analysis on relations between e-learning research trends and effectiveness of learning. International Journal of Smart Home, 7(6), 35–48.CrossRefGoogle Scholar
  23. Kulik, J. A., & Fletcher, J. D. (2015). Effectiveness of intelligent tutoring systems a meta-analytic review. Review of Educational Research.
  24. Larwin, K., & Larwin, D. (2011). A meta-analysis examining the impact of computer-assisted instruction on postsecondary statistics education: 40 years of research. Journal of Research on Technology in Education, 43(3), 253–278.CrossRefGoogle Scholar
  25. Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis. International Journal of Instructional Media, 26(1), 71–85.Google Scholar
  26. Lee, Y. H., Waxman, H., Wu, J. Y., Michko, G., & Lin, G. (2013). Revisit the effect of teaching and learning with technology. Educational Technology & Society, 16(1), 133–146.Google Scholar
  27. Lee, Y. S., Sung, Y. T., Chang, K. E., Liu, T. C., & Chen, W. C. (2014, August). A meta-analysis of the effects of learning languages with mobile devices. In International conference on web-based learning (pp. 106–113). Heidelberg: Springer.Google Scholar
  28. Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215–243.CrossRefGoogle Scholar
  29. Liao, Y. K. C., & Bright, G. W. (1991). Effects of computer programming on cognitive outcomes: A meta-analysis. Journal of Educational Computing Research, 7(3), 251–268.CrossRefGoogle Scholar
  30. Liao, Y. C. (2007). Effects of computer assisted instruction on students’ achievement in Taiwan: A meta-analysis. Computers & Education, 48(2), 216–233.CrossRefGoogle Scholar
  31. Liao, Y. C., & Hao, Y. (2008). Large-scale studies and quantitative methods. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (pp. 1019–1036). London: Springer.CrossRefGoogle Scholar
  32. Liao, Y. C., Chang, H., & Chen, Y. (2007). Effects of computer applications on elementary school students’ achievement: A meta-analysis of students in Taiwan. Computers in the Schools, 24(3), 43–64.CrossRefGoogle Scholar
  33. Lin, H. (2014). Establishing an empirical link between computer-mediated communication (CMC) and SLA: A meta-analysis of the research. Language, Learning and Technology, 18(3), 120–147.Google Scholar
  34. Lin, W. C., Huang, H. T., & Liou, H. C. (2013). The effects of text-based SCMC on SLA: A meta-analysis. Language, Learning and Technology, 17(2), 123–142.Google Scholar
  35. Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, DC: U.S. Department of Education.Google Scholar
  36. Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40.CrossRefGoogle Scholar
  37. Mothibi, G. (2015). A meta-analysis of the relationship between e-learning and students’ academic achievement in higher education. Journal of Education and Practice, 6(9), 6–9.Google Scholar
  38. Roberts, R. M. (2011). Best instructional practices for distance education: A meta-analysis (Order No. 3475479). Available from Education Collection; Education Database; ProQuest Dissertations & Theses A&I. (900728274).Google Scholar
  39. Roblyer, M. D., Castine, W. H., & King, F. J. (1988). Assessing the impact of computer-based instruction: A review of recent research. Computers in the Schools, 5(3/4), 1–149.Google Scholar
  40. Schmidt, F. L., & Oh, I. S. (2013). Methods for second order meta-analysis and illustrative applications. Organizational Behavior and Human Decision Processes, 121(2), 204–218.CrossRefGoogle Scholar
  41. Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R. M., Abrami, P. C., Surkes, M. A., & Woods, J. (2014). The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education, 72, 271–291.CrossRefGoogle Scholar
  42. Shachar, M., & Neumann, Y. (2010). Twenty years of research on the academic performance differences between traditional and distance learning: Summative meta-analysis and trend examination. Journal of Online Learning and Teaching, 6(2), 318.Google Scholar
  43. Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64(2), 489–528.CrossRefGoogle Scholar
  44. Sosa, G. W., Berger, D. E., Saw, A. T., & Mary, J. C. (2011). Effectiveness of computer-assisted instruction in statistics a meta-analysis. Review of Educational Research, 81(1), 97–128.CrossRefGoogle Scholar
  45. Sung, Y. T., Chang, K. E., & Yang, J. M. (2015). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review, 16, 68–84.CrossRefGoogle Scholar
  46. Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252–275.CrossRefGoogle Scholar
  47. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning a second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.CrossRefGoogle Scholar
  48. Tekbiyik, A., & Akdeniz, A. R. (2010, December). A meta-analytical investigation of the influence of computer assisted instruction on achievement in science. In Asia-Pacific forum on science learning and teaching (Vol. 11, No. 2, pp. 1–22). Hong Kong Institute of Education. 10 Lo Ping Road, Tai Po, New Territories, Hong Kong.Google Scholar
  49. Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229–243.CrossRefGoogle Scholar
  50. Voutilainen, A., Saaranen, T., & Sormunen, M. (2017). Conventional vs. e-learning in nursing education: A systematic review and meta-analysis. Nurse Education Today, 50, 97–103.CrossRefGoogle Scholar
  51. Walberg, H. J. (1983). Education, scientific literacy, and economic productivity. Daedalus, 112, 1–28.Google Scholar
  52. Wang, T. L., & Tseng, Y. F. (2011, Augest). Learning effect for students with game-based learning on meta-analysis. In Computer Science & Education (ICCSE), 2011 6th international conference on (pp. 1291–1295). IEEE.Google Scholar
  53. Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412–425.CrossRefGoogle Scholar
  54. Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249.CrossRefGoogle Scholar
  55. Yesilyurt, M. (2010). Meta analysis of the computer assisted studies in science and mathematics: A sample of Turkey. The Turkish Online Journal of Educational Technology, 9(1), 123.Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Centre for Teacher Education, Chinese Culture UniversityTaipeiTaiwan
  2. 2.Language Teaching and Learning Centre, Chinese Culture UniversityTaipeiTaiwan

Section editors and affiliations

  • Margaret Cox
    • 1
  • Joke Voogt
    • 2
  1. 1.King's College LondonLondonUK
  2. 2.Department of Child Development and EducationUniversity of Amsterdam/ Windesheim University of Applied SciencesAmsterdamThe Netherlands

Personalised recommendations