Looking Back, Moving Forward: Impact and Measurability of the Use of Educational Technology

  • Norbert PachlerEmail author
  • Keith Turvey
Reference work entry
Part of the Springer International Handbooks of Education book series (SIHE)


In this chapter, we carry out a critical review of the various historical analyses of the impact of technological interventions in education. The purpose is to analytically explore and learn from some of the methodological limitations and strengths of the approaches adopted to measure and capture the impact of educational technology. This retrospective examination is then used to explicate methodological design principles that can increase the use and value of research evidence regarding the impact of educational technology. Capturing and understanding the impact of digital technologies in and on learning is inherently problematic. It is exacerbated by the continually developing nature of digital technologies and their formal and informal boundary crossing. We posit methodological design principles that are sympathetic to the fact that evidence of the pedagogical application of digital technologies is both borne out of, and brought to bear in, complex and dynamic contexts that are mediated by, and impact upon, the various ways in which technologies are appropriated for educational purpose. The chapter concludes with a call for methodological perspectives that are not confined by paradigm, but are able to bridge and integrate research paradigms in order to respond to the complex sociocultural ecologies within which digital technologies are implicated.


Educational technology Impact Methodological innovation Design principles 


  1. Alexander, R. (2015). Teaching and learning for all? The quality imperative revisited. International Journal of Educational Development, 40, 250–258.CrossRefGoogle Scholar
  2. Bachmair, B., & Pachler, N. (2015). Sustainability for innovative education – The case of mobile learning. Journal of Interactive Media in Education, 1(17), 1–12. Scholar
  3. Bai, Y., Mo, D., Zhang, L., Boswell, M., & Rozelle, S. (2016). The impact of integrated ICT with teaching: Evidence from a randomized controlled trial in rural schools in China. Computers & Education, 96, 1–14. Scholar
  4. Bebell, D., O’Dwyer, L., Russell, M., & Hoffmann, T. (2007) Advancing data collection in the digital age: Methodological challenges and solutions in educational technology research. In Paper presented at the annual meeting of the American Educational Research Association, Chicago.
  5. Beland, L., & Murphy, R. (2015). Communication: Technology, distraction & student performance. In Discussion paper no 1350. LSE: Centre for Economic Performance.Google Scholar
  6. Bruce, B., & Hogan, M. (1998). The disappearance of technology: Toward an ecological model of literacy. In D. Reinking, M. McKenna, L. Labbo, & R. Kieffer (Eds.), Handbook of literacy and technology: Transformations in a post-typographic world (pp. 269–281). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  7. Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30.CrossRefGoogle Scholar
  8. Cox, M. (2013). Formal to informal learning with IT: Research challenges and issues for e-learning. Journal of Computer Assisted Learning, 29(1), 85–105.CrossRefGoogle Scholar
  9. Cox, M., & Abbott, C. (2004). ICT and attainment: A review of the research literature. Coventry: Becta.Google Scholar
  10. Cox, M., & Marshall, G. (2007). Effects of ICT: Do we know what we should know? Education and Information Technologies, 12(2), 59–70.CrossRefGoogle Scholar
  11. Cox, M., Webb, M., Abbott, C., Blakeley, B., Beauchamp, T., & Rhodes, V. (2004). ICT and pedagogy: A review of the research literature. Coventry: Becta.Google Scholar
  12. Crook, C., & Garratt, D. (2011). The positivist paradigm in contemporary social research: The interface of psychology method and socio-cultural theory. In B. Somekh & C. Lewin (Eds.), Theory and methods in social research (2nd ed., pp. 212–219). London: Sage.Google Scholar
  13. Crook, C., Harrison, C., Farrington-Flint, L., Tomás, C., & Underwood, J. (2010). The impact of technology: Value-added classroom practice. Final report. Coventry: Becta.Google Scholar
  14. Definition and Terminology Committee of the Association for Educational Communications and Technology. (2008). Definition. In A. Januszewski & M. Molena (Eds.), Educational technology: A definition with commentary (pp. 1–14). New York: Routledge.Google Scholar
  15. Ertmer, P., Addison, P., Lane, M., Ross, E., & Woods, D. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54–71.CrossRefGoogle Scholar
  16. Harrison, C., Comber, C., Fisher, T., Haw, K., Lewin, C., Lunzer, E., McFarlane, A., Mavers, D., Scrimshaw, P., Somekh, B., & Watling, R. (2002). ImpacCT2: The impact of information and communication technologies on pupil learning and attainment. London: DfES and Becta. Scholar
  17. Haßler, B., Major, L., & Hennessy, S. (2015). Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139–156.CrossRefGoogle Scholar
  18. Hattie, J. (2009). Visible learning; A synthesis of over 800 meta analyses relating to achievement. London: Routledge.Google Scholar
  19. Hew, K., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223–252.CrossRefGoogle Scholar
  20. Higgins, S., Xiao, Z., Katsipataki, M. (2012). The impact of digital technology on learning: Summary for the education endowment foundation. Full report.
  21. ICF Consulting. (2015). Literature review on the impact of technology on learning and teaching. Edinburgh: The Scottish Government.
  22. Jenkinson, J. (2009). Measuring the effectiveness of educational technology: What are we attempting to measure? Electronic Journal of e-learning, 7(3), 273–280.Google Scholar
  23. Kalantzis, M., & Cope, B. (2004). Designs for learning. e-Learning, 1(1), 38–93.Google Scholar
  24. Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media and Technology, 39(1), 6–36.CrossRefGoogle Scholar
  25. Kirschner, P. (2006). (Inter)dependent learning: Learning is interaction. Utrecht: Inaugural professorial address. Scholar
  26. Kirschner, P., & Kester, L. (2016). Towards a research agenda for educational technology research. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 523–541). Hoboken: Wiley.CrossRefGoogle Scholar
  27. Kjeldskov, J., & Graham, C. (2003). A review of Mobile HCI research methods. In Lecture notes in computer science: Human-computer interaction with mobile devices. 5th International Symposium, Mobile HCI 2003 (pp. 317–335). Berlin: Springer.Google Scholar
  28. Latchem, C. (2014). Opening up the educational technology research agenda. British Journal of Educational Technology, 45(1), 3–11.CrossRefGoogle Scholar
  29. Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learning and technology. New York/Oxon: Routledge.Google Scholar
  30. Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38(1), 9–24.CrossRefGoogle Scholar
  31. Luckin, R., Bligh, B., Manches, A., Ainsworth, S., Crook, C., & Noss, R. (2012). Decoding learning: The proof, promise and potential of digital education. London: Nesta. Scholar
  32. Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229.CrossRefGoogle Scholar
  33. OECD. (2015). Students, computers and learning. Making the connection, PISA. Paris: OECD Publishing.CrossRefGoogle Scholar
  34. Pachler, N. (2009). Research methods in mobile and informal learning: Some issues. In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research design (pp. 1–15). Oxford: Peter Lang.Google Scholar
  35. Pachler, N., & Daly, C. (2009). Narrative and learning with web 2.0 technologies: Towards a research agenda. Journal of Computer Assisted Learning, 25, 6–18.CrossRefGoogle Scholar
  36. Pachler, N., Cook, J., & Bradley, C. (2009). “I don’t really see it”: Whither case-based approaches to understanding off-site and on-campus mobile learning? In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research designs (pp. 77–95). Oxford: Peter Lang Publishing.Google Scholar
  37. Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. New York: Springer.CrossRefGoogle Scholar
  38. Pérez-Sanagustín, M., Nussbaum, M., Hillinger, I., Alario-Hoyos, C., Heller, R., Twining, P., & Tsai, C.-C. (2017). Research on ICT in K-12 schools – A review of experimental and survey-based studies in computers & education 2011–2015. Computers & Education, 104, A1–A15.CrossRefGoogle Scholar
  39. Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248.CrossRefGoogle Scholar
  40. Reeves, T., & Oh, E. (2016). The goals and methods of educational technology research over a quarter century (1989-2014). Educational Technology Research and Development, 65(2), 325–339. Scholar
  41. Ross, S., Morrison, G., & Lowther, D. (2010). Educational technology research past and present: Balancing rigor and relevance to impact school learning. Contemporary Educational Technology, 1(1), 17–35.Google Scholar
  42. Scardamalia, M. (2006). Technology for understanding. In K. Leithwood, P. McAdie, N. Bascia, & A. Rodrigue (Eds.), Teaching for deep understanding. What every educator needs to know (pp. 103–109). Thousand Oaks: Corwin Press.Google Scholar
  43. Sibbald, B., & Roland, M. (1998). Understanding controlled trials: Why are randomised controlled trials important? BMJ, 316, 201. Scholar
  44. Somekh, B. (2007). Pedagogy and learning with ICT: Researching the art of innovation. London: Routledge.Google Scholar
  45. Somekh, B., Lewin, C., Mavers, D., Fisher, T., Harrison, C., Haw, K., Lunzer, E., McFarlane, A., & Scrimshaw, P. (2002). ImpaCT2: Pupils’ and teachers’ perceptions in the home, school and community. Coventry: Becta. Scholar
  46. Tamim, R., Bernard, R., Borokhovski, E., Abrami, P., & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.CrossRefGoogle Scholar
  47. Turkle, S. (1995). Life on the screen: Identity in the age of the internet. New York: Simon & Schuster.Google Scholar
  48. Turvey, K. (2012). Constructing narrative ecologies as a site for teachers’ professional learning with new technologies and media in primary education. E-learning & Digital Media, 9(1), 113–126.CrossRefGoogle Scholar
  49. Turvey, K., & Pachler, N. (2016). Problem spaces’: A framework and questions for critical engagement with learning technologies in formal educational contexts. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 113–130). New York: Wiley-Blackwell.CrossRefGoogle Scholar
  50. Twining, P., Heller, R., Nussbaum, M., & Tsai, C.-C. (2017). Some guidance on conducting and reporting qualitative studies. Computers & Education, 106, A1–A9. Scholar
  51. UNESCO [United Nations Educational, Scientific and Cultural Organisation]. (2014). Teaching and learning: Achieving quality for all. In EFA global monitoring report. Paris: UNESCO.
  52. VasabØ, K., & Gudmundsdottir, G. (2014). Methodological challenges when exploring new learning sites in educational research. In G. Gudmundsdottir & K. VasabØ (Eds.), Methodological challenges when exploring digital learning spaces in education (pp. 1–12). Rotterdam: Sense Publishers.Google Scholar
  53. Voogt, J., & Knezek, G. (2008). IT in primary and secondary education: Emerging issues. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (Vol. 1, pp. 117–132). New York: Springer.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of EducationUniversity College LondonLondonUK
  2. 2.University of BrightonBrightonUK

Section editors and affiliations

  • Margaret Cox
    • 1
  • Joke Voogt
    • 2
  1. 1.King's College LondonLondonUK
  2. 2.Department of Child Development and EducationUniversity of Amsterdam/ Windesheim University of Applied SciencesAmsterdamThe Netherlands

Personalised recommendations