Abstract
In this chapter, we carry out a critical review of the various historical analyses of the impact of technological interventions in education. The purpose is to analytically explore and learn from some of the methodological limitations and strengths of the approaches adopted to measure and capture the impact of educational technology. This retrospective examination is then used to explicate methodological design principles that can increase the use and value of research evidence regarding the impact of educational technology. Capturing and understanding the impact of digital technologies in and on learning is inherently problematic. It is exacerbated by the continually developing nature of digital technologies and their formal and informal boundary crossing. We posit methodological design principles that are sympathetic to the fact that evidence of the pedagogical application of digital technologies is both borne out of, and brought to bear in, complex and dynamic contexts that are mediated by, and impact upon, the various ways in which technologies are appropriated for educational purpose. The chapter concludes with a call for methodological perspectives that are not confined by paradigm, but are able to bridge and integrate research paradigms in order to respond to the complex sociocultural ecologies within which digital technologies are implicated.
References
Alexander, R. (2015). Teaching and learning for all? The quality imperative revisited. International Journal of Educational Development, 40, 250–258.
Bachmair, B., & Pachler, N. (2015). Sustainability for innovative education – The case of mobile learning. Journal of Interactive Media in Education, 1(17), 1–12. https://doi.org/10.5334/jime.ay.
Bai, Y., Mo, D., Zhang, L., Boswell, M., & Rozelle, S. (2016). The impact of integrated ICT with teaching: Evidence from a randomized controlled trial in rural schools in China. Computers & Education, 96, 1–14. https://doi.org/10.1016/j.compedu.2016.02.005.
Bebell, D., O’Dwyer, L., Russell, M., & Hoffmann, T. (2007) Advancing data collection in the digital age: Methodological challenges and solutions in educational technology research. In Paper presented at the annual meeting of the American Educational Research Association, Chicago. http://www.bc.edu/research/intasc/PDF/Methodological%20challenges_v2.2.pdf
Beland, L., & Murphy, R. (2015). Communication: Technology, distraction & student performance. In Discussion paper no 1350. LSE: Centre for Economic Performance.
Bruce, B., & Hogan, M. (1998). The disappearance of technology: Toward an ecological model of literacy. In D. Reinking, M. McKenna, L. Labbo, & R. Kieffer (Eds.), Handbook of literacy and technology: Transformations in a post-typographic world (pp. 269–281). Mahwah: Lawrence Erlbaum Associates.
Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30.
Cox, M. (2013). Formal to informal learning with IT: Research challenges and issues for e-learning. Journal of Computer Assisted Learning, 29(1), 85–105.
Cox, M. (in press). Researching IT in education: Meeting the challenges of an ever-changing environment. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.
Cox, M., & Abbott, C. (2004). ICT and attainment: A review of the research literature. Coventry: Becta.
Cox, M., & Marshall, G. (2007). Effects of ICT: Do we know what we should know? Education and Information Technologies, 12(2), 59–70.
Cox, M., Webb, M., Abbott, C., Blakeley, B., Beauchamp, T., & Rhodes, V. (2004). ICT and pedagogy: A review of the research literature. Coventry: Becta.
Crook, C., & Garratt, D. (2011). The positivist paradigm in contemporary social research: The interface of psychology method and socio-cultural theory. In B. Somekh & C. Lewin (Eds.), Theory and methods in social research (2nd ed., pp. 212–219). London: Sage.
Crook, C., Harrison, C., Farrington-Flint, L., Tomás, C., & Underwood, J. (2010). The impact of technology: Value-added classroom practice. Final report. Coventry: Becta.
Definition and Terminology Committee of the Association for Educational Communications and Technology. (2008). Definition. In A. Januszewski & M. Molena (Eds.), Educational technology: A definition with commentary (pp. 1–14). New York: Routledge.
Ertmer, P., Addison, P., Lane, M., Ross, E., & Woods, D. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54–71.
Harrison, C., Comber, C., Fisher, T., Haw, K., Lewin, C., Lunzer, E., McFarlane, A., Mavers, D., Scrimshaw, P., Somekh, B., & Watling, R. (2002). ImpacCT2: The impact of information and communication technologies on pupil learning and attainment. London: DfES and Becta. http://dera.ioe.ac.uk/1572/.
Haßler, B., Major, L., & Hennessy, S. (2015). Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139–156.
Hattie, J. (2009). Visible learning; A synthesis of over 800 meta analyses relating to achievement. London: Routledge.
Hew, K., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223–252.
Higgins, S., Xiao, Z., Katsipataki, M. (2012). The impact of digital technology on learning: Summary for the education endowment foundation. Full report. https://educationendowmentfoundation.org.uk/evidence/teaching-learning-toolkit/digital-technology/
ICF Consulting. (2015). Literature review on the impact of technology on learning and teaching. Edinburgh: The Scottish Government. http://dera.ioe.ac.uk/id/eprint/24843
Jenkinson, J. (2009). Measuring the effectiveness of educational technology: What are we attempting to measure? Electronic Journal of e-learning, 7(3), 273–280.
Kalantzis, M., & Cope, B. (2004). Designs for learning. e-Learning, 1(1), 38–93.
Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media and Technology, 39(1), 6–36.
Kirschner, P. (2006). (Inter)dependent learning: Learning is interaction. Utrecht: Inaugural professorial address. http://dspace.learningnetworks.org/bitstream/1820/1619/1/OratieKirschner-LearningisInteraction.pdf.
Kirschner, P., & Kester, L. (2016). Towards a research agenda for educational technology research. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 523–541). Hoboken: Wiley.
Kjeldskov, J., & Graham, C. (2003). A review of Mobile HCI research methods. In Lecture notes in computer science: Human-computer interaction with mobile devices. 5th International Symposium, Mobile HCI 2003 (pp. 317–335). Berlin: Springer.
Latchem, C. (2014). Opening up the educational technology research agenda. British Journal of Educational Technology, 45(1), 3–11.
Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learning and technology. New York/Oxon: Routledge.
Laurillard, D. (in press). Teaching as a design science: Teachers building, testing and sharing pedagogic ideas. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.
Liao, Y.-K. C., & Lai, W.-C. (in press). Meta-analyses of large scale data sets: A tool for assessing the impact of ICT in education. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.
Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38(1), 9–24.
Luckin, R., Bligh, B., Manches, A., Ainsworth, S., Crook, C., & Noss, R. (2012). Decoding learning: The proof, promise and potential of digital education. London: Nesta. http://www.nesta.org.uk/sites/default/files/decoding_learning_report.pdf.
Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229.
OECD. (2015). Students, computers and learning. Making the connection, PISA. Paris: OECD Publishing.
Pachler, N. (2009). Research methods in mobile and informal learning: Some issues. In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research design (pp. 1–15). Oxford: Peter Lang.
Pachler, N., & Daly, C. (2009). Narrative and learning with web 2.0 technologies: Towards a research agenda. Journal of Computer Assisted Learning, 25, 6–18.
Pachler, N., Cook, J., & Bradley, C. (2009). “I don’t really see it”: Whither case-based approaches to understanding off-site and on-campus mobile learning? In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research designs (pp. 77–95). Oxford: Peter Lang Publishing.
Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. New York: Springer.
Pérez-Sanagustín, M., Nussbaum, M., Hillinger, I., Alario-Hoyos, C., Heller, R., Twining, P., & Tsai, C.-C. (2017). Research on ICT in K-12 schools – A review of experimental and survey-based studies in computers & education 2011–2015. Computers & Education, 104, A1–A15.
Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248.
Reeves, T., & Oh, E. (2016). The goals and methods of educational technology research over a quarter century (1989-2014). Educational Technology Research and Development, 65(2), 325–339. https://doi.org/10.1007/s11423-016-9474-1.
Ross, S., Morrison, G., & Lowther, D. (2010). Educational technology research past and present: Balancing rigor and relevance to impact school learning. Contemporary Educational Technology, 1(1), 17–35.
Scardamalia, M. (2006). Technology for understanding. In K. Leithwood, P. McAdie, N. Bascia, & A. Rodrigue (Eds.), Teaching for deep understanding. What every educator needs to know (pp. 103–109). Thousand Oaks: Corwin Press.
Sibbald, B., & Roland, M. (1998). Understanding controlled trials: Why are randomised controlled trials important? BMJ, 316, 201. https://doi.org/10.1136/bmj.316.7126.201.
Somekh, B. (2007). Pedagogy and learning with ICT: Researching the art of innovation. London: Routledge.
Somekh, B., Lewin, C., Mavers, D., Fisher, T., Harrison, C., Haw, K., Lunzer, E., McFarlane, A., & Scrimshaw, P. (2002). ImpaCT2: Pupils’ and teachers’ perceptions in the home, school and community. Coventry: Becta. http://dera.ioe.ac.uk/1573/.
Tamim, R., Bernard, R., Borokhovski, E., Abrami, P., & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.
Turkle, S. (1995). Life on the screen: Identity in the age of the internet. New York: Simon & Schuster.
Turvey, K. (2012). Constructing narrative ecologies as a site for teachers’ professional learning with new technologies and media in primary education. E-learning & Digital Media, 9(1), 113–126.
Turvey, K., & Pachler, N. (2016). Problem spaces’: A framework and questions for critical engagement with learning technologies in formal educational contexts. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 113–130). New York: Wiley-Blackwell.
Twining, P., Heller, R., Nussbaum, M., & Tsai, C.-C. (2017). Some guidance on conducting and reporting qualitative studies. Computers & Education, 106, A1–A9. https://doi.org/10.1016/j.compedu.2016.12.002.
UNESCO [United Nations Educational, Scientific and Cultural Organisation]. (2014). Teaching and learning: Achieving quality for all. In EFA global monitoring report. Paris: UNESCO. http://en.unesco.org/gem-report/report/2014/teaching-and-learning-achieving-quality-all#sthash.2hVKuzp5.dpbs.
VasabØ, K., & Gudmundsdottir, G. (2014). Methodological challenges when exploring new learning sites in educational research. In G. Gudmundsdottir & K. VasabØ (Eds.), Methodological challenges when exploring digital learning spaces in education (pp. 1–12). Rotterdam: Sense Publishers.
Voogt, J., & Knezek, G. (2008). IT in primary and secondary education: Emerging issues. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (Vol. 1, pp. 117–132). New York: Springer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this entry
Cite this entry
Pachler, N., Turvey, K. (2018). Looking Back, Moving Forward: Impact and Measurability of the Use of Educational Technology. In: Voogt, J., Knezek, G., Christensen, R., Lai, KW. (eds) Second Handbook of Information Technology in Primary and Secondary Education . Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-319-53803-7_82-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-53803-7_82-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53803-7
Online ISBN: 978-3-319-53803-7
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education