Quantitative Methodology

  • Luke K. Fryer
  • Jenifer Larson-HallEmail author
  • Jeffrey Stewart


This chapter explores the methodological choices made in an illustrative complex and longitudinal study of classroom interest in a language task. They walk the reader through choices that must be made in a quantitative analysis step by step while also advocating for best practices in quantitative research, such as using technology as a partner in research methodology, strengthening statistical power by repeated testing of the same participants, and strengthening validity of study results by using a longitudinal design. The chapter’s aim is not to provide a comprehensive treatment of all possible methodological choices the reader may make, but to instead make vivid for the reader how such choices are made by teacher practitioners conducting actual research projects.


Quantitative research Methodology Statistics Longitudinal Technology Motivation Classroom language learning Chatbots 


  1. Blom, E., & Unsworth, S. (Eds.). (2010). Experimental methods in language acquisition research. Amsterdam: John Benjamins.Google Scholar
  2. Brown, J. D. (2004). Research methods for applied linguistics: Scope, characteristics, and standards. In A. Davies & C. Elder (Eds.), The handbook of applied linguistics (pp. 476–501). Oxford: Blackwell Publishing Ltd.CrossRefGoogle Scholar
  3. Brown, J. D. (2014). Mixed methods research for TESOL. Edinburgh: Edinburgh University Press.Google Scholar
  4. Brown, J. D. (2015). Why bother learning advanced quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 9–20). New York, NY: Routledge.CrossRefGoogle Scholar
  5. Brown, J. D., & Rodgers, T. S. (2003). Doing second language research. Oxford: Oxford University Press.Google Scholar
  6. Brown, T. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.Google Scholar
  7. Chatterjee, S., & Hadi, A. S. (2015). Regression analysis by example. New York: John Wiley & Sons.Google Scholar
  8. Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.Google Scholar
  9. Devellis, R. F. (2012). Scale development: Theory and application (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
  10. Dewey, J. (1913). Interest and effort in education. Boston: Houghton Mifflin Company.CrossRefGoogle Scholar
  11. Duff, P. (2010). Research approaches in applied linguistics. In R. B. Kaplan (Ed.), Oxford handbook of applied linguistics (2nd ed., pp. 45–59). New York: Oxford University Press.Google Scholar
  12. Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 56–83.CrossRefGoogle Scholar
  13. Fryer, L. K., Ainley, M., & Thompson, A. (2016). Modelling the links between students’ interest in a domain, the tasks they experience and their interest in a course: Isn’t interest what university is all about? Learning and Individual Differences, 50, 57–165.CrossRefGoogle Scholar
  14. Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of AI and Human task partners. Computers in Human Behavior. Scholar
  15. Fryer, L. K., & Carpenter, R. (2006). Bots as language learning tools. Language Learning & Technology, 10(3), 8–14.Google Scholar
  16. Fryer, L. K., & Nakao, K. (2009). Online English practice for Japanese University students: Assessing chatbots. Paper presented at the Japan Association for Language Teaching national conference, Tokyo. Permanent Online Location:
  17. Hattie, J. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London and New York: Routledge and Taylor & Francis.Google Scholar
  18. Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.CrossRefGoogle Scholar
  19. Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in Human Behavior, 49, 245–250.CrossRefGoogle Scholar
  20. Hudson, T. (2015). Presenting quantitative data visually. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 78–105). New York: Routledge.CrossRefGoogle Scholar
  21. Hudson, T., & Llosa, L. (2015). Design issues and inference in experimental L2 research. Language Learning, 65(S1), 76–96.CrossRefGoogle Scholar
  22. Jang, E. E., Wagner, M., & Park, G. (2014). Mixed methods research in language testing and assessment. Annual Review of Applied Linguistics, 34, 123–153.CrossRefGoogle Scholar
  23. Jia, J., & Chen, W. (2008). Motivate the learners to practice English through playing with Chatbot CSIEC. In Z. Pan, X. Zhang, A. Rhalib, W. Woo, & Y. Li (Eds.), Technologies for E-learning and digital entertainment (pp. 180–191). New York: Springer.CrossRefGoogle Scholar
  24. Kline, R. B. (2011). Principles and practices of structural equation modeling (3rd ed.). New York: Guilford Press.Google Scholar
  25. Larson-Hall, J. (2015). A guide to doing statistics in second language research using SPSS and R (2nd ed.). New York: Routledge.CrossRefGoogle Scholar
  26. Larson-Hall, J., & Herrington, R. (2009). Improving data analysis in second language acquisition by utilizing modern developments in applied statistics. Applied Linguistics, 31(3), 368–390.CrossRefGoogle Scholar
  27. Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(S1), 127–159.CrossRefGoogle Scholar
  28. Loehlin, J. C. (1998). Latent variable models: An introduction to factor, path, and structural analysis. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.Google Scholar
  29. Loewen, S., & Gass, S. M. (2009). Research timeline: The use of statistics in L2 acquisition research. Language Teaching, 42(2), 181–196.CrossRefGoogle Scholar
  30. Mackey, A., & Gass, S. M. (Eds.). (2011). Research methods in second language acquisition: A practical guide. Oxford: Wiley-Blackwell.Google Scholar
  31. Norris, J. M. (2015). Statistical significance testing in second language research: Basic problems and suggestions for reform. Language Learning, 65(S1), 97–126.CrossRefGoogle Scholar
  32. Norris, J. M., Ross, S. J., & Schoonen, R. (2015). Improving second language quantitative research. Language Learning, 65(S1), 1–8.CrossRefGoogle Scholar
  33. Ortega, L., & Byrnes, H. (2008). The longitudinal study of advanced L2 capacities: An introduction. In L. Ortega & H. Byrnes (Eds.), The longitudinal study of advanced L2 capacities (pp. 3–20). New York: Routledge.Google Scholar
  34. Ortega, L., & Iberri-Shea, G. (2005). Longitudinal research in second language acquisition: Recent trends. Annual Review of Applied Linguistics, 25, 26–45.CrossRefGoogle Scholar
  35. Paltridge, B., & Phakiti, A. (Eds.). (2015). Research methods in applied linguistics: A practical resource (2nd ed.). London: Bloomsbury Academic.Google Scholar
  36. Pedhazur, E. J. (1982). Multiple regression in behavioral research (2nd ed.). New York: Holt, Rinehart and Winston.Google Scholar
  37. Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90(1), 175–181.CrossRefGoogle Scholar
  38. Phakiti, A. (2014). Experimental research methods in language learning. London: Bloomsbury Academic.Google Scholar
  39. Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687.CrossRefGoogle Scholar
  40. Plonsky, L., & Oswald, F. L. (2014). Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912.CrossRefGoogle Scholar
  41. Porte, G. K. (2010). Appraising research in second language learning: A practical approach to critical analysis of quantitative research (2nd ed.). Philadelphia: John Benjamins.CrossRefGoogle Scholar
  42. Roever, C., & Phakiti, A. (2018). Quantitative methods for second language research: A problem-solving approach. London and New York: Routledge.Google Scholar
  43. Schiefele, U. (1991). Interest, learning, and motivation. Educational Psychologist, 26(3&4), 299–323.CrossRefGoogle Scholar
  44. Tobias, S. (1995). Interest and metacognitive word knowledge. Journal of Educational Psychology, 87(3), 399–405.CrossRefGoogle Scholar
  45. Tracz, S. M. (1992). The interpretation of beta weights in path analysis. Multiple Linear Regression Viewpoints, 19, 7–15.Google Scholar
  46. Tufte, E. (2001). The visual display of quantitative information. Cheshire, CT: Graphics Press.Google Scholar
  47. Tukey, J. W. (1960). A survey of sampling from contaminated distributions. In I. Olkin, S. G. Ghurye, W. Hoeffding, W. G. Madow, & H. B. Mann (Eds.), Contributions to probability and statistics: Essays in honour of Harold Hotelling (pp. 448–485). Stanford, CA: Stanford University Press.Google Scholar
  48. Walker, A., & White, G. (2013). Oxford handbooks for language teachers: Technology enhanced language learning. Oxford: Oxford University Press.Google Scholar
  49. Wilcox, R. (2001). Fundamentals of modern statistical methods: Substantially improving power and accuracy. New York: Springer-Verlag.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  • Luke K. Fryer
    • 1
  • Jenifer Larson-Hall
    • 2
    Email author
  • Jeffrey Stewart
    • 3
  1. 1.University of Hong KongHong KongHong Kong
  2. 2.University of KitakyushuKitakyushuJapan
  3. 3.Kyushu Sangyo UniversityFukuokaJapan

Personalised recommendations