Advertisement

Abstract

This chapter traces the history of replication research in the field of applied linguistics, culminating in a discussion of current views of replication research as a means of evaluating the internal and external validity of a study, illuminating phenomena of interest, and ultimately, driving both theory and pedagogy forward. It provides an overview of different types of replication studies (exact, approximate, conceptual) with recent examples from the field. Challenges concerning the interpretation of replication results, as well as ongoing controversies over the replication of qualitative studies, are discussed. The author concludes with current recommendations for facilitating replication research, including those pertaining to reporting and data sharing.

Keywords

Replication Approximate replication Conceptual replication Exact replication Non-supportive replication Qualitative replication Supportive replication 

References

  1. Abbuhl, R. (2012a). Practical methods for teaching replication to applied linguistics studies. In G. Porte (Ed.), Replication research in applied linguistics (pp. 135–150). Cambridge: Cambridge University Press.Google Scholar
  2. Abbuhl, R. (2012b). When, when, and how to replicate research. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition (pp. 296–312). Oxford: Wiley-Blackwell.Google Scholar
  3. Basturkmen, H. (2014). Replication research in comparative genre analysis in English for academic purposes. Language Teaching, 47(3), 377–386.Google Scholar
  4. Benson, L., & Borrego, M. (2015). The role of replication in engineering education research. Journal of Engineering Education, 104(4), 388–392.Google Scholar
  5. Bikowski, D., & Schulze, M. (2015). Replication and evaluation in CALL. CALICO Journal, 32(2), i–v.Google Scholar
  6. Bitchener, J., & Knoch, U. (2015). Written corrective feedback studies: Approximate replication of Bitchener & Knoch (2010a) and Van Beuningen, De Jong & Kuiken (2012). Language Teaching, 48(3), 405–414.Google Scholar
  7. Booth, P. (2013). Vocabulary knowledge in relation to memory and analysis: An approximate replication of Milton’s (2007) study on lexical profiles and learning style. Language Teaching, 46(3), 335–354.Google Scholar
  8. Bronstein, R. (1990). Publication politics, experimenter bias and the replication process in social science research. Journal of Social Behavior and Personality, 5(4), 71–81.Google Scholar
  9. Burman, L., Reed, W., & Alm, J. (2010). A call for replication studies. Public Finance Review, 38(6), 787–793.Google Scholar
  10. Casanave, C. (2012). Heading in the wrong direction? A response to Porte and Richards. Journal of Second Language Writing, 21(3), 296–297.Google Scholar
  11. Chun, D. (2012). Replication studies in CALL research. CALICO Journal, 29(4), 591–600.Google Scholar
  12. Crandall, C., & Sherman, J. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93–99.Google Scholar
  13. Earp, B., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6(621), 1–11.Google Scholar
  14. Easley, R., Madden, C., & Dunn, M. (2000). Conducting market science: The role of replication in the research process. Journal of Business Research, 48(1), 83–92.Google Scholar
  15. Frankenberg-Garcia, A. (2014). The use of corpus examples for language comprehension and production. ReCALL, 26(2), 128–146.Google Scholar
  16. Gass, S., & Valmori, L. (2015). Replication in interaction and working memory research: Révész (2012) and Goo (2012). Language Teaching, 48(4), 545–555.Google Scholar
  17. Golden, M. (1995). Replication and non-quantitative research. PS: Political Science and Politics, 28(3), 481–483.Google Scholar
  18. Hendrik, C. (1990). Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior and Personality, 5(4), 41–49.Google Scholar
  19. Hubbard, R., & Armstrong, J. (1994). Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11(3), 233–248.Google Scholar
  20. Ioannidis, J. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.Google Scholar
  21. John, L., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.Google Scholar
  22. Johnson, M., Mercado, L., & Acevedo, A. (2012). The effect of planning sub-processes on L2 writing fluency, grammatical complexity, and lexical complexity. Journal of Second Language Writing, 21(3), 264–282.Google Scholar
  23. Johnson, M., & Nicodemus, C. (2015). Testing a threshold: An approximate replication of Johnson, Mercado & Acevedo 2012. Language Teaching, 49(2), 251–274.Google Scholar
  24. King, K., & Mackey, A. (2016). Research methodology in second language studies: Trends, concerns, and new directions. Modern Language Journal, 100(s1), 209–227.Google Scholar
  25. Klein, R. A., Ratliff, R. A., Vianello, M., Adams, R. A., Jr., Bahník, S., Bernstein, M. J., … Nosek, B. A. (2014). Investigating variation in replicability: A “ManyLabs” replication project. Social Psychology, 45(3), 142–152.Google Scholar
  26. Language Teaching Review Panel. (2008). Replication studies in language learning and teaching: Questions and answers. Language Teaching, 41(1), 1–14.Google Scholar
  27. Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(S1), 127–159.Google Scholar
  28. LeBel, E. (2015). A new replication norm for psychology. Collabra, 1(1), 1–13.Google Scholar
  29. Leow, R. (2015). The roles of attention and (un)awareness in SLA: Conceptual replication of N.C. Ellis & Sagarra (2010a) and Leung & Williams (2012). Language Teaching, 48(1), 117–129.Google Scholar
  30. Lim, H., & Godfroid, A. (2015). Automatization in second language sentence processing: A partial, conceptual replication of Hulstijn, Van Gelderen, and Schoonen’s 2009 study. Applied Psycholinguistics, 36(5), 1247–1282.Google Scholar
  31. Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66–81.Google Scholar
  32. Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.Google Scholar
  33. Mackey, A., & Marsden, E. (2016). Advancing methodology and practice: The IRIS repository of instruments for research into second languages. New York: Routledge.Google Scholar
  34. Makel, M., & Plucker, J. (2015). An introduction to replication research in gifted education: Shiny and new is not the same as useful. Gifted Child Quarterly, 59(3), 157–164.Google Scholar
  35. Makel, M., Plucker, J., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 532–542.Google Scholar
  36. Markee, N. (2017). Are replication studies possible in qualitative second/foreign language classroom research? A call for comparative re-production research. Language Teaching, 50(3), 367–383.Google Scholar
  37. Marsden, E., & Mackey, A. (2013, June). IRIS and replication. Paper presented at the 9th International Symposium on Bilingualism, Singapore.Google Scholar
  38. Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository: Advancing research practice and methodology. In A. Mackey & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository for research into second languages (pp. 1–21). New York: Routledge.Google Scholar
  39. Matsuda, P. (2012). On the nature of second language writing: Replication in a postmodern field. Journal of Second Language Writing, 21(3), 300–302.Google Scholar
  40. Maxwell, S., Lau, M., & Howard, G. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487–498.Google Scholar
  41. McNeeley, S., & Warner, J. (2015). Replication in criminology: A necessary practice. European Journal of Criminology, 12(5), 581–597.Google Scholar
  42. Mezias, S., & Regnier, M. (2007). Walking the walk as well as talking the talk: Replication and the normal science paradigm in strategic management research. Strategic Organization, 5(3), 283–296.Google Scholar
  43. Moravcsik, A. (2014). Transparency: The revolution in qualitative research. PS: Political Science & Politics, 47(1), 48–53.Google Scholar
  44. Mu, C., & Matsuda, P. (2016). Replication in L2 writing research: Journal of second language writing authors’ perceptions. TESOL Quarterly, 50(1), 201–219.Google Scholar
  45. Nassaji, H. (2012). Significance tests and generalizability of research results: A case for replication. In G. Porte (Ed.), Replication research in applied linguistics (pp. 92–115). Cambridge: Cambridge University Press.Google Scholar
  46. Neuliep, J., & Crandall, R. (1990). Editorial bias against replication research. Journal of Social Behavior and Personality, 5(4), 85–90.Google Scholar
  47. Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7(6), 657–660.Google Scholar
  48. Pashler, H., & Harris, C. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536.Google Scholar
  49. Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.Google Scholar
  50. Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics (pp. 116–132). Cambridge: Cambridge University Press.Google Scholar
  51. Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687.Google Scholar
  52. Plonsky, L. (2015). Quantitative considerations for improving replicability in CALL and applied linguistics. CALICO Journal, 32(2), 232–244.Google Scholar
  53. Plonsky, L., Egbert, J., & Laflair, G. (2015). Bootstrapping in applied linguistics: Assessing its potential using shared data. Applied Linguistics, 36(5), 591–610.Google Scholar
  54. Polio, C. (2012a). No paradigm wars please! Journal of Second Language Writing, 21(3), 294–295.Google Scholar
  55. Polio, C. (2012b). Replication in published applied linguistics research: An historical perspective. In G. Porte (Ed.), Replication research in applied linguistics (pp. 47–91). Cambridge: Cambridge University Press.Google Scholar
  56. Polio, C., & Gass, S. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition, 19(4), 499–508.Google Scholar
  57. Porte, G. (2012). Replication research in applied linguistics. Cambridge: Cambridge University Press.Google Scholar
  58. Porte, G. (2013). Who needs replication research? CALICO Journal, 30(1), 10–15.Google Scholar
  59. Porte, G., & Richards, K. (2012). Focus article: Replication in second language writing research. Journal of Second Language Writing, 21(3), 284–293.Google Scholar
  60. Reis, H., & Lee, K. (2016). Promise, peril, and perspective: Addressing concerns about reproducibility in social-personality psychology. Journal of Experimental Social Psychology, 66, 148–152.Google Scholar
  61. Richards, K. (2009). Trends in qualitative research in language teaching since 2000. Language Teaching, 42(2), 147–180.Google Scholar
  62. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.Google Scholar
  63. Rott, S., & Gavin, B. (2015). Comprehending and learning from Internet sources: A conceptual replication study of Goldman, Braasch, Wiley, Greasser and Brodowinska (2012). CALICO Journal, 32(2), 323–354.Google Scholar
  64. Sakaluk, J. (2016). Exploring small, confirming big: An alternative system to the new statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology, 66, 47–54.Google Scholar
  65. Santos, T. (1989). Replication in applied linguistics research. TESOL Quarterly, 23(4), 699–702.Google Scholar
  66. Sasaki, M. (2012). An alternative approach to replication studies in second language writing: An ecological perspective. Journal of Second Language Writing, 21(3), 303–305.Google Scholar
  67. Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212–226.Google Scholar
  68. Schofield, J. (2002). Increasing the generalizability of qualitative research. In A. Huberman & M. Miles (Eds.), The qualitative researcher’s companion (pp. 171–203). Thousand Oaks, CA: SAGE.Google Scholar
  69. Schweinsberg, M., Madan, N., Vianello, M., Amy Sommer, S., Jordan, J., Tierney, W., … Uhlmann, E. L. (2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55–67.Google Scholar
  70. Simmons, J., Nelson, L., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.Google Scholar
  71. Smith, B., & Schulze, M. (2013). Thirty years of the CALICO Journal—Replicate, replicate, replicate. CALICO Journal, 30(1), i–iv.Google Scholar
  72. Stake, R. (2005). Qualitative case studies. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (pp. 443–466). Thousand Oaks, CA: SAGE.Google Scholar
  73. Valdman, A. (1993). Replication study. Studies in Second Language Acquisition, 15(4), 505.Google Scholar
  74. Webb, S. (2015). Learning vocabulary through meaning-focused input: Replication of Elley (1989) and Liu & Nation (1985). Language Teaching, 49(1), 129–140.Google Scholar
  75. Willans, F., & Leung, C. (2016). Empirical foundations for medium of instruction policies: Approximate replications of Afolayan (1976) and Siegel (1997b). Language Teaching, 49(4), 549–563.Google Scholar
  76. Yoon, H.-J., & Polio, C. (2017). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51(2), 275–301.Google Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Department of LinguisticsCalifornia State University at Long BeachLong BeachUSA

Personalised recommendations