Advertisement

Abstract

This chapter traces the history of replication research in the field of applied linguistics, culminating in a discussion of current views of replication research as a means of evaluating the internal and external validity of a study, illuminating phenomena of interest, and ultimately, driving both theory and pedagogy forward. It provides an overview of different types of replication studies (exact, approximate, conceptual) with recent examples from the field. Challenges concerning the interpretation of replication results, as well as ongoing controversies over the replication of qualitative studies, are discussed. The author concludes with current recommendations for facilitating replication research, including those pertaining to reporting and data sharing.

Keywords

Replication Approximate replication Conceptual replication Exact replication Non-supportive replication Qualitative replication Supportive replication 

References

  1. Abbuhl, R. (2012a). Practical methods for teaching replication to applied linguistics studies. In G. Porte (Ed.), Replication research in applied linguistics (pp. 135–150). Cambridge: Cambridge University Press.Google Scholar
  2. Abbuhl, R. (2012b). When, when, and how to replicate research. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition (pp. 296–312). Oxford: Wiley-Blackwell.CrossRefGoogle Scholar
  3. Basturkmen, H. (2014). Replication research in comparative genre analysis in English for academic purposes. Language Teaching, 47(3), 377–386.CrossRefGoogle Scholar
  4. Benson, L., & Borrego, M. (2015). The role of replication in engineering education research. Journal of Engineering Education, 104(4), 388–392.CrossRefGoogle Scholar
  5. Bikowski, D., & Schulze, M. (2015). Replication and evaluation in CALL. CALICO Journal, 32(2), i–v.CrossRefGoogle Scholar
  6. Bitchener, J., & Knoch, U. (2015). Written corrective feedback studies: Approximate replication of Bitchener & Knoch (2010a) and Van Beuningen, De Jong & Kuiken (2012). Language Teaching, 48(3), 405–414.CrossRefGoogle Scholar
  7. Booth, P. (2013). Vocabulary knowledge in relation to memory and analysis: An approximate replication of Milton’s (2007) study on lexical profiles and learning style. Language Teaching, 46(3), 335–354.CrossRefGoogle Scholar
  8. Bronstein, R. (1990). Publication politics, experimenter bias and the replication process in social science research. Journal of Social Behavior and Personality, 5(4), 71–81.Google Scholar
  9. Burman, L., Reed, W., & Alm, J. (2010). A call for replication studies. Public Finance Review, 38(6), 787–793.CrossRefGoogle Scholar
  10. Casanave, C. (2012). Heading in the wrong direction? A response to Porte and Richards. Journal of Second Language Writing, 21(3), 296–297.CrossRefGoogle Scholar
  11. Chun, D. (2012). Replication studies in CALL research. CALICO Journal, 29(4), 591–600.CrossRefGoogle Scholar
  12. Crandall, C., & Sherman, J. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93–99.CrossRefGoogle Scholar
  13. Earp, B., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6(621), 1–11.Google Scholar
  14. Easley, R., Madden, C., & Dunn, M. (2000). Conducting market science: The role of replication in the research process. Journal of Business Research, 48(1), 83–92.CrossRefGoogle Scholar
  15. Frankenberg-Garcia, A. (2014). The use of corpus examples for language comprehension and production. ReCALL, 26(2), 128–146.CrossRefGoogle Scholar
  16. Gass, S., & Valmori, L. (2015). Replication in interaction and working memory research: Révész (2012) and Goo (2012). Language Teaching, 48(4), 545–555.CrossRefGoogle Scholar
  17. Golden, M. (1995). Replication and non-quantitative research. PS: Political Science and Politics, 28(3), 481–483.Google Scholar
  18. Hendrik, C. (1990). Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior and Personality, 5(4), 41–49.Google Scholar
  19. Hubbard, R., & Armstrong, J. (1994). Replications and extensions in marketing: Rarely published but quite contrary. International Journal of Research in Marketing, 11(3), 233–248.CrossRefGoogle Scholar
  20. Ioannidis, J. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.CrossRefGoogle Scholar
  21. John, L., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.CrossRefGoogle Scholar
  22. Johnson, M., Mercado, L., & Acevedo, A. (2012). The effect of planning sub-processes on L2 writing fluency, grammatical complexity, and lexical complexity. Journal of Second Language Writing, 21(3), 264–282.CrossRefGoogle Scholar
  23. Johnson, M., & Nicodemus, C. (2015). Testing a threshold: An approximate replication of Johnson, Mercado & Acevedo 2012. Language Teaching, 49(2), 251–274.CrossRefGoogle Scholar
  24. King, K., & Mackey, A. (2016). Research methodology in second language studies: Trends, concerns, and new directions. Modern Language Journal, 100(s1), 209–227.CrossRefGoogle Scholar
  25. Klein, R. A., Ratliff, R. A., Vianello, M., Adams, R. A., Jr., Bahník, S., Bernstein, M. J., … Nosek, B. A. (2014). Investigating variation in replicability: A “ManyLabs” replication project. Social Psychology, 45(3), 142–152.CrossRefGoogle Scholar
  26. Language Teaching Review Panel. (2008). Replication studies in language learning and teaching: Questions and answers. Language Teaching, 41(1), 1–14.Google Scholar
  27. Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(S1), 127–159.CrossRefGoogle Scholar
  28. LeBel, E. (2015). A new replication norm for psychology. Collabra, 1(1), 1–13.CrossRefGoogle Scholar
  29. Leow, R. (2015). The roles of attention and (un)awareness in SLA: Conceptual replication of N.C. Ellis & Sagarra (2010a) and Leung & Williams (2012). Language Teaching, 48(1), 117–129.CrossRefGoogle Scholar
  30. Lim, H., & Godfroid, A. (2015). Automatization in second language sentence processing: A partial, conceptual replication of Hulstijn, Van Gelderen, and Schoonen’s 2009 study. Applied Psycholinguistics, 36(5), 1247–1282.CrossRefGoogle Scholar
  31. Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66–81.CrossRefGoogle Scholar
  32. Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.CrossRefGoogle Scholar
  33. Mackey, A., & Marsden, E. (2016). Advancing methodology and practice: The IRIS repository of instruments for research into second languages. New York: Routledge.Google Scholar
  34. Makel, M., & Plucker, J. (2015). An introduction to replication research in gifted education: Shiny and new is not the same as useful. Gifted Child Quarterly, 59(3), 157–164.CrossRefGoogle Scholar
  35. Makel, M., Plucker, J., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 532–542.CrossRefGoogle Scholar
  36. Markee, N. (2017). Are replication studies possible in qualitative second/foreign language classroom research? A call for comparative re-production research. Language Teaching, 50(3), 367–383.CrossRefGoogle Scholar
  37. Marsden, E., & Mackey, A. (2013, June). IRIS and replication. Paper presented at the 9th International Symposium on Bilingualism, Singapore.Google Scholar
  38. Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository: Advancing research practice and methodology. In A. Mackey & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository for research into second languages (pp. 1–21). New York: Routledge.Google Scholar
  39. Matsuda, P. (2012). On the nature of second language writing: Replication in a postmodern field. Journal of Second Language Writing, 21(3), 300–302.CrossRefGoogle Scholar
  40. Maxwell, S., Lau, M., & Howard, G. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487–498.CrossRefGoogle Scholar
  41. McNeeley, S., & Warner, J. (2015). Replication in criminology: A necessary practice. European Journal of Criminology, 12(5), 581–597.CrossRefGoogle Scholar
  42. Mezias, S., & Regnier, M. (2007). Walking the walk as well as talking the talk: Replication and the normal science paradigm in strategic management research. Strategic Organization, 5(3), 283–296.CrossRefGoogle Scholar
  43. Moravcsik, A. (2014). Transparency: The revolution in qualitative research. PS: Political Science & Politics, 47(1), 48–53.Google Scholar
  44. Mu, C., & Matsuda, P. (2016). Replication in L2 writing research: Journal of second language writing authors’ perceptions. TESOL Quarterly, 50(1), 201–219.CrossRefGoogle Scholar
  45. Nassaji, H. (2012). Significance tests and generalizability of research results: A case for replication. In G. Porte (Ed.), Replication research in applied linguistics (pp. 92–115). Cambridge: Cambridge University Press.Google Scholar
  46. Neuliep, J., & Crandall, R. (1990). Editorial bias against replication research. Journal of Social Behavior and Personality, 5(4), 85–90.Google Scholar
  47. Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7(6), 657–660.CrossRefGoogle Scholar
  48. Pashler, H., & Harris, C. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536.CrossRefGoogle Scholar
  49. Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.CrossRefGoogle Scholar
  50. Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics (pp. 116–132). Cambridge: Cambridge University Press.Google Scholar
  51. Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687.CrossRefGoogle Scholar
  52. Plonsky, L. (2015). Quantitative considerations for improving replicability in CALL and applied linguistics. CALICO Journal, 32(2), 232–244.Google Scholar
  53. Plonsky, L., Egbert, J., & Laflair, G. (2015). Bootstrapping in applied linguistics: Assessing its potential using shared data. Applied Linguistics, 36(5), 591–610.Google Scholar
  54. Polio, C. (2012a). No paradigm wars please! Journal of Second Language Writing, 21(3), 294–295.CrossRefGoogle Scholar
  55. Polio, C. (2012b). Replication in published applied linguistics research: An historical perspective. In G. Porte (Ed.), Replication research in applied linguistics (pp. 47–91). Cambridge: Cambridge University Press.Google Scholar
  56. Polio, C., & Gass, S. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition, 19(4), 499–508.CrossRefGoogle Scholar
  57. Porte, G. (2012). Replication research in applied linguistics. Cambridge: Cambridge University Press.Google Scholar
  58. Porte, G. (2013). Who needs replication research? CALICO Journal, 30(1), 10–15.CrossRefGoogle Scholar
  59. Porte, G., & Richards, K. (2012). Focus article: Replication in second language writing research. Journal of Second Language Writing, 21(3), 284–293.CrossRefGoogle Scholar
  60. Reis, H., & Lee, K. (2016). Promise, peril, and perspective: Addressing concerns about reproducibility in social-personality psychology. Journal of Experimental Social Psychology, 66, 148–152.CrossRefGoogle Scholar
  61. Richards, K. (2009). Trends in qualitative research in language teaching since 2000. Language Teaching, 42(2), 147–180.CrossRefGoogle Scholar
  62. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.CrossRefGoogle Scholar
  63. Rott, S., & Gavin, B. (2015). Comprehending and learning from Internet sources: A conceptual replication study of Goldman, Braasch, Wiley, Greasser and Brodowinska (2012). CALICO Journal, 32(2), 323–354.CrossRefGoogle Scholar
  64. Sakaluk, J. (2016). Exploring small, confirming big: An alternative system to the new statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology, 66, 47–54.CrossRefGoogle Scholar
  65. Santos, T. (1989). Replication in applied linguistics research. TESOL Quarterly, 23(4), 699–702.CrossRefGoogle Scholar
  66. Sasaki, M. (2012). An alternative approach to replication studies in second language writing: An ecological perspective. Journal of Second Language Writing, 21(3), 303–305.CrossRefGoogle Scholar
  67. Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212–226.CrossRefGoogle Scholar
  68. Schofield, J. (2002). Increasing the generalizability of qualitative research. In A. Huberman & M. Miles (Eds.), The qualitative researcher’s companion (pp. 171–203). Thousand Oaks, CA: SAGE.Google Scholar
  69. Schweinsberg, M., Madan, N., Vianello, M., Amy Sommer, S., Jordan, J., Tierney, W., … Uhlmann, E. L. (2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55–67.CrossRefGoogle Scholar
  70. Simmons, J., Nelson, L., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefGoogle Scholar
  71. Smith, B., & Schulze, M. (2013). Thirty years of the CALICO Journal—Replicate, replicate, replicate. CALICO Journal, 30(1), i–iv.CrossRefGoogle Scholar
  72. Stake, R. (2005). Qualitative case studies. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (pp. 443–466). Thousand Oaks, CA: SAGE.Google Scholar
  73. Valdman, A. (1993). Replication study. Studies in Second Language Acquisition, 15(4), 505.CrossRefGoogle Scholar
  74. Webb, S. (2015). Learning vocabulary through meaning-focused input: Replication of Elley (1989) and Liu & Nation (1985). Language Teaching, 49(1), 129–140.CrossRefGoogle Scholar
  75. Willans, F., & Leung, C. (2016). Empirical foundations for medium of instruction policies: Approximate replications of Afolayan (1976) and Siegel (1997b). Language Teaching, 49(4), 549–563.CrossRefGoogle Scholar
  76. Yoon, H.-J., & Polio, C. (2017). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51(2), 275–301.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Department of LinguisticsCalifornia State University at Long BeachLong BeachUSA

Personalised recommendations