Advertisement

Behavior Research Methods

, Volume 51, Issue 1, pp 316–331 | Cite as

Concealed correlations meta-analysis: A new method for synthesizing standardized regression coefficients

  • Belén Fernández-CastillaEmail author
  • Ariel M. Aloe
  • Lies Declercq
  • Laleh Jamshidi
  • Patrick Onghena
  • S. Natasha Beretvas
  • Wim Van den Noortgate
Article

Abstract

The synthesis of standardized regression coefficients is still a controversial issue in the field of meta-analysis. The difficulty lies in the fact that the standardized regression coefficients belonging to regression models that include different sets of covariates do not represent the same parameter, and thus their direct combination is meaningless. In the present study, a new approach called concealed correlations meta-analysis is proposed that allows for using the common information that standardized regression coefficients from different regression models contain to improve the precision of a combined focal standardized regression coefficient estimate. The performance of this new approach was compared with that of two other approaches: (1) carrying out separate meta-analyses for standardized regression coefficients from studies that used the same regression model, and (2) performing a meta-regression on the focal standardized regression coefficients while including an indicator variable as a moderator indicating the regression model to which each standardized regression coefficient belongs. The comparison was done through a simulation study. The results showed that, as expected, the proposed approach led to more accurate estimates of the combined standardized regression coefficients under both random- and fixed-effect models.

Keywords

Meta-analysis Standardized regression coefficients Effect size 

References

  1. Aloe, A. M. (2015). Inaccuracy of regression results in replacing bivariate correlations. Research Synthesis Methods, 6, 21–27.CrossRefGoogle Scholar
  2. Aloe, A. M., & Becker, B. J. (2009). Teacher verbal ability and school outcomes: Where is the evidence? Educational Researcher, 38, 612–624.CrossRefGoogle Scholar
  3. Aloe, A. M., & Becker, B. J. (2012). An effect size for regression predictors in meta-analysis. Journal of Educational and Behavioral Statistics, 37, 278–297.CrossRefGoogle Scholar
  4. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational and Behavioral Statistics, 17, 341–362.CrossRefGoogle Scholar
  5. Becker, B. J., & Wu, M. J. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414–429.CrossRefGoogle Scholar
  6. Bini, L. M., Coelho, A. G., & Diniz-Filho, J. A. F. (2001). Is the relationship between population density and body size consistent across independent studies? A meta-analytical approach. Revista Brasileira de Biologia, 61, 1–6.CrossRefGoogle Scholar
  7. Borenstein, M. (1994). Effect sizes for continuous data. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis and meta-analysis (pp. 221–235). New York: Sage.Google Scholar
  8. Bowman, N. A. (2012). Effect sizes and statistical methods for meta-analysis in higher education. Research in Higher Education, 53, 375–382.CrossRefGoogle Scholar
  9. Cooley, W. W., & Lohnes, P. R. (1971). Multiple correlation and the method of least squares. In Multivariate data analysis (pp. 49–95). New York: Wiley.Google Scholar
  10. Ferguson, C. J. (2015a). Do angry birds make for angry children? A meta-analysis of video game influences on children’s and adolescents’ aggression, mental health, prosocial behavior, and academic performance. Perspectives on Psychological Science, 10, 646–666.CrossRefGoogle Scholar
  11. Ferguson, C. J. (2015b). Pay no attention to that data behind the curtain: On angry birds, happy children, scholarly squabbles, publication bias, and why betas rule metas. Perspectives on Psychological Science, 10, 683–691.CrossRefGoogle Scholar
  12. Greenland, S., Maclure, M., Schlesselman, J. J., Poole, C., & Morgenstern, H. (1991). Standardized regression coefficients: A further critique and review of some alternatives. Epidemiology, 2, 387–392.CrossRefGoogle Scholar
  13. Hanushek, E. A. (1974). Efficient estimators for regressing regression coefficients. The American Statistician, 28, 66–67.Google Scholar
  14. Hedges, L. V., & Olkin, I. (1985). Statistical method for meta-analysis. New York: Academic Press.Google Scholar
  15. Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.CrossRefGoogle Scholar
  16. Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling. Sociological Methods & Research, 26, 329–367.CrossRefGoogle Scholar
  17. Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227–235.CrossRefGoogle Scholar
  18. Kim, R. S. (2011). Standardized regression coefficients as indices of effect sizes in meta-analysis. Electronic theses, treatises and dissertations (ETDs). Available from http://purl.flvc.org/fsu/fd/FSU_migr_etd-3109
  19. Lunn, D. J., Thomas, A., Best N., & Spiegelhalter, D. (2000). WinBUGS—A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.  https://doi.org/10.1023/A:1008929526011 CrossRefGoogle Scholar
  20. Meng, X.-L., Rosenthal R., & Rubin, D. B. (1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111, 172–175.  https://doi.org/10.1037/0033-2909.111.1.172 CrossRefGoogle Scholar
  21. Nieminen, P., Lehtiniemi, H., Vähäkangas, K., Huusko, A., & Rautio, A. (2013). Standardised regression coefficient as an effect size index in summarising findings in epidemiological studies. Epidemiology, Biostatistics and Public Health, 10, e8854:1–15.Google Scholar
  22. Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2014). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47, 1274–1294.Google Scholar
  23. Novick, M. R., Jackson, P. H., Thayer, D. T., & Cole, N. S. (1972). Estimating multiple regressions in m groups: A cross-validation study. British Journal of Mathematical and Statistical Psychology, 25, 33–50.CrossRefGoogle Scholar
  24. Paul, P. A., Lipps, P. E., & Madden, L. V. (2006). Meta-analysis of regression coefficients for the relationship between Fusarium head blight and deoxynivalenol content of wheat. Phytopathology, 96, 951–961.CrossRefGoogle Scholar
  25. Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90, 175–181.CrossRefGoogle Scholar
  26. Pratt, T., Cullen F., Sellers C., Winfree, T., Madensen T., Daigle L., . . . Gau, J. M. (2009). The empirical status of social learning theory: A meta-analysis. Justice Quarterly, 27, 765–802.CrossRefGoogle Scholar
  27. Stanley, T. D., & Jarrell, S. B. (1989). Meta-regression analysis: A quantitative method of literature surveys. Journal of Economic Surveys, 3, 161–170.CrossRefGoogle Scholar
  28. Yang, Z. J., Aloe, A. M., & Feeley, T. H. (2014). Risk information seeking and processing model: A meta-analysis. Journal of Communication, 64, 20–41.CrossRefGoogle Scholar
  29. Yin, R. K., Schmidt, R. J., & Besag, F. (2006). Aggregating student achievement trends across states with different tests: Using standardized slopes as effect sizes. Peabody Journal of Education, 81(2), 47–61.CrossRefGoogle Scholar
  30. Yoneoka, D., & Henmi, M. (2017). Synthesis of linear regression coefficients by recovering the within-study covariance matrix from summary statistics. Research Synthesis Methods, 8, 212–219.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Faculty of Psychology and Educational Sciences, KU LeuvenUniversity of LeuvenKortrijkBelgium
  2. 2.Imec-Itec, KU LeuvenUniversity of LeuvenLeuvenBelgium
  3. 3.University of IowaIowa CityUSA
  4. 4.University of TexasAustinUSA

Personalised recommendations