Sample Size Considerations in Prevention Research Applications of Multilevel Modeling and Structural Equation Modeling
- 1.1k Downloads
When the goal of prevention research is to capture in statistical models some measure of the dynamic complexity in structures and processes implicated in problem behavior and its prevention, approaches such as multilevel modeling (MLM) and structural equation modeling (SEM) are indicated. Yet the assumptions that must be satisfied if these approaches are to be used responsibly raise concerns regarding their use in prevention research involving smaller samples. In this article, we discuss in nontechnical terms the role of sample size in MLM and SEM and present findings from the latest simulation work on the performance of each approach at sample sizes typical of prevention research. For each statistical approach, we draw from extant simulation studies to establish lower bounds for sample size (e.g., MLM can be applied with as few as ten groups comprising ten members with normally distributed data, restricted maximum likelihood estimation, and a focus on fixed effects; sample sizes as small as N = 50 can produce reliable SEM results with normally distributed data and at least three reliable indicators per factor) and suggest strategies for making the best use of the modeling approach when N is near the lower bound.
KeywordsSample size Multilevel modeling Structural equation modeling
During the writing of this manuscript, the authors were supported by National Institute on Drug Abuse (NIDA) Grant P30 DA023026. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NIDA.
- Bandalos, D. L., & Gagné, P. (2012). Simulation methods in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 92–108). New York: Guildford Press.Google Scholar
- Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken: Wiley.Google Scholar
- Curran, P. J., Lee, T., Howard, A. L., Lane, S., & MacCallum, R. (2012). Disaggregating within-person and between-person effects in multilevel and structural equation growth models. In J. R. Harring & G. R. Hancock (Eds.), Advances in longitudinal models in the social and behavioral sciences (pp. 217–253). Charlotte: Information Age Publishing.Google Scholar
- Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 1–38. Retrieved from http://www.jstor.org/stable/2984875?origin=JSTOR-pdf.Google Scholar
- Hopkin, C. R., Hoyle, R. H., & Gottfredson, N. C. (2013). Maximizing the yield of small samples in prevention research: A review of general strategies and best practices. Manuscript submitted for publication.Google Scholar
- Hoyle, R. H. (2011). Structural equation modeling for social and personality psychology. London: Sage Publications.Google Scholar
- Hoyle, R. H., & Kenny, D. A. (1999). Sample size, reliability, and tests of statistical mediation. In R. H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 195–222). Thousand Oaks: Sage Publications.Google Scholar
- Kaplan, D. (1995). Statistical power in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 100–117). Newbury Park: Sage Publications.Google Scholar
- Kreft, I. G. G., & de Leeuw, J. (1988). Introducing multilevel modeling. Thousand Oaks: Sage Publications.Google Scholar
- Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2002). Reliable estimation of generalized linear mixed models using adaptive quadrature. The Stata Journal, 2, 1–21. Retrieved from http://www.stata-journal.com/article.html?article=st0005.Google Scholar
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks: Sage Publications.Google Scholar
- Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for Windows [Computer software]. Lincolnwood: Scientific Software.Google Scholar
- Raudenbush, S. W., Spybrook, J., Congdon, R., Liu, X., & Martinez, A. (2011). Optimal Design Software for multi-level and longitudinal research (version 3.01) [Software]. Available from www.wtgrantfoundation.org.
- Shiyko, M. P., Lanza, S. T., Tan, X., Li, R., & Shiffman, S. (2012). Using the time-varying effect model (TVEM) to examine dynamic associations between negative affect and self-confidence on smoking urges: Differences between successful quitters and relapsers. Prevention Science, 13, 288–299. doi: 10.1007/s11121-011-0264-z.PubMedCentralCrossRefPubMedGoogle Scholar
- Simon, T. R., Ikeda, R. M., Smith, E. P., Reese, L. E., Rabiner, D. L., Miller-Johnson, S., et al. (2008). The multisite violence prevention project: Impact of a universal school-based violence prevention program on social-cognitive outcomes. Prevention Science, 9, 231–244. doi: 10.1007/s11121-008-0101-1.PubMedCentralCrossRefGoogle Scholar
- Snijders, T. A. B., & Bosker, R. J. (2004). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage Publications.Google Scholar
- Steiger, J. H., & Lind, J. C. (1980). Statistically based tests for the number of common factors. Iowa City: Paper presented at the Meeting of the Psychometric Society.Google Scholar