Behavior Research Methods

, Volume 50, Issue 2, pp 786–803 | Cite as

Analyzing individual growth with clustered longitudinal data: A comparison between model-based and design-based multilevel approaches

  • Hsien-Yuan Hsu
  • John J. H. Lin
  • Susan T. Skidmore
Article

Abstract

To prevent biased estimates of intraindividual growth and interindividual variability when working with clustered longitudinal data (e.g., repeated measures nested within students; students nested within schools), individual dependency should be considered. A Monte Carlo study was conducted to examine to what extent two model-based approaches (multilevel latent growth curve model – MLGCM, and maximum model – MM) and one design-based approach (design-based latent growth curve model – D-LGCM) could produce unbiased and efficient parameter estimates of intraindividual growth and interindividual variability given clustered longitudinal data. The solutions of a single-level latent growth curve model (SLGCM) were also provided to demonstrate the consequences of ignoring individual dependency. Design factors considered in the present simulation study were as follows: number of clusters (NC = 10, 30, 50, 100, 150, 200, and 500) and cluster size (CS = 5, 10, and 20). According to our results, when intraindividual growth is of interest, researchers are free to implement MLGCM, MM, or D-LGCM. With regard to interindividual variability, MLGCM and MM were capable of producing accurate parameter estimates and SEs. However, when D-LGCM and SLGCM were applied, parameter estimates of interindividual variability were not comprised exclusively of the variability in individual (e.g., students) growth but instead were the combined variability of individual and cluster (e.g., school) growth, which cannot be interpreted. The take-home message is that D-LGCM does not qualify as an alternative approach to analyzing clustered longitudinal data if interindividual variability is of interest.

Keywords

Clustered longitudinal data Design-based approach Model-based approach Multilevel latent growth curve model 

References

  1. Bandura, A. (1993). Perceived self-efficacy in cognitive develdevelopment and functioning. Educational Psychologist, 28(2), 117–148.CrossRefGoogle Scholar
  2. Baumert, J., Nagy, G., & Lehmann, R. (2012). Cumulative advantages and the emergence of social and ethnic inequality: Matthew effects in reading and mathematics development within elementary schools? Child Development, 83(4), 1347–1367. doi: 10.1111/j.1467-8624.2012.01779.x CrossRefPubMedGoogle Scholar
  3. Bovaird, J. A. (2007). Multilevel structural equation models for contextual factors. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies. Mahwah: Lawrence Erlbaum Associates.Google Scholar
  4. Chen, Q., Kwok, O.-M., Luo, W., & Willson, V. L. (2010). The impact of ignoring a level of nesting structure in multilevel growth mixture models: A Monte Carlo study. Structural Equation Modeling, 17(4), 570–589. doi: 10.1080/10705511.2010.510046 CrossRefGoogle Scholar
  5. Curran, P. J., Obeidat, K., & Losardo, D. (2010). Twelve frequently asked questions about growth curve modeling. Journal of Cognition and Development, 11(2), 121–136. doi: 10.1080/15248371003699969 CrossRefPubMedPubMedCentralGoogle Scholar
  6. Diallo, T. M. O., Morin, A. J. S., & Parker, P. D. (2014). Statistical power of latent growth curve models to detect quadratic growth. Behavior Research Methods, 46(2), 357–371. doi: 10.3758/s13428-013-0395-1 CrossRefPubMedGoogle Scholar
  7. Duncan, T. E., Duncan, S. C., Alpert, A., Hops, H., Stoolmiller, M., & Muthén, B. O. (1997). Latent variable modeling of longitudinal and multilevel substance use data. Multivariate Behavioral Research, 32(3), 275–318. doi: 10.1207/s15327906mbr3203_3 CrossRefPubMedGoogle Scholar
  8. Duncan, T. E., Duncan, S. C., & Strycker, L. A. (2006). An Introduction to Latent Variable Growth Curve Modeling Concepts, Issues, and Applications (2nd ed.). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  9. Ferragut, M., Blanca, M. J., & Ortiz-Tallo, M. (2014). Psychological virtues during adolescence: A longitudinal study of gender differences. European Journal of Developmental Psychology, 11(5), 521–531. doi: 10.1080/17405629.2013.876403 CrossRefGoogle Scholar
  10. Grimm, K. J., Ram, N., & Estabrook, R. (2016). Growth modeling: Structural equation and multilevel modeling approaches. New York: Guilford Press.Google Scholar
  11. Heck, R. H., & Thomas, S. L. (2009). An introduction to multilevel modeling techniques (2nd ed.). New York: Routledge.Google Scholar
  12. Heeringa, S. G., West, B. T., & Berglund, P. A. (2010). Applied survey data analysis. Boca Raton: Chapman & Hall/CRC.CrossRefGoogle Scholar
  13. Hertzog, C., Oertzen, T. v., & Ghisletta, P. (2008). Evaluating the power of latent growth curve models to detect individual differences in change. Structural Equation Modeling, 15, 541–563. doi: 10.1080/10705510802338983 CrossRefGoogle Scholar
  14. Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26(3), 329–367. doi: 10.1177/0049124198026003003 CrossRefGoogle Scholar
  15. Hox, J. J. (2010). Multilevel Analysis Techniques and Applications (2nd ed.). New York: Routledge.Google Scholar
  16. Hox, J. J., & Maas, C. J. M. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8(2), 157–174. doi: 10.1207/S15328007SEM0802_1 CrossRefGoogle Scholar
  17. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.CrossRefGoogle Scholar
  18. Ingels, S. J., Pratt, D. J., Rogers, J. E., Siegel, P. H., & Stutts, E. S. (2005). Education Longitudinal Study of 2002: Base-year to first follow-up data file documentation (NCES 2006–344). Retrieved from Washington, DC.Google Scholar
  19. Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8(3), 325–352. doi: 10.1207/S15328007SEM0803_1 CrossRefGoogle Scholar
  20. Kwok, O., West, S. G., & Green, S. B. (2007). The impact of misspecifying the within-subject covariance structure in multiwave longitudinal multilevel models: A Monte Carlo study. Multivariate Behavioral Research, 42(3), 557–592. doi: 10.1080/00273170701540537 CrossRefGoogle Scholar
  21. Lai, M. H. C., & Kwok, O. (2015). Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule. The Journal of Experimental Education, 83(3), 423–438. doi: 10.1080/00220973.2014.907229 CrossRefGoogle Scholar
  22. Luo, W., & Kwok, O. (2009). The impacts of ignoring a crossed factor in analyzing cross-classified data. Multivariate Behavioral Research, 44(2), 182–212. doi: 10.1080/00273170902794214 CrossRefPubMedGoogle Scholar
  23. Ma, X., & Ma, L. (2004). Modeling stability of growth between mathematics and science achievement during middle and high school. Evaluation Review, 28(2), 104–122. doi: 10.1177/0193841X03261025 CrossRefPubMedGoogle Scholar
  24. Ma, X., & Wilkins, J. M. (2007). Mathematics coursework regulates growth in mathematics achievement. Journal for Research in Mathematics Education, 38(3), 230–257.Google Scholar
  25. Meyers, J. L., & Beretvas, N. (2006). The impact of inappropriate modeling of cross-classified data structures. Multivariate Behavioral Research, 41(4), 473–497. doi: 10.1207/s15327906mbr4104_3 CrossRefPubMedGoogle Scholar
  26. Miller, J. D., Kimmel, L., Hoffer, T. B., & Nelson, C. (2000). Longitudinal study of American youth: User's manual. Evanston: Northwestern University, International Center for the Advancement of Scientific Literacy.Google Scholar
  27. Moerbeek, M. (2004). The consequence of ignoring a level of nesting in multilevel analysis. Multivariate Behavioral Research, 39(1), 129–149. doi: 10.1207/s15327906mbr3901_5 CrossRefPubMedGoogle Scholar
  28. Muthén, B. O. (1997). Latent variable growth modeling with multilevel data. In M. Berkane (Ed.), Latent variable modeling and applications to causality (pp. 149-161). New York, NY.Google Scholar
  29. Muthén, B. O. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of Quantitative Methodology for the Social Sciences (pp. 345–368). Newbury Park: Sage Publications.Google Scholar
  30. Muthén, B. O., & Asparouhov, T. (2009). Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework. In J. Hox & J. K. Roberts (Eds.), Handbook of Advanced Multilevel Analysis. New York: Routledge.Google Scholar
  31. Muthén, B. O., & Asparouhov, T. (2011). Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework. In J. Hox & J. K. Roberts (Eds.), The Handbook of Advanced Multilevel Analysis. New York: Routledge.Google Scholar
  32. Muthén, B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2(4), 371–402. doi: 10.1037/1082-989X.2.4.371 CrossRefGoogle Scholar
  33. Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316. doi: 10.2307/271070 CrossRefGoogle Scholar
  34. Muthén, L. K., & Muthén, B. O. (1998-2015). Mplus user’s guild (7th ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  35. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(4), 599–620.CrossRefGoogle Scholar
  36. Pfost, M., Hattie, J., Dörfler, T., & Artelt, C. (2014). Individual differences in reading development: A review of 25 years of empirical research on Matthew Effects in reading. Review of Educational Research, 84(2), 203–244. doi: 10.3102/0034654313509492 CrossRefGoogle Scholar
  37. Pornprasertmanit, S., Lee, J., & Preacher, K. J. (2014). Ignoring clustering in confirmatory factor analysis: Some consequences for model fit and standardized. Multivariate Behavioral Research, 49, 518–543. doi: 10.1080/00273171.2014.933762 CrossRefPubMedGoogle Scholar
  38. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods (2nd ed.). Thousand Oaks: Sage.Google Scholar
  39. Schaie, K. W. (1983). Longitudinal studies of adult psychological development. New York: Guilford Press.Google Scholar
  40. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis, modeling change and event occurrence. New York: Oxford University Press.CrossRefGoogle Scholar
  41. Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oaks: Sage.Google Scholar
  42. Stapleton, L. M. (2006). Using Multilevel structural equation modeling techniques with complex sample data. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Secnd Course. Greenwich: Information Age.Google Scholar
  43. Stapleton, L. M. (2008). Variance estimation using replication methods in structural equation modeling with complex sample data. Structural Equation Modeling, 15(2), 183–210. doi: 10.1080/10705510801922316 CrossRefGoogle Scholar
  44. Wu, J.-Y., & Kwok, O. (2012). Using SEM to analyze complex survey data: A comparison between design-based single-level and model-based multilevel approaches. Structural Equation Modeling, 19(1), 16–35. doi: 10.1080/10705511.2012.634703 CrossRefGoogle Scholar
  45. Wu, J.-Y., Kwok, O., & Willson, V. L. (2015). Using design-based latent growth curve modeling with cluster-level predictor to address dependency. Journal of Experimental Education, 82(4), 431–454. doi: 10.1080/00220973.2013.876226 CrossRefGoogle Scholar
  46. Zhang, Z., & Wang, L. (2009). Statistical power analysis for growth curve models using SAS. Behavior Research Methods, 41(4), 1083–1094. doi: 10.3758/BRM.41.4.1083 CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Hsien-Yuan Hsu
    • 1
  • John J. H. Lin
    • 2
  • Susan T. Skidmore
    • 3
  1. 1.Children’s Learning InstituteUniversity of Texas Health Science Center at HoustonHoustonUSA
  2. 2.Office of Institutional ResearchNational Central UniversityTaoyuanTaiwan
  3. 3.Department of Educational LeadershipSam Houston State UniversityHuntsvilleUSA

Personalised recommendations