A Comparison of Bootstrap Confidence Intervals for Multi-level Longitudinal Data Using Monte-Carlo Simulation

  • Mark ReiserEmail author
  • Lanlan Yao
  • Xiao Wang
  • Jeanne Wilcox
  • Shelley Gray
Part of the ICSA Book Series in Statistics book series (ICSABSS)


Longitudinal investigations, where subjects are followed over time, have played an increasingly prominent role in medicine, health, and psychology in the last decades. This chapter will address inference for a two-level mixed model for a longitudinal study where observational units are clustered at both levels. Bootstrap confidence intervals for model parameters are investigated under the issues of non-normality and limited sample size of the original data. A one stage case-resampling bootstrap will be established for constructing confidence intervals by sampling clusters with replacement at the higher level. A two-stage case-resampling bootstrap will be developed by sampling clusters with replacement at the higher level and then sampling with replacement at the lower level also. Monte-Carlo simulations will be utilized to evaluate the effectiveness of these bootstrap methods with various size clusters for the mixed-effects model in terms of bias, standard deviation and confidence interval coverage for the fixed effects as well as for variance components of the random effects . The results show that the parametric bootstrap and cluster bootstrap at the higher level perform better than the two-stage bootstrap . The bootstrap methods will be applied to a longitudinal study of preschool children nested within classrooms.


Cluster bootstrap Two-stage bootstrap Parametric bootstrap Monte-Carlo bootstrap Mixed-effects linear model Hierarchical linear model Repeated measures Nested design Classroom-based study 



This research was supported by the U.S. Department of Education, Institute of Educational Sciences Grant R324A110048. The opinions expressed in this chapter are those of the authors and no official endorsement by the IES should be inferred.


  1. Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi: 10.18637/jss.v067.i01.CrossRefGoogle Scholar
  2. Beale, E. M. L., & Little, R. J. A. (1975). Missing value in multivariate analysis. Journal of the Royal Statistical Society, Series B, 37, 129–145.Google Scholar
  3. Benton, D. (2002). Performance of the parametric bootstrap method in small sample interval estimates. Advances and Applications in Statistics, 2(3), 269–285.Google Scholar
  4. Boos, D. (2003). Introduction to the bootstrap world. Statistical Science, 18(2), 168–174.Google Scholar
  5. Burdick, R. K., & Graybill, F. A. (1988). The present status of confidence interval estimation on variance components in balanced and unbalanced random models. Communications in Statistics—Theory and Methods (Special Issue on Analysis of the Unbalanced Mixed Model), 17, 1165–1195.Google Scholar
  6. Carpenter, J. R. (2003). A novel bootstrap procedure for assessing the relationship between class size and achievement. Applied Statistics, 431–443.Google Scholar
  7. Demidenko, E. (2013). Mixed models: Theory and applications with R (2nd ed.). New York: Wiley.zbMATHGoogle Scholar
  8. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman and Hall.Google Scholar
  9. Efron, B. (1987). Better bootstrap confidence interval. Journal of American Statistical Association, 82(397). Theory and Methods.Google Scholar
  10. Efron, B. (1986). Bootstrap of the mean in the infinite variance case. The Annals of Statistics, 724–731.Google Scholar
  11. Field, C. A., Pang, Z., & Welsh, A. H. (2008). Bootstrapping data with multiple levels of variation. The Canadian Journal of Statistics, 521–539.Google Scholar
  12. Field, C. A. (2007). Bootstrapping clustered data. Journal of the Royal Statistical Society: Series B, 369–390.Google Scholar
  13. Fitzmaurice, G. M., & Laird. N. (2004). Applied longitudinal analysis. New York: Wiley.Google Scholar
  14. Harville, D. A. (1997). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72(358), 320–338.MathSciNetCrossRefzbMATHGoogle Scholar
  15. Laird, N. M., & Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics, 38(4).Google Scholar
  16. Patterson, H. D., & Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika, 58, 545–54.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Pinheiro, J. C., & Bates, D. (2000). Mixed-effects models for s and s-plus. New York: Springer.CrossRefzbMATHGoogle Scholar
  18. Scherman, M., & Cessie, S. (1997). A comparison between bootstrap methods and generalized estimating equations for correlated outcomes in generalized linear models. Communications in Statistics, Simulation and Communication, 26, 901–925.MathSciNetCrossRefzbMATHGoogle Scholar
  19. Searle, S., McCullough, C., & Casella, G. (1992). Variance components. New York: Wiley.CrossRefzbMATHGoogle Scholar
  20. Shieh, Y. Y. (2002). The application of bootstrap methodology to multilevel mixed effects linear model under conditions of error term nonnormality. Proceedings of the Joint Statistical Meetings, Biometric Section (pp. 3191–3196). Alexandria, VA: American Statistical Association.Google Scholar
  21. Thai, H. T. (2013). A comparison of bootstrap approaches for estimating uncertainty of parameters in linear mixed-effects models. Pharmaceutical Statistics.Google Scholar
  22. Thomas, J. D., & Efron, B. (1996). Bootstrap confidence intervals. Statistical Science, 11(3), 189–228.Google Scholar
  23. Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer.zbMATHGoogle Scholar
  24. Wilcox, J., & Gray, S. (2011). Efficacy of the TELL language and literacy curriculum for preschoolers with developmental speech and/or language impairment. Early Childhood Research Quarterly, 26, 278–294.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Mark Reiser
    • 1
    Email author
  • Lanlan Yao
    • 1
  • Xiao Wang
    • 2
  • Jeanne Wilcox
    • 3
  • Shelley Gray
    • 4
  1. 1.School of Mathematical and Statistical ScienceArizona State UniversityTempeUSA
  2. 2.Statistics and Data CorporationTempeUSA
  3. 3.Division of Educational Leadership and InnovationArizona State UniversityTempeUSA
  4. 4.Speech and Hearing ScienceArizona State UniversityTempeUSA

Personalised recommendations