, Volume 84, Issue 1, pp 212–235 | Cite as

Simplified Estimation and Testing in Unbalanced Repeated Measures Designs

  • Martin SpiessEmail author
  • Pascal Jordan
  • Mike Wendt


In this paper we propose a simple estimator for unbalanced repeated measures design models where each unit is observed at least once in each cell of the experimental design. The estimator does not require a model of the error covariance structure. Thus, circularity of the error covariance matrix and estimation of correlation parameters and variances are not necessary. Together with a weak assumption about the reason for the varying number of observations, the proposed estimator and its variance estimator are unbiased. As an alternative to confidence intervals based on the normality assumption, a bias-corrected and accelerated bootstrap technique is considered. We also propose the naive percentile bootstrap for Wald-type tests where the standard Wald test may break down when the number of observations is small relative to the number of parameters to be estimated. In a simulation study we illustrate the properties of the estimator and the bootstrap techniques to calculate confidence intervals and conduct hypothesis tests in small and large samples under normality and non-normality of the errors. The results imply that the simple estimator is only slightly less efficient than an estimator that correctly assumes a block structure of the error correlation matrix, a special case of which is an equi-correlation matrix. Application of the estimator and the bootstrap technique is illustrated using data from a task switch experiment based on an experimental within design with 32 cells and 33 participants.


repeated measures design unbalanced design generalized estimating equations bootstrap Wald test task switching paradigm 

Supplementary material

11336_2018_9620_MOESM1_ESM.pdf (228 kb)
Supplementary material 1 (pdf 227 KB) (109 kb)
Supplementary material 2 (zip 109 KB)


  1. Allport, D. A., Styles, E. A., & Hsieh, S. (1994). Shifting intentional set: Exploring the dynamic control of tasks. In C. Umilta & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 421–452). Cambridge, MA: MIT Press.Google Scholar
  2. Bond, S., & Windmeijer, F. (2005). Reliable inference for GMM estimators? Finite sample properties of alternative test procedures in linear panel models. Econometric Reviews, 24(1), 1–37.CrossRefGoogle Scholar
  3. Bradley, R. C. (1981). Central limit theorems under weak dependence. Journal of Multivariate Analysis, 11, 1–16.CrossRefGoogle Scholar
  4. Cheng, G., Yu, Z., & Huang, J. Z. (2013). The cluster bootstrap consistency in generalized estimating equations. Journal of Multivariate Analysis, 115, 33–47.CrossRefGoogle Scholar
  5. Chung, K. L. (2001). A course in probability theory (3rd ed.). San Diego: Academic Press.Google Scholar
  6. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.CrossRefGoogle Scholar
  7. Fitts, P. M., & Deininger, R. L. (1954). S-R compatibility: Correspondence among paired elements within stimulus and response codes. Journal of Experimental Psychology, 48, 483–492.CrossRefGoogle Scholar
  8. Greenhouse, S. W., & Geisser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24(2), 95–112.CrossRefGoogle Scholar
  9. Hall, P., & Horowitz, J. L. (1996). Bootstrap critical values for tests based on generalized-method-of-moments estimators. Econometrica, 64(4), 891–916.CrossRefGoogle Scholar
  10. Huynh, H., & Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1(1), 69–82.CrossRefGoogle Scholar
  11. Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., et al. (2010). Control and interference in task switching—A review. Psychological Bulletin, 136, 849–874.CrossRefGoogle Scholar
  12. Klass, M., & Teicher, H. (1987). The central limit theorem for exchangeable random variables without moments. The Annals of Probability, 15, 138–151.CrossRefGoogle Scholar
  13. Liang, K.-Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22.CrossRefGoogle Scholar
  14. MacKinnon, J. G. (2002). Bootstrap inference in econometrics. Canadian Journal of Economics, 35, 615–645.CrossRefGoogle Scholar
  15. Mauchly, J. W. (1940). Significance test for sphericity of a normal \(n\)-variate distribution. Annals of Mathematical Statistics, 11, 204–209.CrossRefGoogle Scholar
  16. McDonald, R. P. (1976). The McDonald–Swaminathan matrix calculus: Clarifications, extensions, and illustrations. General Systems, 21, 87–94.Google Scholar
  17. McDonald, R. P., & Swaminathan, H. (1973). A simple calculus with applications to multivariate analysis. General Systems, 18, 37–54.Google Scholar
  18. Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442.Google Scholar
  19. Newey, W. H., & McFadden, D. (1994). Large sample estimation and hypothesis testing. In R. F. Engle & D. L. McFadden (Eds.), Handbook of econometrics (Vol. IV, pp. 2111–2245). Amsterdam: Elsevier.Google Scholar
  20. Newman, C. H. (1984). Asymptotic independence and limit theorems for positively and negatively dependent random variables. Inequalities in Statistics and Probability, IMS Lecture Notes-Monograph Series, 5, 127–140.CrossRefGoogle Scholar
  21. Proctor, R. W., & Vu, K.-P. L. (2006). Stimulus-response compatibility principles: Data, theory, and application. Boca Raton, FL: CRC Press.Google Scholar
  22. R Core Team. (2018). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
  23. Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1995). Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the American Statistical Association, 90(429), 106–121.CrossRefGoogle Scholar
  24. Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, 207–231.CrossRefGoogle Scholar
  25. Rotnitzky, A., & Jewell, N. P. (1990). Hypothesis testing of regression parameters in semiparametric generalized linear models for cluster correlated data. Biometrika, 77(3), 485–497.CrossRefGoogle Scholar
  26. Schneider, D. (2015). Attentional control of response selection in task switching. Journal of Experimental Psychology: Human Perception and Performance, 41, 1315–1324.Google Scholar
  27. Spiess, M. (2006). Estimation of a two-equation panel model with mixed continuous and ordered categorical outcomes and missing data. Applied Statistics, 55(4), 525–538.Google Scholar
  28. Spiess, M., & Hamerle, A. (1996). On the properties of GEE estimators in the presence of invariant covariates. Biometrical Journal, 38(8), 931–940.CrossRefGoogle Scholar
  29. Swaminathan, H. (1976). Matrix calculus for functions of partitioned matrices. General Systems, 21, 95–98.Google Scholar
  30. Vandierendonck, A., Liefooghe, B., & Verbruggen, F. (2010). Task switching: Interplay of reconfiguration and interference control. Psychological Bulletin, 136, 601–626.CrossRefGoogle Scholar
  31. Wilcox, R. R. (2017). Introduction to robust estimation and hypothesis testing (4th ed.). San Diego: Academic Press.Google Scholar
  32. Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.). New York: McGraw-Hill.Google Scholar
  33. Zyskind, G. (1967). On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. The Annals of Mathematical Statistics, 38, 1092–1109.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2018

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of HamburgHamburgGermany
  2. 2.Faculty of Human SciencesMedical School HamburgHamburgGermany

Personalised recommendations