Adequacy of approximations to distributions of test statistics in complex mixed linear models

  • G. Bruce Schaalje
  • Justin B. McBride
  • Gilbert W. Fellingham


A recent study of lady beetle antennae was a small sample repeated measures design involving a complex covariance structure. Distributions of test statistics based on mixed models fitted to such data are unknown, but two recently developed methods for approximating the distributions of test statistics in mixed linear models have been included as options in the latest release of the MIXED procedure of SAS®. One method (FC, from Fai and Cornelius) computes degrees of freedom of an approximating F distribution for the test statistic using spectral decomposition of the hypothesis matrix together with repeated application of a method for single-degree-of-freedom tests. The other method (KR, from Kenward and Roger) adjusts the estimated covariance matrix of the parameter estimates, computes a scale adjustment to the test statistic, and computes the degrees of freedom of an approximating F distribution. Using the two methods, p values for a hypothesis of interest in the lady beetle study were quite different. Simulation studies on the Proc MIXED implementation of these methods showed that Type I error rates of both methods are affected by covariance structure complexity, sample size, and imbalance. Nonetheless, the KR method performs well in situations with fairly complicated covariance structures when sample sizes are moderate to small and the design is reasonably balanced. The KR method should be used in preference to the FC method, although it had inflated Type I error rates for complex covariance structures combined with small sample sizes.

Key words

Ante-dependence Covariance structures Degrees of freedom Residual maximum likelihood Satterth waite approximation Simulation Type I error rates 


  1. Fai, A. H. T., and Cornelius, P. L. (1996), “Approximate F-tests of Multiple Degree of Freedom Hypotheses in Generalized Least Squares Analyses of Unbalanced Split-plot Experiments,” Journal of Statistical Computing and Simulation, 54, 363–378.MATHCrossRefMathSciNetGoogle Scholar
  2. Giesbrecht, F. G., and Burns, J. C. (1985), “Two-Stage Analysis Based on a Mixed Model: Large-sample Asymptotic Theory and Small-Sample Simulation Results,” Biometrics, 41, 853–862.CrossRefGoogle Scholar
  3. Hamilton, R. M., Dogan, E. B., Schaalje, G. B., and Booth, G. M. (1999), “Olfactory Response Of The Lady Beetle Hippodamia Convergens (Coleoptera: coccinellidae) To Prey Related Odors, Including A SEM Study Of The Antennal Sensilla,” Environmental Entomology, 28, 812–822.Google Scholar
  4. Jeske, D. R., and Harville, D. A. (1988), “Prediction Interval Procedures and (Fixed-Random) Confidence-Inverval Procedures For Mixed Linear Model,” Communications in Statistics A. Theory and Methods, 17, 1053–1087.MATHCrossRefMathSciNetGoogle Scholar
  5. Kackar, R. N., and Harville, D. A. (1984), “Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models,” Journal of the American Statistical Association, 79, 853–862.MATHCrossRefMathSciNetGoogle Scholar
  6. Kenward, M. G., and Roger, J. H. (1997), “Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood,” Biometrics, 53, 983–997.MATHCrossRefGoogle Scholar
  7. Lehmann, E. (1998), Elements of Large Sample Theory, New York: Springer-Verlag.Google Scholar
  8. Littell, R. C., Milliken, G. A., Stroup, W. W., and Wolfinger, R. D. (1996), SAS System for Mixed Models, New York: SAS Institute Inc.Google Scholar
  9. McCulloch, C. E., and Searle, S. R. (2001), Generalized, Linear, and Mixed Models, New York: Wiley.MATHGoogle Scholar
  10. McLean, R. A., and Sanders, W. L. (1988), “Approximating Degrees of Freedom for Standard Errors in Mixed Linear Models,” in Proceedings of the Statistical Computing Section, Alexandria, VA: American Statistical Association, pp. 50–59.Google Scholar
  11. Patterson, H. D. and Thompson, R. (1971), “Recovery of Inter-Block Information When Block Sizes are Unequal,” Biometrika 58: 545–554.MATHCrossRefMathSciNetGoogle Scholar
  12. Ripley, B. E. (1987), Stochastic Simulation, New York: Wiley.MATHCrossRefGoogle Scholar
  13. SAS Institute, Inc. (1999), SAS version 8, Online Help, Cary, NC: SAS Institute.Google Scholar
  14. Satterthwaite, F. E. (1941), “Synthesis of Variance,” Psychometrika, 6, 309–316.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© International Biometric Society 2002

Authors and Affiliations

  • G. Bruce Schaalje
    • 1
  • Justin B. McBride
    • 2
  • Gilbert W. Fellingham
    • 3
  1. 1.Department of StatisticsBrigham Young UniversityProvo
  2. 2.3M Electronic Handling and Protection DivisionAustin
  3. 3.Department of StatisticsBrigham Young UniversityProvo

Personalised recommendations