The Sensitivity Analysis of Two-Level Hierarchical Linear Models to Outliers

  • Jue WangEmail author
  • Zhenqiu Lu
  • Allan S. Cohen
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)


The hierarchical linear model (HLM) has become popular in behavioral research, and has been widely used in various educational studies in recent years. Violations of model assumptions can have significant impact on the model estimates. The purpose of this study is to conduct a sensitivity analysis of two-level HLM by exploring the influence of outliers on parameter estimates of HLM under normality assumptions. A simulation study is performed to examine the bias of parameter estimates with different numbers and magnitudes of outliers given different sample sizes. Results indicated that the bias of parameter estimates increased with the magnitudes and number of outliers. The estimates have bias with a few outliers. A robust method Huber sandwich estimator corrected the standard errors efficiently when there was a large proportion of outliers.


Hierarchical linear models Outliers Robust method 


  1. Aitkin, M., Anderson, D., & Hinde, J. (1981). Statistical modeling of data on teaching styles. Journal of the Royal Statistical Society, Series A, 144(4), 419–461.CrossRefGoogle Scholar
  2. Barnett, V., & Lewis, T. (1994). Outliers in statistical data (Vol. 3). New York: Wiley.zbMATHGoogle Scholar
  3. Bijleveld, C. C. J. H., van der Kamp, L. J. T., Mooijaart, A., van der Kloot, W. A., van der Leeden, R., & van der Burg, E. (1998). Longitudinal data analysis: Designs, models and methods. Thousand Oaks, CA: Sage.Google Scholar
  4. Cronbach, L. J., & Webb, N. (1975). Between and within-class effects in a reported aptitude-by-treatment interaction: Reanalysis of a study by G. L. Anderson. Journal of Educational Psychology, 6, 717–724.Google Scholar
  5. Diggle, P., Liang, K.-Y., & Zeger, S. (1994). Analysis of longitudinal data. Oxford: Clarendon.Google Scholar
  6. Field, A. P. (2009). Discovering statistics using spss (and sex and drugs and rock ‘n’ roll) (3rd ed.). Los Angeles/London: Sage.Google Scholar
  7. Frank, K. A. (1998). Quantitative methods for studying social context in multilevels and through interpersonal relations. Review of Research in Education, 23, 171–216.Google Scholar
  8. Freedman, D. A. (2006). On the so-called ‘Huber sandwich estimator’ and ‘robust standard errors’. The American Statistician, 60(4), 299. doi: 10.2307/27643806.MathSciNetCrossRefGoogle Scholar
  9. Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11(1). 1–21.CrossRefGoogle Scholar
  10. Hogg, R. (1979). Statistical robustness: One view of its use in applications today. American Statistician, 33, 108–115.MathSciNetzbMATHGoogle Scholar
  11. Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Paper presented at the Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability.Google Scholar
  12. King, G., & Roberts, M. (2012). How robust standard errors expose methodological problems they do not fix. Annual Meeting of the Society for Political Methodology, Duke University.Google Scholar
  13. Liang, K.-Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22.MathSciNetCrossRefzbMATHGoogle Scholar
  14. Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006) ‘SAS®;’ System for Mixed Models (2nd ed.). Cary, NC: SAS Institute Inc.Google Scholar
  15. Morris, C. N. (1995). Hierarchical models for educational data: An overview. Journal of Educational and Behavioral Statistics, 20(2), 190–200.CrossRefGoogle Scholar
  16. Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression: a second course in statistics. Addison-Wesley Series in Behavioral Science: Quantitative Methods. Google Scholar
  17. Rachman-Moore, D., & Wolfe, R. G. (1984). Robust analysis of a nonlinear model for multilevel educational survey data. Journal of Educational Statistics, 9(4), 277–293.CrossRefGoogle Scholar
  18. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  19. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351–357.CrossRefGoogle Scholar
  20. SAS Institute Inc. (2013). SAS/STAT®;13.1 user’s guide. Cary, NC: SAS Institute Inc.Google Scholar
  21. Searle, S. R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York: Wiley.CrossRefzbMATHGoogle Scholar
  22. Seltzer, M. (1993). Sensitivity analysis for fixed effects in the hierarchical model: A Gibbs sampling approach. Journal of Educational Statistics, 18(3), 207–235.CrossRefGoogle Scholar
  23. Seltzer, M., & Choi, K. (2003). Sensitivity analysis for hierarchical models: Downweighting and identifying extreme cases using the t distribution. Multilevel modeling: Methodological advances, issues, and applications, 25–52.Google Scholar
  24. Seltzer, M., Novak, J., Choi, K., & Lim, N. (2002). Sensitivity analysis for hierarchical models employing “t” level-1 assumptions. Journal of Educational and Behavioral Statistics, 27(2), 181–222.CrossRefGoogle Scholar
  25. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica: Journal of the Econometric Society, 48, 817–838.MathSciNetCrossRefzbMATHGoogle Scholar
  26. White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25.MathSciNetCrossRefzbMATHGoogle Scholar
  27. Ypma, T. J. (1995). Historical development of the Newton-Raphson method. SIAM Review, 37(4), 531–551.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Zou, C., Tseng, S.-T., & Wang, Z. (2014). Outlier detection in general profiles using penalized regression method. IIE Transactions, 46(2), 106–117.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.University of GeorgiaAthensUSA

Personalised recommendations