Ceiling and floor effects are often observed in social and behavioral science. The current study examines ceiling/floor effects in the context of the t-test and ANOVA, two frequently used statistical methods in experimental studies. Our literature review indicated that most researchers treated ceiling or floor data as if these data were true values, and that some researchers used statistical methods such as discarding ceiling or floor data in conducting the t-test and ANOVA. The current study evaluates the performance of these conventional methods for t-test and ANOVA with ceiling or floor data. Our evaluation also includes censored regression with regard to its capacity for handling ceiling/floor data. Furthermore, we propose an easy-to-use method that handles ceiling or floor data in t-tests and ANOVA by using properties of truncated normal distributions. Simulation studies were conducted to compare the performance of the methods in handling ceiling or floor data for t-test and ANOVA. Overall, the proposed method showed greater accuracy in effect size estimation and better-controlled Type I error rates over other evaluated methods. We developed an easy-to-use software package and web applications to help researchers implement the proposed method. Recommendations and future directions are discussed.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
F* can be systematically smaller than F given large samples with small variances where F is too liberal. F* can be systematically larger than F given large samples with large variances where F is too conservative. Moreover, when cell sample sizes are equal, F and F* are identical, but the denominator degrees of freedom are different. See Maxwell et al., (2018) for more details.
Aitkin, M. A. (1964). Correlation in a singly truncated bivariate normal distribution. Psychometrika, 29(3), 263–270. https://doi.org/10.1007/BF02289723
Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31(2), 144–152. https://doi.org/10.1111/j.2044-8317.1978.tb00581.x
Brown, M. B., & Forsythe, A. B. (1974). Robust Tests for the Equality of Variances. Journal of the American Statistical Association, 69(346), 364. https://doi.org/10.2307/2285659
Chiu, Y.-C., & Egner, T. (2015). Inhibition-Induced Forgetting: When More Control Leads to Less Memory . Psychological Science , 26(1), 27–38. https://doi.org/10.1177/0956797614553945
Cohen, A. C. J. (1959). Simplified estimators for the normal distribution when samples are single censored or truncated. Technometrics, 1(3), 217–237. https://doi.org/10.2307/1266442
Coman, A., & Berry, J. N. (2015). Infectious Cognition: Risk Perception Affects Socially Shared Retrieval-Induced Forgetting of Medical Information . Psychological Science , 26(12), 1965–1971. https://doi.org/10.1177/0956797615609438
Delacre, M., Lakens, D., & Leys, C. (2017). Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test. International Review of Social Psychology, 30(1), 92–101. https://doi.org/10.5334/irsp.82
Dompnier, B., Darnon, C., Meier, E., Brandner, C., Smeding, A., & Butera, F. (2015). Improving Low Achievers’ Academic Performance at University by Changing the Social Value of Mastery Goals. American Educational Research Journal, 52(4), 720–749. https://doi.org/10.3102/0002831215585137
Fantuzzo, J. W., Gadsden, V. L., & McDermott, P. A. (2011). An Integrated Curriculum to Improve Mathematics, Language, and Literacy for Head Start Children. American Educational Research Journal, 48(3), 763–793. https://doi.org/10.3102/0002831210385446
Greene, W. H. (2002). Econometric Analysis. In Econometric Analysis.
Henningsen A. (2011). Censreg: Censored Regression (Tobit) Models. R package version 0.5, http://CRAN.R-project.org/package=censReg
Jennings, M. A., & Cribbie, R. A. (2016). Comparing Pre-Post Change Across Groups: Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores. Journal of Data Science, 14, 205–230.
Kim, R., Peters, M. A. K., & Shams, L. (2012). 0 + 1 > 1: How Adding Noninformative Sound Improves Performance on a Visual Task . Psychological Science , 23(1), 6–12. https://doi.org/10.1177/0956797611420662
Liu, Q., & Wang, L. (2018). DACF: Data Analysis with Ceiling and/or Floor Data. CRAN
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing Experiments and Analyzing Data: A Model Comparison Perspective (3rd ed.). New York: Routledge.
Miller GA. (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63(2):81–97. https://doi.org/10.1037/h0043158
Muthen, B. (1990). Moments of the censored and truncated bivariate normal distribution. British Journal of Mathematical and Statistical Psychology, 43(1), 131–143.
Muthén, L. K., & Muthén, B. O. (2002). How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power. Structural Equation Modeling: A Multidisciplinary Journal, 9(4), 599–620. https://doi.org/10.1207/S15328007SEM0904_8
Olsen, M. K., & Schafer, J. L. (2001). A Two-Part Random-Effects Model for Semicontinuous Longitudinal Data. Journal of the American Statistical Association, 96(454), 730–745. https://doi.org/10.1198/016214501753168389
Piccinin, A. M., Muniz-Terrera, G., Clouston, S., Reynolds, C. A., Thorvaldsson, V., Deary, I. J., … Spiro, A. (2013). Coordinated analysis of age, sex, and education effects on change in MMSE scores. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 68(3), 374–390.
Priebe, K., Kleindienst, N., Zimmer, J., Koudela, S., Ebner-Priemer, U., & Bohus, M. (2013). Frequency of intrusions and flashbacks in patients with posttraumatic stress disorder related to childhood sexual abuse: An electronic diary study. Psychological Assessment, 25(4), 1370–1376. https://doi.org/10.1037/a0033816
Salthouse, T. A. (2004). Localizing age-related individual differences in a hierarchical structure. Intelligence, 32(6), 541–561. https://doi.org/10.1016/j.intell.2004.07.003
Schweizer, K. (2016). A confirmatory factor model for the investigation of cognitive data showing a ceiling effect: an example. In Quantitative Psychology Research (pp. 187–197). Springer International Publishing.
Sokol-Hessner, P., Lackovic, S. F., Tobe, R. H., Camerer, C. F., Leventhal, B. L., & Phelps, E. A. (2015). Determinants of Propranolol’s Selective Effect on Loss Aversion. Psychological Science, 26(7), 1123–1130. https://doi.org/10.1177/0956797615582026
Timeo, S., Farroni, T., & Maass, A. (2017). Race and Color: Two Sides of One Story? Development of Biases in Categorical Perception. Child Development, 88(1), 83–102. https://doi.org/10.1111/cdev.12564
Tobin, J. (1958). Estimation of Relationships for Limited Dependent Variables. Econometrica, 26(1), 24–36. https://doi.org/10.2307/1907382
Ulber, J., Hamann, K., & Tomasello, M. (2016). Extrinsic Rewards Diminish Costly Sharing in 3-Year-Olds. Child Development, 87(4), 1192–1203. https://doi.org/10.1111/cdev.12534
Uttl, B. (2005). Measurement of Individual Differences. Psychological Science, 16(6), 460–467. https://doi.org/10.1111/j.0956-7976.2005.01557.x
Wang, L., & Zhang, Z. (2011). Estimating and Testing Mediation Effects with Censored Data. Structural Equation Modeling: A Multidisciplinary Journal, 18(1), 18–34. https://doi.org/10.1080/10705511.2011.534324
Wang, L., Zhang, Z., McArdle, J. J., & Salthouse, T. A. (2008). Investigating Ceiling Effects in Longitudinal Data Analysis. Multivariate Behav Res, 43(3), 476–496. https://doi.org/10.1080/00273170802285941
Welch, B. L. (1947). The generalisation of student’s problems when several different population variances are involved. Biometrika, 34(1–2), 28–35. https://doi.org/10.1093/BIOMET/34.1-2.28
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Lijuan Wang is grateful for the support from NIH 1R01HD091235, NIH 1R01HD087319, and NIH 1R01HD088482.
About this article
Cite this article
Liu, Q., Wang, L. t-Test and ANOVA for data with ceiling and/or floor effects. Behav Res 53, 264–277 (2021). https://doi.org/10.3758/s13428-020-01407-2
- Ceiling effect
- Floor effect