Abstract
The AUC (area under ROC curve) is a commonly used metric to assess discrimination of risk prediction rules; however, standard errors of AUC are usually based on the Mann–Whitney U test that assumes independence of sampling units. For ophthalmologic applications, it is desirable to assess risk prediction rules based on eye-specific outcome variables which are generally highly, but not perfectly correlated in fellow eyes [e.g. progression of individual eyes to age-related macular degeneration (AMD)]. In this article, we use the extended Mann–Whitney U test (Rosner and Glynn, Biometrics 65:188–197, 2009) for the case where subunits within a cluster may have different progression status and assess discrimination of different prediction rules in this setting. Both data analyses based on progression of AMD and simulation studies show reasonable accuracy of this extended Mann–Whitney U test to assess discrimination of eye-specific risk prediction rules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Diagn Radiol 143:29–36
Hodges JL Jr, Lehmann EL (1956) The efficiency of some nonparametric competitors of the t test. Ann Math Stat 27:324–335
Kendall MG, Stuart A (1969) The advanced theory of statistics. Hafner, New York
Li G, Zhou K (2008) A unified approach to nonparametric comparison of receiver operating characteristic curves for longitudinal and clustered data. J Am Stat Assoc 103(482):705–713
National Eye Institute (2011) Age-related macular degeneration. http://www.nei.nih.gov/health/maculardegen/armd_facts.asp. Retrieved March 2011
Obuchowski NA (2003) Receiver operating characteristic curves and their use in radiology. Radiology 229:3–8
Obuchowski NA, McClish DK (1997) Sample size determination for diagnostic accuracy studies involving binormal roc curve indices. Stat Med 16(13):1529–1542
Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the roc curve to reclassification and beyond. Stat Med 27:157–172
Rosner B, Glynn RJ (2009) Power and sample size estimation for the wilcoxon rank sum test with application to comparisons of c statistics from alternative prediction models. Biometrics 65:188–97
Rosner B, Glynn RJ, Lee MT (2006) Extension of the rank sum test for clustered data: two-group comparisons with group membership defined at the subunit level. Biometrics 62:1251–1259
Rubin DB (1987) Multiple imputation for non-response in surveys. Wiley, New York
Seddon JM, Cote J, Rosner B (2003) Progression of age-related macular degeneration. Arch Ophthalmol 121:1728–1737
Toledano AY, Gatsonis CA (1996) GEEs for ordinal categorical data: arbitrary patterns of missing responses and missingness in a key covariate. Biometrics 55:488–496
Zou KH, O’Malley AJ, Mauri L (2007) Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 115:654–657
Acknowledgments
This work was supported by the National Institutes of Health Grant EY12269 from the National Eye Institute.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Rosner, B., Qiu, W. & Lee, ML.T. Assessing discrimination of risk prediction rules in a clustered data setting. Lifetime Data Anal 19, 242–256 (2013). https://doi.org/10.1007/s10985-012-9240-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10985-012-9240-6