Abstract
Receiver Operating Characteristic analysis is now generally recognized as the most appropriate methodology for evaluating the diagnostic performance of medical imaging procedures (1–7). ROC analysis has been used in the field of psychophysics for three decades, and its theory and experimental methodology have been developed in considerable detail (8–13). Perhaps surprisingly, the statistical properties of ROC measures had received relatively little attention until several years ago, when the limited size of practical data sets in medical applications indicated the need for careful study of this issue. Recent progress in the statistical analysis of ROC data includes the work of Metz and Kronman (14,15), who developed a bivariate test for the statistical significance of differences between ROC curves measured from independent data sets; the work of Hanley and McNeil, who studied the statistical properties of the area under an ROC curve and developed techniques to predict the number of cases required tc demonstrate the significance of differences between ROC “Area Indexes” measured from either independent (16) or correlated (17) data sets; and the work of Swets and Pickett (7), who identified three components of variation in ROC measures and outlined a general statistical protocol for testing the significance of differences in the Area Index.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Goodenough, D.J., Rossmann, K., and Lusted, L.B.: Radiographic applications of receiver operating characteristic (ROC) curves. Radiology 110: 89, 1974.
Metz, C.E., Starr, S.J., Lusted, L.B., and Rossmann, K.: Progress in evaluation of human observer visual detection performance using the ROC curve approach. In: Information Processing in Scintigraphy (C. Raynaud and A. E. Todd-Pokropek, eds.). Orsay, France: Commissariat a l’Energie Atomique, Departement de Biologie, Service Hospitalier Frederic Joliot, 1975.
McNeil, B.J., Keeler, E., and Adelstein, S.J.: Primer on certain elements of medical decision making. N. Engl. J. Med. 293: 211, 1975.
Metz, C.E.: Basic principles of ROC analysis. Seminars Nucl. Med. 8: 283, 1978.
Turner, D.A.: An intuitive approach to receiver operating characteristic curve analysis. J. Nucl. Med. 19: 213, 1978.
Swets, J.A.: ROC analysis applied to the evaluation of medical imaging techniques. Invest. Radiol. 14: 109, 1979.
Swets, J.A. and Pickett, R.M.: Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York: Academic Press, 1982.
Tanner, W.P. Jr. and Swets, J.A.: A decision-making theory of visual detection. Psych. Rev. 61: 401, 1954.
Swets, J.A., Tanner W.P. Jr., and Birdsall, T.G.: Decision processes in perception. Psych. Rev. 68: 301, 1961.
Swets, J.A. (ed).: Signal Detection and Recognition by Human Observers. New York: Wiley, 1964.
Swets, J.A.: The relative operating characteristic in psychology. Science 182: 990, 1973.
Green, D.M. and Swets, J.A.: Signal Detection Theory and Psychophysics. (rev. ed.), Huntington NY: Krieger, 1974.
Egan, J.P.: Signal Detection Theory and ROC Analysis. New York: Academic Press, 1975.
Metz, C.E. and Kronman, H.B.: A test for the statistical significance of differences between ROC curves. In: Information Processing in Medical Imaging (R. DiPaola and E. Kahn, eds.). Paris: INSERM (Vol. 88), 1980.
Metz, C.E. and Kronman, H.B.: Statistical significance tests for binormal ROC curves. J. Math. Psych. 22: 218, 1980.
Hanley, J.A. and McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29, 1982.
Hanley, J.A. and McNeil, B.J.: A method of comparing the areas under Receiver operating characteristic curves derived from the same cases. Radiology 148: 839, 1983.
Metz, C.E.: Applications of ROC analysis in diagnostic image evaluation. In: The Physics of Medical Imaging: Recording System Measurements and Techniques (A. G. Haus, ed.). New York: Am. Inst. Physics, 1979.
Dorfman, D.D. and Alf, E.: Maximum-likelihood estimation of parameters of signal detection theory and determination of confidence intervals — rating method data. J. Math. Psych. 6: 487, 1969.
Grey, D.R. and Morgan, B.J.T.: Some aspects of ROC curve fitting: normal and logistic models. J. Math. Psych. 9: 128, 1972.
Kendall, M. and Stuart, A.: The Advanced Theory of Statistics, Vol. 2 (4th ed.). New York: MacMillan, 1979, Chapter 18.
Zelen, M. and Severo, N.C.: Probability functions. Chapter 26 in: Handbook of Mathematical Functions (M. Abramowitz and I. A. Stegun, eds.). Washington, D.C.: National Bureau of Standards, 1968.
Hanley, J.A. and McNeil, B.J.: Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Presented at the 4th Annual Meeting of the Society for Medical Decision Making, Boston, October 27, 1982. Abstracted in: Med. Decis. Making 2: 371, 1982.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1984 Martinus Nijhoff Publishers, The Hague
About this chapter
Cite this chapter
Metz, C.E., Wang, PL., Kronman, H.B. (1984). A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data. In: Deconinck, F. (eds) Information Processing in Medical Imaging. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-6045-9_25
Download citation
DOI: https://doi.org/10.1007/978-94-009-6045-9_25
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-009-6047-3
Online ISBN: 978-94-009-6045-9
eBook Packages: Springer Book Archive