Advertisement

Receiver Operating Characteristic (ROC) Packages Comparison in R

  • Daniela Ferreira da Cunha
  • Ana Cristina BragaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10405)

Abstract

The Receiver Operating Characteristic (ROC) curve analysis and the resulting plot can be used as a tool to select optimal models of possibility and to discard those of inferior quality from the cost of context (or class distribution). Presently, this type of analysis is used in a variety of fields from the medical community, bioinformatics, military and finance. There is a variety of software packages available for ROC analysis, and this analysis will focus on those specific of R and open source. The chosen packages were: ROCR, Verification, caTools, Comp2ROC, and Epi available on CRAN, and the ROC library from Bioconductor. This work intends to make a comparative analysis of the main characteristics of these R packages.

Keywords

ROC curves Comp2ROC ROCR pROC caTools Verification Bioconductor 

Notes

Acknowledgments

This work was supported by FCT - (Fundação para a Ciência e Tecnologia) within the Project Scope: UID/CEC/00319/2013.

References

  1. 1.
    Coelho, S., Braga, A.C.: Performance evaluation of two software for analysis through ROC curves: Comp2ROC vs SPSS. In: Gervasi, O., Murgante, B., Misra, S., Gavrilova, M.L., Rocha, A.M.A.C., Torre, C., Taniar, D., Apduhan, B.O. (eds.) ICCSA 2015. LNCS, vol. 9156, pp. 144–156. Springer, Cham (2015). doi: 10.1007/978-3-319-21407-8_11 CrossRefGoogle Scholar
  2. 2.
    Swets, J.A.: The relative operating characteristic in psychology: a technique for isolating effects of response bias finds wide use in the study of perception and cognition. Science 182, 990–1000 (1973)CrossRefGoogle Scholar
  3. 3.
    Metz, C.E.: Some practical issues of experimental design and data analysis in radiological ROC studies. Invest. Radiol. 24(3), 234–45 (1989)CrossRefGoogle Scholar
  4. 4.
    Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.C., Müller, M.: pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 12, 77 (2011)CrossRefGoogle Scholar
  5. 5.
    Sing, T., Sander, O., Beerenwinkel, N., Lengauer, T.: ROCR: visualizing classifier performance in R. Bioinformatics 21, 3940–3941 (2005)CrossRefGoogle Scholar
  6. 6.
    Sing, T., Sander, O., Beerenwinkel, N., Lengauer, T.: ROCR: visualizing the performance of scoring classifiers. Version 1.0-7 (2015). https://cran.r-project.org/web/packages/ROCR/index.html
  7. 7.
    Lasko, T., Bhagwat, J.G., Zou, K.H., Ohno-Machado, L.: Evaluation, receiver operating characteristic. Test Accuracy J. Biomed. Inform. 38, 404–415 (2005)CrossRefGoogle Scholar
  8. 8.
    Fawcett, T.: ROC graphs: notes and practical considerations for data mining researchers, pp. 1–27. In: HP Inven (2003)Google Scholar
  9. 9.
    Gonçalves, L., Subtil, A., Oliveira, M.R., Bermudez, P.Z.: ROC curve estimation: an overview. REVSTAT Stat. J. 1, 1–20 (2014)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Obuchowski, N.A.: Receiver operating characteristic curves and their use in radiology. Radiology 229, 3–8 (2003)CrossRefGoogle Scholar
  11. 11.
    Braga, A.C., Costa, L., Oliveira, P.: ROC Curves in medical decision. In: 46th Scientific Meeting of the Italian Statistical Society (SIS), Rome, Italy, 20–22 June 2012Google Scholar
  12. 12.
    Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, C., Müller, M.: Package pROC: display and analyze ROC curves version. Version 1.7.2 (2015). http://web.expasy.org/pROC/files/pROC_1.7.2_R_manual.pdf
  13. 13.
    NCAR: verification: weather forecast verification utilities. Version 1.42 (2015). https://cran.r-project.org/web/packages/verification/index.html
  14. 14.
    Braga, A., Carvalho, S., Santiago, A.M.: Comp2ROC: compare two ROC curves that intersect. Version 1.1.4 (2016). https://cran.r-project.org/web/packages/Comp. 2ROC/Comp. 2ROC.pdf
  15. 15.
    Carey, V., Redestig, H.: Utilities for ROC, with uarray focus. Version 1.48.0 (2016). https://www.bioconductor.org/packages/release/bioc/manuals/ROC/man/ROC.pdf
  16. 16.
    Carstensen, B., Plummer, M., Laara, E., Hills, M.: Package ‘Epi’: a package for statistical analysis in epidemiology. Version 2.0 (2016). https://cran.r-project.org/web/packages/Epi/index.html
  17. 17.
  18. 18.
    Tuszynski, J.: Package ‘caTools’: tools- moving window statistics, GIF, Base64, ROC AUC, etc. Version 1.17.1 (2015). https://cran.r-project.org/web/packages/caTools/index.html

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Daniela Ferreira da Cunha
    • 1
  • Ana Cristina Braga
    • 2
    Email author
  1. 1.Departamento de Engenharia GeotécnicaInstituto Superior de Engenharia do PortoPortoPortugal
  2. 2.ALGORITMI CentreUniversity of MinhoBragaPortugal

Personalised recommendations