Skip to main content
SpringerLink
Log in

Receiver Operating Characteristic (ROC) Packages Comparison in R

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2017 (ICCSA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10405))

Included in the following conference series:

Abstract

The Receiver Operating Characteristic (ROC) curve analysis and the resulting plot can be used as a tool to select optimal models of possibility and to discard those of inferior quality from the cost of context (or class distribution). Presently, this type of analysis is used in a variety of fields from the medical community, bioinformatics, military and finance. There is a variety of software packages available for ROC analysis, and this analysis will focus on those specific of R and open source. The chosen packages were: ROCR, Verification, caTools, Comp2ROC, and Epi available on CRAN, and the ROC library from Bioconductor. This work intends to make a comparative analysis of the main characteristics of these R packages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    As a rule, each of these packages brings specific dataset upon installation

References

  1. Coelho, S., Braga, A.C.: Performance evaluation of two software for analysis through ROC curves: Comp2ROC vs SPSS. In: Gervasi, O., Murgante, B., Misra, S., Gavrilova, M.L., Rocha, A.M.A.C., Torre, C., Taniar, D., Apduhan, B.O. (eds.) ICCSA 2015. LNCS, vol. 9156, pp. 144–156. Springer, Cham (2015). doi:10.1007/978-3-319-21407-8_11

    Chapter  Google Scholar 

  2. Swets, J.A.: The relative operating characteristic in psychology: a technique for isolating effects of response bias finds wide use in the study of perception and cognition. Science 182, 990–1000 (1973)

    Article  Google Scholar 

  3. Metz, C.E.: Some practical issues of experimental design and data analysis in radiological ROC studies. Invest. Radiol. 24(3), 234–45 (1989)

    Article  Google Scholar 

  4. Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.C., Müller, M.: pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 12, 77 (2011)

    Article  Google Scholar 

  5. Sing, T., Sander, O., Beerenwinkel, N., Lengauer, T.: ROCR: visualizing classifier performance in R. Bioinformatics 21, 3940–3941 (2005)

    Article  Google Scholar 

  6. Sing, T., Sander, O., Beerenwinkel, N., Lengauer, T.: ROCR: visualizing the performance of scoring classifiers. Version 1.0-7 (2015). https://cran.r-project.org/web/packages/ROCR/index.html

  7. Lasko, T., Bhagwat, J.G., Zou, K.H., Ohno-Machado, L.: Evaluation, receiver operating characteristic. Test Accuracy J. Biomed. Inform. 38, 404–415 (2005)

    Article  Google Scholar 

  8. Fawcett, T.: ROC graphs: notes and practical considerations for data mining researchers, pp. 1–27. In: HP Inven (2003)

    Google Scholar 

  9. Gonçalves, L., Subtil, A., Oliveira, M.R., Bermudez, P.Z.: ROC curve estimation: an overview. REVSTAT Stat. J. 1, 1–20 (2014)

    MathSciNet  MATH  Google Scholar 

  10. Obuchowski, N.A.: Receiver operating characteristic curves and their use in radiology. Radiology 229, 3–8 (2003)

    Article  Google Scholar 

  11. Braga, A.C., Costa, L., Oliveira, P.: ROC Curves in medical decision. In: 46th Scientific Meeting of the Italian Statistical Society (SIS), Rome, Italy, 20–22 June 2012

    Google Scholar 

  12. Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, C., Müller, M.: Package pROC: display and analyze ROC curves version. Version 1.7.2 (2015). http://web.expasy.org/pROC/files/pROC_1.7.2_R_manual.pdf

  13. NCAR: verification: weather forecast verification utilities. Version 1.42 (2015). https://cran.r-project.org/web/packages/verification/index.html

  14. Braga, A., Carvalho, S., Santiago, A.M.: Comp2ROC: compare two ROC curves that intersect. Version 1.1.4 (2016). https://cran.r-project.org/web/packages/Comp. 2ROC/Comp. 2ROC.pdf

  15. Carey, V., Redestig, H.: Utilities for ROC, with uarray focus. Version 1.48.0 (2016). https://www.bioconductor.org/packages/release/bioc/manuals/ROC/man/ROC.pdf

  16. Carstensen, B., Plummer, M., Laara, E., Hills, M.: Package ‘Epi’: a package for statistical analysis in epidemiology. Version 2.0 (2016). https://cran.r-project.org/web/packages/Epi/index.html

  17. Hornik, K.: R-FAQ (2016). https://CRAN.R-project.org/doc/FAQ/R-FAQ.html

  18. Tuszynski, J.: Package ‘caTools’: tools- moving window statistics, GIF, Base64, ROC AUC, etc. Version 1.17.1 (2015). https://cran.r-project.org/web/packages/caTools/index.html

Download references

Acknowledgments

This work was supported by FCT - (Fundação para a Ciência e Tecnologia) within the Project Scope: UID/CEC/00319/2013.

Author information

Authors and Affiliations

  1. Departamento de Engenharia Geotécnica, Instituto Superior de Engenharia do Porto, Porto, Portugal

    Daniela Ferreira da Cunha

  2. ALGORITMI Centre, University of Minho, 4710-057, Braga, Portugal

    Ana Cristina Braga

Authors
  1. Daniela Ferreira da Cunha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Cristina Braga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Cristina Braga .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Italy

    Beniamino Murgante

  3. Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Trieste, Trieste, Italy

    Giuseppe Borruso

  5. Polytechnic University of Bari, Bari, Italy

    Carmelo M. Torre

  6. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  7. Monash University, Clayton, Victoria, Australia

    David Taniar

  8. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  9. Saint Petersburg State University, Saint Petersburg, Russia

    Elena Stankova

  10. University of Trieste, Trieste, Italy

    Alfredo Cuzzocrea

Appendix

Appendix

Fig. 2.
figure 2

ROC package (blue: Reasonable, red: Useless, green: Perfect) (Color figure online)

Full size image
Fig. 3.
figure 3

ROC curve with verification package

Full size image
Fig. 4.
figure 4

Comp2ROC package (Color figure online)

Full size image
Fig. 5.
figure 5

ROC curve with caTools package for the Reasonable Test

Full size image
Fig. 6.
figure 6

ROC curve with Epi package for the Reasonable Test. The Area Under the Curve (AUC) values are given, as well as the sensitivity (Sens), specificity (Spec) and predictive values (PV) at the optimal response cut-points (Ir.eta).

Full size image
Fig. 7.
figure 7

ROC curve with Bioconductor for the Reasonable Test (blue) (Color figure online)

Full size image
Fig. 8.
figure 8

pROC package (Color figure online)

Full size image

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

da Cunha, D.F., Braga, A.C. (2017). Receiver Operating Characteristic (ROC) Packages Comparison in R. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2017. ICCSA 2017. Lecture Notes in Computer Science(), vol 10405. Springer, Cham. https://doi.org/10.1007/978-3-319-62395-5_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62395-5_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-62394-8

  • Online ISBN: 978-3-319-62395-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation