Abstract
We address the problem of binary classification when applied to non-communicable diseases. In such problems the data are typically skewed towards samples of healthy subjects. Because of this, traditional performance metrics (such as accuracy) are not suitable. Furthermore, classifiers are typically trained with the assumption that the benefit or cost associated with decision outcomes are the same. In the case of non-communicable diseases this is not necessarily the case since it is more important to err on the side of treatment of the disease rather on the side of over-diagnosis. In this paper we consider the use of benefits/costs for evaluation of classifiers and we also propose how the Logistic Regression cost function can be modified to account for these benefits and costs for better training to achieve the desired goal. We then illustrate the advantage of the approach for the case of identifying diabetes and breast cancer.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Association, American Diabetes: Statistics about diabetes, July 2017. http://www.diabetes.org/diabetes-basics/statistics/
Bahnsen, A.C., Aouada, D., Ottersten, B.: Example-dependent cost-sensitive logistic regression for credit scoring. In: Proceedings of the 2014 13th International Conference on Machine Learning and Applications ICMLA 2014, pp. 263–269. IEEE Computer Society, Washington, DC, USA (2014). https://doi.org/10.1109/ICMLA.2014.48. http://dx.doi.org/10.1109/ICMLA.2014.48
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Int. Res. 16(1), 321–357 (2002). http://dl.acm.org/citation.cfm?id=1622407.1622416
Diabetes.co.uk: Diabetes life expectancy. https://www.diabetes.co.uk/diabetes-life-expectancy.html
Garrido, F., Verbeke, W., Bravo, C.: A robust profit measure for binary classification model evaluation. Expert Syst. Appl. 92, 154–160 (2018). https://doi.org/10.1016/j.eswa.2017.09.045, http://www.sciencedirect.com/science/article/pii/S0957417417306498
Gundgurthi, A., Kharb, S., Dutta, M.K., Pakhetra, R., Garg, M.K.: Insulin poisoning with suicidal intent. Indian J. Endocrinol. Metabol. 16(Suppl1), S120 – S122 (2012). https://doi.org/10.4103/2230-8210.94254, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3354941/
Haixiang, G., Yijing, L., Shang, J., Mingyun, G., Yuanyue, H., Bing, G.: Learning from class-imbalanced data. Expert Syst. Appl. 73(C), 220–239 (2017). https://doi.org/10.1016/j.eswa.2016.12.035
He, F., Yang, H., Miao, Y., Louis, R.: A cost sensitive and class-imbalance classification method based on neural network for disease diagnosis. In: 2016 8th International Conference on Information Technology in Medicine and Education (ITME), pp. 7–10, December 2016. https://doi.org/10.1109/ITME.2016.0012
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009). https://doi.org/10.1109/TKDE.2008.239
Helper, Health Cost: How much does diabetes medication cost? October 2013. http://health.costhelper.com/diabetes-medication.html
Huizen, J.: Type 2 diabetes and life expectancy, May 2017. https://www.medicalnewstoday.com/articles/317477.php
Institute, Health Care Cost: Issue brief 10: Per capita health care spending on diabetes: 2009–2013, May 2015. http://www.healthcostinstitute.org/files/HCCIDiabetesIssueBrief205-7-15.pdf
Kaggle.com: Pima Indians diabetes database. https://www.kaggle.com/uciml/pima-indians-diabetes-database/data
Kavakiotis, I., Tsave, O., Salifoglou, A., Maglaveras, N., Vlahavas, I., Chouvarda, I.: Machine learning and data mining methods in diabetes research. Comput. Struct. Biotechnol. J. 15, 104–116 (2017). https://doi.org/10.1016/j.csbj.2016.12.005, http://www.sciencedirect.com/science/article/pii/S2001037016300733
Krawczyk, B., Schaefer, G., Woźniak, M.: A cost-sensitive ensemble classifier for breast cancer classification. In: 2013 IEEE 8th International Symposium on Applied Computational Intelligence and Informatics (SACI), pp. 427–430, May 2013. https://doi.org/10.1109/SACI.2013.6609012
Li, L., Chen, M., Wang, H., Li, H.: Cosfuc: a cost sensitive fuzzy clustering approach for medical prediction. In: 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 2, pp. 127–131, October 2008. https://doi.org/10.1109/FSKD.2008.378
Repository U.M.L: Breast cancer wisconsin (diagnostic) data set, November 1995. https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)
Santos-RodrÃguez, R., GarcÃa-GarcÃa, D., Cid-Sueiro, J.: Cost-sensitive classification based on bregman divergences for medical diagnosis. In: 2009 International Conference on Machine Learning and Applications, pp. 551–556, December 2009. https://doi.org/10.1109/ICMLA.2009.82
Smith, J.W., Everhart, J., Dickson, W., Knowler, W., Johannes, R.: Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In: Proceedings of the Annual Symposium on Computer Application in Medical Care, pp. 261–265, November 1988
Verbraken, T., Verbeke, W., Baesens, B.: A novel profit maximizing metric for measuring classification performance of customer churn prediction models. IEEE Trans. Knowl. Data Eng. 25, 961–973 (2013)
Zhang, D., Shen, D.: Multicost: multi-stage cost-sensitive classification of alzheimer’s disease. In: Suzuki, K., Wang, F., Shen, D., Yan, P. (eds.) Machine Learning in Medical Imaging, pp. 344–351. Springer, Heidelberg (2011)
Zhou, Z.H., Liu, X.Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2006). https://doi.org/10.1109/TKDE.2006.17
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sooklal, S., Hosein, P. (2020). A Benefit Optimization Approach to the Evaluation of Classification Algorithms. In: Hemanth, D., Kose, U. (eds) Artificial Intelligence and Applied Mathematics in Engineering Problems. ICAIAME 2019. Lecture Notes on Data Engineering and Communications Technologies, vol 43. Springer, Cham. https://doi.org/10.1007/978-3-030-36178-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-36178-5_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36177-8
Online ISBN: 978-3-030-36178-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)