Skip to main content
SpringerLink
Log in

Imbalance Effects on Classification Using Binary Logistic Regression

  • Conference paper
  • First Online:
Soft Computing in Data Science (SCDS 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 652))

Included in the following conference series:

Abstract

Classification problems involving imbalance data will affect the performance of classifiers. In predictive analytics, logistic regression is a statistical technique which is often used as a benchmark when other classifiers, such as Naïve Bayes, decision tree, artificial neural network and support vector machine, are applied to a classification problem. This study investigates the effect of imbalanced ratio in the response variable on the parameter estimate of the binary logistic regression via a simulation study. Datasets were simulated with controlled different percentages of imbalance ratio (IR), from 1 % to 50 %, and for various sample sizes. The simulated datasets were then modeled using binary logistic regression. The bias in the estimates was measured using MSE (Mean Square Error). The simulation results provided evidence that imbalance ratio affects the parameter estimates where severe imbalance (IR = 1 %, 2 %, 5 %) has higher MSE. Additionally, the effects of high imbalance (IR ≤ 5 %) will be more severe when sample size is small (n = 100 & n = 500). Further investigation using real dataset from the UCI repository (Bupa Liver (n = 345) and Diabetes Messidor, n = 1151)) confirmed the imbalanced ratio effect on the parameter estimates and the odds ratio, and thus will lead to misleading results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Datir, A.A., Wadhe, A.P.: Review on need of data mining techniques for biomedical field. Int. J. Comput. Inf. Technol. Bioinforma. 2, 1–5 (2014)

    Google Scholar 

  2. Mena, L., Gonzalez, J.A.: Machine learning for imbalanced datasets: application in medical diagnostic. In: Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2006), pp. 574–579. AAAI Press (2006). http://www.informatik.uni-trier.de/~ley/db/conf/flairs/flairs2006.html

  3. Oztekin, A., Delen, D., Kong, Z.J.: Predicting the graft survival for heart-lung transplantation patients: an integrated data mining methodology. Int. J. Med. Inform. 78, e84–e96 (2009)

    Article  Google Scholar 

  4. Sathian, B.: Reporting dichotomous data using logistic regression in medical research: the scenario in developing countries. Nepal J. Epidemiol. 1, 111–113 (2011)

    Google Scholar 

  5. Uyar, A., Bener, A., Ciray, H., Bahceci, M.: Handling the imbalance problem of IVF implantation prediction. IAENG Int. J. Comput. Sci. 37 (2010)

    Google Scholar 

  6. Akbani, R., Kwek, S.S., Japkowicz, N.: Applying support vector machines to imbalanced datasets. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 39–50. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  7. Burez, J., Van den Poel, D.: Handling class imbalance in customer churn prediction. Expert Syst. Appl. 36, 4626–4636 (2009)

    Article  Google Scholar 

  8. Ogwueleka, F.: Data mining application in credit card fraud detection system. J. Eng. Sci. Technol. 6, 311–322 (2011)

    Google Scholar 

  9. Nikulin, V., McLachlan, G.J.: Classification of imbalanced marketing data with balanced random sets. In: JLMR: Workshop and Conference Proceedings, vol. 7, pp. 89–100 (2009). http://jmlr.csail.mit.edu/proceedings/papers/v7/nikulin09/nikulin09.pdf

  10. Sobran, N., Ahmad, A., Ibrahim, Z.: Classification of Imbalanced Dataset Using Conventional Naïve Bayes Classifier in 35–42 (2013). http://worldconferences.net/proceedings/aics2013/toc/papers_aics2013/A021-NURMAISARAHMOHDSOBRAN-ClassificationofImabalanceddatasetusingconventionalnaivebayesclassifier.pdf

  11. Thogmartin, W.E., Knutson, M.G., Sauer, J.R.: Predicting regional abundance of rare grassland birds with a hierarchical spatial count model. Condor 108, 25–46 (2006)

    Article  Google Scholar 

  12. Chawla, N.V., Japkowicz, N., Kotcz, A.: Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. Newsl. 6, 1 (2004)

    Article  Google Scholar 

  13. Drummond, C., Holte, R.: Severe class imbalance: why better algorithms aren’t the answer. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 539–546. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  14. He, H., Garcia, E.E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21, 1263–1284 (2009)

    Article  Google Scholar 

  15. Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6, 429–449 (2002)

    MATH  Google Scholar 

  16. Japkowicz, N.: Learning from imbalanced data sets: a comparison of various strategies. In: AAAI Workshop on Learning from Imbalanced Data Sets 0–5 (2000). doi:10.1007/s13398-014-0173-7.2

  17. Lemnaru, C., Potolea, R.: Imbalanced classification problems: systematic study, issues and best practices. In: Zhang, R., Zhang, J., Zhang, Z., Filipe, J., Cordeiro, J. (eds.) ICEIS 2011. LNBIP, vol. 102, pp. 35–50. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  18. Longadge, R., Dongre, S.S., Malik, L.: Class imbalance problem in data mining review. Int. J. Comput. Sci. Netw. 2, 83–87 (2013)

    Google Scholar 

  19. Van Hulse, J., Khoshgoftaar, T.M., Napolitano, A.: Experimental perspectives on learning from imbalanced data. In: Proceedings of 24th International Conference on Machine Learning, pp. 935–942 (2007). doi:10.1145/1273496.1273614

  20. Visa, S., Ralescu, A.: Issues in mining imbalanced data sets - a review paper. In: Proceedings of the Sixteen Midwest Artificial Intelligence and Cognitive Science Conference, MAICS-2005, pp. 67–73 (2005)

    Google Scholar 

  21. Weiss, G.M.: Foundations of imbalanced learning. In: He, H., Ma, Y. (eds.) Imbalanced Learning: Foundations, Algorithms, Applications, pp. 13–42. Wiley & IEEE Press (2013). http://storm.cis.fordham.edu/gweiss/papers/foundations-imbalanced-13.pdf

  22. Dong, Y., Guo, H., Zhi, W., Fan, M.: Class imbalance oriented logistic regression. In: 2014 International Conference Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 187–192 (2014). doi:10.1109/CyberC.2014.42

  23. Goel, G., Maguire, L., Li, Y., McLoone, S.: Evaluation of sampling methods for learning from imbalanced data. Intell. Comput. Theor. 7995, 392–401 (2013)

    Google Scholar 

  24. Weiss, G.M., Provost, F.: Learning when training data are costly: the effect of class distribution on tree induction. J. Arti. Intell. Res. 19, 315–354 (2003)

    MATH  Google Scholar 

  25. Chawla, N.V.: C4. 5 and imbalanced data sets: investigating the effect of sampling method, probabilistic estimate, and decision tree structure. In: Proceedings of the International Conference on Machine Learning, Workshop Learning from Imbalanced Data Set II (2003). https://www3.nd.edu/~dial/papers/ICML03.pdf

  26. Cohen, G., Hilario, M., Sax, H., Hugonnet, S., Geissbuhler, A.: Learning from imbalanced data in surveillance of nosocomial infection. Artif. Intell. Med. 37, 7–18 (2006)

    Article  Google Scholar 

  27. Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem bagging, boosting, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 99, 1–22 (2011)

    Google Scholar 

  28. Blagus, R., Lusa, L.: Class prediction for high-dimensional class-imbalanced data. BMC Bioinform. 11, 523 (2010)

    Article  Google Scholar 

  29. Anand, A., Pugalenthi, G., Fogel, G.B., Suganthan, P.N.: An approach for classification of highly imbalanced data using weighting and undersampling. Amino Acids 39, 1385–1391 (2010)

    Article  Google Scholar 

  30. Batista, G., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6, 20 (2004)

    Article  Google Scholar 

  31. Prati, R.C., Batista, G.E.A.P.A., Silva, D.F.: Class imbalance revisited: a new experimental setup to assess the performance of treatment methods. Knowl. Inf. Syst. 45, 247–270 (2014)

    Google Scholar 

  32. Sarmanova, A., Albayrak, S.: Alleviating class imbalance problem in data mining. In: Signal Processing and Communications Applications Conference, pp. 1–4 (2013)

    Google Scholar 

  33. Hosmer, D.W., Lemeshow, S.: Applied Logistic Regression Second Edition. Applied Logistic Regression (2004). doi:10.1002/0471722146

    Google Scholar 

  34. Wallace, B.C., Dahabreh, I.J.: Class Probability Estimates are Unreliable for Imbalanced Data (and How to Fix Them). ICDM (2012). http://www.cebm.brown.edu/static/papers/wallace-dahabreh-icdm-12-preprint.pdf

  35. Hamid, H.A., Yap, B.W., Xie, X.-J., Abd Rahman, H.A.: Assessing the Effects of Different Types of Covariates for Binary Logistic Regression. 425, 425–430 (2015)

    Google Scholar 

  36. Forsyth, R.S.: BUPA Liver Disorders (1990). https://archive.ics.uci.edu/ml/datasets/Liver+Disorders

  37. Antal, B., Hajdu, A.: An ensemble-based system for automatic screening of diabetic retinopathy. Knowl. Based Syst. 60, 20–27 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

Our gratitude goes to the Research Management Institute (RMI) Universiti Teknologi MARA and the Ministry of Higher Education (MOHE) Malaysia for the funding of this research under the Malaysian Fundamental Research Grant, 600- RMI/FRGS 5/3 (16/2012). We also thank Prof. Dr. Haibo He (Rhodes Island University), Prof. Dr. Ronaldo Prati (Universidade Federal do ABC), Dr. Pam Davey and Dr. Carolle Birrell (University of Wollongong) for sharing their knowledge and providing valuable comments for this study.

Author information

Authors and Affiliations

  1. Faculty of Computer and Mathematical Sciences, Centre of Statistical and Decision Science Studies, Universiti Teknologi MARA, 40450, Shah Alam, Selangor, Malaysia

    Hezlin Aryani Abd Rahman

  2. Faculty of Computer and Mathematical Sciences, Advanced Analytics Engineering Centre, Universiti Teknologi MARA, 40450, Shah Alam, Selangor, Malaysia

    Bee Wah Yap

Authors
  1. Hezlin Aryani Abd Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bee Wah Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hezlin Aryani Abd Rahman .

Editor information

Editors and Affiliations

  1. University of Tennessee, Knoxville, Tennessee, USA

    Michael W. Berry

  2. Universiti Teknologi MARA, Shah Alam, Malaysia

    Azlinah Hj. Mohamed

  3. Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Shah Alam, Malaysia

    Bee Wah Yap

Appendix

Appendix

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Abd Rahman, H.A., Yap, B.W. (2016). Imbalance Effects on Classification Using Binary Logistic Regression. In: Berry, M., Hj. Mohamed, A., Yap, B. (eds) Soft Computing in Data Science. SCDS 2016. Communications in Computer and Information Science, vol 652. Springer, Singapore. https://doi.org/10.1007/978-981-10-2777-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-2777-2_12

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-2776-5

  • Online ISBN: 978-981-10-2777-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation