Advertisement

Imbalance Effects on Classification Using Binary Logistic Regression

  • Hezlin Aryani Abd RahmanEmail author
  • Bee Wah Yap
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 652)

Abstract

Classification problems involving imbalance data will affect the performance of classifiers. In predictive analytics, logistic regression is a statistical technique which is often used as a benchmark when other classifiers, such as Naïve Bayes, decision tree, artificial neural network and support vector machine, are applied to a classification problem. This study investigates the effect of imbalanced ratio in the response variable on the parameter estimate of the binary logistic regression via a simulation study. Datasets were simulated with controlled different percentages of imbalance ratio (IR), from 1 % to 50 %, and for various sample sizes. The simulated datasets were then modeled using binary logistic regression. The bias in the estimates was measured using MSE (Mean Square Error). The simulation results provided evidence that imbalance ratio affects the parameter estimates where severe imbalance (IR = 1 %, 2 %, 5 %) has higher MSE. Additionally, the effects of high imbalance (IR ≤ 5 %) will be more severe when sample size is small (n = 100 & n = 500). Further investigation using real dataset from the UCI repository (Bupa Liver (n = 345) and Diabetes Messidor, n = 1151)) confirmed the imbalanced ratio effect on the parameter estimates and the odds ratio, and thus will lead to misleading results.

Keywords

Imbalance data Parameter estimates Logistic regression Simulation, predictive analytics 

Notes

Acknowledgements

Our gratitude goes to the Research Management Institute (RMI) Universiti Teknologi MARA and the Ministry of Higher Education (MOHE) Malaysia for the funding of this research under the Malaysian Fundamental Research Grant, 600- RMI/FRGS 5/3 (16/2012). We also thank Prof. Dr. Haibo He (Rhodes Island University), Prof. Dr. Ronaldo Prati (Universidade Federal do ABC), Dr. Pam Davey and Dr. Carolle Birrell (University of Wollongong) for sharing their knowledge and providing valuable comments for this study.

References

  1. 1.
    Datir, A.A., Wadhe, A.P.: Review on need of data mining techniques for biomedical field. Int. J. Comput. Inf. Technol. Bioinforma. 2, 1–5 (2014)Google Scholar
  2. 2.
    Mena, L., Gonzalez, J.A.: Machine learning for imbalanced datasets: application in medical diagnostic. In: Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2006), pp. 574–579. AAAI Press (2006). http://www.informatik.uni-trier.de/~ley/db/conf/flairs/flairs2006.html
  3. 3.
    Oztekin, A., Delen, D., Kong, Z.J.: Predicting the graft survival for heart-lung transplantation patients: an integrated data mining methodology. Int. J. Med. Inform. 78, e84–e96 (2009)CrossRefGoogle Scholar
  4. 4.
    Sathian, B.: Reporting dichotomous data using logistic regression in medical research: the scenario in developing countries. Nepal J. Epidemiol. 1, 111–113 (2011)Google Scholar
  5. 5.
    Uyar, A., Bener, A., Ciray, H., Bahceci, M.: Handling the imbalance problem of IVF implantation prediction. IAENG Int. J. Comput. Sci. 37 (2010)Google Scholar
  6. 6.
    Akbani, R., Kwek, S.S., Japkowicz, N.: Applying support vector machines to imbalanced datasets. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 39–50. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Burez, J., Van den Poel, D.: Handling class imbalance in customer churn prediction. Expert Syst. Appl. 36, 4626–4636 (2009)CrossRefGoogle Scholar
  8. 8.
    Ogwueleka, F.: Data mining application in credit card fraud detection system. J. Eng. Sci. Technol. 6, 311–322 (2011)Google Scholar
  9. 9.
    Nikulin, V., McLachlan, G.J.: Classification of imbalanced marketing data with balanced random sets. In: JLMR: Workshop and Conference Proceedings, vol. 7, pp. 89–100 (2009). http://jmlr.csail.mit.edu/proceedings/papers/v7/nikulin09/nikulin09.pdf
  10. 10.
    Sobran, N., Ahmad, A., Ibrahim, Z.: Classification of Imbalanced Dataset Using Conventional Naïve Bayes Classifier in 35–42 (2013). http://worldconferences.net/proceedings/aics2013/toc/papers_aics2013/A021-NURMAISARAHMOHDSOBRAN-ClassificationofImabalanceddatasetusingconventionalnaivebayesclassifier.pdf
  11. 11.
    Thogmartin, W.E., Knutson, M.G., Sauer, J.R.: Predicting regional abundance of rare grassland birds with a hierarchical spatial count model. Condor 108, 25–46 (2006)CrossRefGoogle Scholar
  12. 12.
    Chawla, N.V., Japkowicz, N., Kotcz, A.: Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. Newsl. 6, 1 (2004)CrossRefGoogle Scholar
  13. 13.
    Drummond, C., Holte, R.: Severe class imbalance: why better algorithms aren’t the answer. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 539–546. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    He, H., Garcia, E.E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21, 1263–1284 (2009)CrossRefGoogle Scholar
  15. 15.
    Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6, 429–449 (2002)zbMATHGoogle Scholar
  16. 16.
    Japkowicz, N.: Learning from imbalanced data sets: a comparison of various strategies. In: AAAI Workshop on Learning from Imbalanced Data Sets 0–5 (2000). doi: 10.1007/s13398-014-0173-7.2
  17. 17.
    Lemnaru, C., Potolea, R.: Imbalanced classification problems: systematic study, issues and best practices. In: Zhang, R., Zhang, J., Zhang, Z., Filipe, J., Cordeiro, J. (eds.) ICEIS 2011. LNBIP, vol. 102, pp. 35–50. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Longadge, R., Dongre, S.S., Malik, L.: Class imbalance problem in data mining review. Int. J. Comput. Sci. Netw. 2, 83–87 (2013)Google Scholar
  19. 19.
    Van Hulse, J., Khoshgoftaar, T.M., Napolitano, A.: Experimental perspectives on learning from imbalanced data. In: Proceedings of 24th International Conference on Machine Learning, pp. 935–942 (2007). doi: 10.1145/1273496.1273614
  20. 20.
    Visa, S., Ralescu, A.: Issues in mining imbalanced data sets - a review paper. In: Proceedings of the Sixteen Midwest Artificial Intelligence and Cognitive Science Conference, MAICS-2005, pp. 67–73 (2005)Google Scholar
  21. 21.
    Weiss, G.M.: Foundations of imbalanced learning. In: He, H., Ma, Y. (eds.) Imbalanced Learning: Foundations, Algorithms, Applications, pp. 13–42. Wiley & IEEE Press (2013). http://storm.cis.fordham.edu/gweiss/papers/foundations-imbalanced-13.pdf
  22. 22.
    Dong, Y., Guo, H., Zhi, W., Fan, M.: Class imbalance oriented logistic regression. In: 2014 International Conference Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 187–192 (2014). doi: 10.1109/CyberC.2014.42
  23. 23.
    Goel, G., Maguire, L., Li, Y., McLoone, S.: Evaluation of sampling methods for learning from imbalanced data. Intell. Comput. Theor. 7995, 392–401 (2013)Google Scholar
  24. 24.
    Weiss, G.M., Provost, F.: Learning when training data are costly: the effect of class distribution on tree induction. J. Arti. Intell. Res. 19, 315–354 (2003)zbMATHGoogle Scholar
  25. 25.
    Chawla, N.V.: C4. 5 and imbalanced data sets: investigating the effect of sampling method, probabilistic estimate, and decision tree structure. In: Proceedings of the International Conference on Machine Learning, Workshop Learning from Imbalanced Data Set II (2003). https://www3.nd.edu/~dial/papers/ICML03.pdf
  26. 26.
    Cohen, G., Hilario, M., Sax, H., Hugonnet, S., Geissbuhler, A.: Learning from imbalanced data in surveillance of nosocomial infection. Artif. Intell. Med. 37, 7–18 (2006)CrossRefGoogle Scholar
  27. 27.
    Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem bagging, boosting, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 99, 1–22 (2011)Google Scholar
  28. 28.
    Blagus, R., Lusa, L.: Class prediction for high-dimensional class-imbalanced data. BMC Bioinform. 11, 523 (2010)CrossRefGoogle Scholar
  29. 29.
    Anand, A., Pugalenthi, G., Fogel, G.B., Suganthan, P.N.: An approach for classification of highly imbalanced data using weighting and undersampling. Amino Acids 39, 1385–1391 (2010)CrossRefGoogle Scholar
  30. 30.
    Batista, G., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6, 20 (2004)CrossRefGoogle Scholar
  31. 31.
    Prati, R.C., Batista, G.E.A.P.A., Silva, D.F.: Class imbalance revisited: a new experimental setup to assess the performance of treatment methods. Knowl. Inf. Syst. 45, 247–270 (2014)Google Scholar
  32. 32.
    Sarmanova, A., Albayrak, S.: Alleviating class imbalance problem in data mining. In: Signal Processing and Communications Applications Conference, pp. 1–4 (2013)Google Scholar
  33. 33.
    Hosmer, D.W., Lemeshow, S.: Applied Logistic Regression Second Edition. Applied Logistic Regression (2004). doi: 10.1002/0471722146 Google Scholar
  34. 34.
    Wallace, B.C., Dahabreh, I.J.: Class Probability Estimates are Unreliable for Imbalanced Data (and How to Fix Them). ICDM (2012). http://www.cebm.brown.edu/static/papers/wallace-dahabreh-icdm-12-preprint.pdf
  35. 35.
    Hamid, H.A., Yap, B.W., Xie, X.-J., Abd Rahman, H.A.: Assessing the Effects of Different Types of Covariates for Binary Logistic Regression. 425, 425–430 (2015)Google Scholar
  36. 36.
    Forsyth, R.S.: BUPA Liver Disorders (1990). https://archive.ics.uci.edu/ml/datasets/Liver+Disorders
  37. 37.
    Antal, B., Hajdu, A.: An ensemble-based system for automatic screening of diabetic retinopathy. Knowl. Based Syst. 60, 20–27 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2016

Authors and Affiliations

  1. 1.Faculty of Computer and Mathematical Sciences, Centre of Statistical and Decision Science StudiesUniversiti Teknologi MARAShah AlamMalaysia
  2. 2.Faculty of Computer and Mathematical Sciences, Advanced Analytics Engineering CentreUniversiti Teknologi MARAShah AlamMalaysia

Personalised recommendations