Advertisement

Empirical Similarity for Absent Data Generation in Imbalanced Classification

  • Arash PourhabibEmail author
Conference paper
First Online:
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 69)

Abstract

When the training data in a two-class classification problem is overwhelmed by one class, most classification techniques fail to correctly identify the data points belonging to the underrepresented class. This paper proposes Similarity-based Imbalanced Classification (SBIC) that simultaneously optimizes the weights of the empirical similarity function and identifies the locations of absent data points, i.e. unobserved data points from the minority class. Similar to cost-sensitive approaches, SBIC operates on an algorithmic level to handle imbalanced structures and similar to synthetic data generation approaches, it utilizes the properties of unobserved data points. The main contribution of the paper is to show that a similarity function can be used to tackle imbalanced datasets. The results of applying the proposed method to imbalanced datasets suggests that SBIC is comparable to, and in some cases outperforms, other commonly used classification techniques for imbalanced datasets.

Keywords

Empirical similarity Imbalanced classification Synthetic data generation 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The research was partly supported by OSU Foundation for the National Energy Solutions Institute - Smart Energy Source, grant 20-96680. This work was completed utilizing the High Performance Computing Center facilities of Oklahoma State University at Stillwater.

References

  1. 1.
    He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)CrossRefGoogle Scholar
  2. 2.
    Byon, E., Shrivastava, A.K., Ding, Y.: A classification procedure for highly imbalanced class sizes. IIE Trans. 42(4), 288–303 (2010)CrossRefGoogle Scholar
  3. 3.
    Pourhabib, A., Mallick, B.K., Ding, Y.: Absent data generating classifier for imbalanced class sizes. J. Mach. Learn. Res. 16, 2695–2724 (2015)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müllers, K.R.: Fisher discriminant analysis with kernels, in neural networks for signal processing IX. In: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41–48, August 1999Google Scholar
  5. 5.
    Gilboa, I., Lieberman, O., Schmeidler, D.: A similarity-based approach to prediction. J. Econom. 162(1), 124–131 (2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Gilboa, I., Lieberman, O., Schmeidler, D.: Empirical similarity. Rev. Econ. Stat. 88(3), 433–444 (2006)CrossRefGoogle Scholar
  7. 7.
    Park, C., Huang, J.Z., Ding, Y.: A computable plug-in estimator of minimum volume sets for novelty detection. Oper. Res. 58(5), 1469–1480 (2010)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Efron, B.: The Jackknife, the Bootstrap and Other Resampling Plans. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 38. SIAM, Philadelphia (1982)CrossRefGoogle Scholar
  9. 9.
    Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 42(4), 463–484 (2012)CrossRefGoogle Scholar
  10. 10.
    Chen, J.J., Tsai, C.A., Young, J.F., Kodell, R.L.: Classification ensembles for unbalanced class sizes in predictive toxicology. SAR QSAR Environ. Res. 16(6), 517–529 (2005)CrossRefGoogle Scholar
  11. 11.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefGoogle Scholar
  12. 12.
    Han, H., Wang, W.Y., Mao, B.H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. Advances in Intelligent Computing. Lecture Notes in Computer Science, vol. 3644, pp. 878–887. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Chen, S., He, H., Garcia, E.A.: RAMOBoost: ranked minority oversampling in boosting. IEEE Trans. Neural Netw. 21(10), 1624–1642 (2010)CrossRefGoogle Scholar
  14. 14.
    Barua, S., Islam, M.M., Yao, X., Murase, K.: MWMOTE-majority weighted minority oversampling technique for imbalanced data set learning. IEEE Trans. Knowl. Data Eng. 26(2), 405–425 (2014)CrossRefGoogle Scholar
  15. 15.
    Ramentol, E., Caballero, Y., Bello, R., Herrera, F.: SMOTE-RSB*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using smote and rough sets theory. Knowl. Inf. Syst. 33(2), 245–265 (2012)CrossRefGoogle Scholar
  16. 16.
    Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 973–978 (2001)Google Scholar
  17. 17.
    Masnadi-Shirazi, H., Vasconcelos, N.: Risk minimization, probability elicitation, and cost-sensitive SVMs. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 759–766 (2010)Google Scholar
  18. 18.
    Zhou, Z.-H., Liu, X.-Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2006)CrossRefGoogle Scholar
  19. 19.
    Sun, Y., Kamel, M.S., Wong, A.K., Wang, Y.: Cost-sensitive boosting for classification of imbalanced data. Pattern Recognit. 40(12), 3358–3378 (2007)CrossRefGoogle Scholar
  20. 20.
    Li, Q.P.: Speaker Authentication. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Xie, S., Imani, M., Dougherty, E.R., Braga-Neto, U.M.: Nonstationary linear discriminant analysis. In: 2017 51st Asilomar Conference on Signals, Systems, and Computers, pp. 161–165, October 2017Google Scholar
  22. 22.
    de Mantaras, R.L., Armengol, E.: Machine learning from examples: inductive and lazy methods. Data Knowl. Eng. 25(1), 99–123 (1998)CrossRefGoogle Scholar
  23. 23.
    Billot, A., Gilboa, I., Schmeidler, D.: Axiomatization of an exponential similarity function. Math. Soc. Sci. 55(2), 107–115 (2008)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Byrd, R.H., Gilbert, J.C., Nocedal, J.: A trust region method based on interior point techniques for nonlinear programming. Math. Program. 89(1), 149–185 (2000)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics, 2nd edn. Springer, New York (2009)CrossRefGoogle Scholar
  26. 26.
    Liu, X.-Y., Wu, J., Zhou, Z.-H.: Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybern. B Cybern. 39(2), 539–550 (2009)CrossRefGoogle Scholar
  27. 27.
    Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)CrossRefGoogle Scholar
  28. 28.
    Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. 30(7), 1145–1159 (1997)CrossRefGoogle Scholar
  29. 29.
    Veropoulos, K., Campbell, C., Cristianini, N., et al.: Controlling the sensitivity of support vector machines. In: Proceedings of the International Joint Conference on AI, pp. 55–60 (1999)Google Scholar
  30. 30.
    Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/mllastaccessed07/2014
  31. 31.
    Center for evidence-based medicine. http://www.cebm.brown.edu/static/imbalanced-datasets.zip (2014). Accessed July 2014
  32. 32.
    Dems̆ar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Industrial Engineering and ManagementOklahoma State UniversityStillwaterUSA

Personalised recommendations

Cite paper

Buy options