Hybrid of Intelligent Minority Oversampling and PSO-Based Intelligent Majority Undersampling for Learning from Imbalanced Datasets

  • Seba SusanEmail author
  • Amitesh Kumar
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 941)


Learning from imbalanced datasets poses a major research challenge today due to the imbalanced nature of real-world datasets where samples of some entities are few in number, while some other entities have thousands of samples available. A novel hybrid scheme of intelligently oversampling the minority class followed by subsequent intelligent undersampling of the majority class, is proposed in this paper for learning from imbalanced datasets. Different oversampling techniques: SMOTE and the intelligent oversampling versions of Borderline-SMOTE, Adaptive Synthetic Sampling (ADASYN) and MWMOTE, are considered in combination with Sample Subset Optimization (SSO) that is an intelligent majority undersampling technique based on the evolutionary optimization algorithm of Particle Swarm Optimization (PSO). The datasets after balance-correction are applied to the decision tree classifier. Experiments on benchmark datasets from the UCI repository prove the efficiency of our method due to the higher classification accuracies obtained as compared to the baseline methods.


Imbalanced learning Imbalanced datasets Oversampling Undersampling 


  1. 1.
    Yan, B., Han, G., Sun, M., Ye, S.: A novel region adaptive SMOTE algorithm for intrusion detection on imbalanced problem. In: 2017 3rd IEEE International Conference on Computer and Communications (ICCC), pp. 1281–1286. IEEE (2017)Google Scholar
  2. 2.
    Zhang, W., Kobeissi, S., Tomko, S., Challis, C.: Adaptive sampling scheme for learning in severely imbalanced large scale data. In: Asian Conference on Machine Learning, pp. 240–247 (2017)Google Scholar
  3. 3.
    Bunkhumpornpat, C., Sinapiromsaran, K.: CORE: core-based synthetic minority over-sampling and borderline majority under-sampling technique. Int. J. Data Mining Bioinf. 12(1), 44–58 (2015)CrossRefGoogle Scholar
  4. 4.
    Huang, C.-L.: A particle-based simplified swarm optimization algorithm for reliability redundancy allocation problems. Reliab. Eng. Syst. Saf. 142, 221–230 (2015)CrossRefGoogle Scholar
  5. 5.
    Yang, P., Yoo, P.D., Fernando, J., Zhou, B.B., Zhang, Z., Zomaya, A.Y.: Sample subset optimization techniques for imbalanced and ensemble learning problems in bioinformatics applications. IEEE Trans. Cybern. 44(3), 445–455 (2014)CrossRefGoogle Scholar
  6. 6.
    Lunardon, N., Menardi, G., Torelli, N.: ROSE: a package for binary imbalanced learning. R J. 6(1) (2014)Google Scholar
  7. 7.
    Peng, Y., Yao, J.: AdaOUBoost: adaptive over-sampling and under-sampling to boost the concept learning in large scale imbalanced datasets. In: Proceedings of the International Conference on Multimedia Information Retrieval, pp. 111–118. ACM (2010)Google Scholar
  8. 8.
    Del Valle, Y., Venayagamoorthy, G.K., Mohagheghi, S., Hernandez, J.-C., Harley, R.G.: Particle swarm optimization: basic concepts, variants and applications in power systems. IEEE Trans. Evol. Comput. 12(2), 171–195 (2008)CrossRefGoogle Scholar
  9. 9.
    Mease, D., Wyner, A.J., Buja, A.: Boosted classification trees and class probability/quantile estimation. J. Mach. Learn. Res. 8(Mar), 409–439 (2007)zbMATHGoogle Scholar
  10. 10.
    He, H., Bai, Y., Garcia, E.A., Li, S.: ADASYN: adaptive synthetic sampling approach for imbalanced learning. In: IEEE International Joint Conference on Neural Networks, 2008, IJCNN 2008, (IEEE World Congress on Computational Intelligence), pp. 1322–1328. IEEE (2008)Google Scholar
  11. 11.
    Han, H., Wang, W.-Y., Mao, B.-H.: Borderline-SMOTE: a new over-sampling method in imbalanced datasets learning. In: International Conference on Intelligent Computing, pp. 878–887. Springer, Heidelberg (2005)Google Scholar
  12. 12.
    Jo, T., Japkowicz, N.: Class imbalances versus small disjuncts. ACM Sigkdd Explor. Newsl. 6(1), 40–49 (2004)CrossRefGoogle Scholar
  13. 13.
    Haykin, S., Network, N.: A comprehensive foundation. Neural Netw. 2(2004), 41 (2004)Google Scholar
  14. 14.
    Mani, I., Zhang, I.: kNN approach to unbalanced data distributions: a case study involving information extraction. In: Proceedings of Workshop on Learning from Imbalanced Datasets, vol. 126 (2003)Google Scholar
  15. 15.
    Chawla, N.V., Lazarevic, A., Hall, L.O., Bowyer, K.W.: SMOTEBoost: improving prediction of the minority class in boosting. In: European Conference on Principles of Data Mining and Knowledge Discovery, pp. 107–119. Springer, Heidelberg (2003)Google Scholar
  16. 16.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefGoogle Scholar
  17. 17.
    Kubat, M., Matwin, S.: Addressing the curse of imbalanced training sets: one-sided selection. In: ICML, vol. 97, pp. 179–186 (1997)Google Scholar
  18. 18.
    Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: 1995 Proceedings of the Sixth International Symposium on Micro Machine and Human Science, MHS 1995, pp. 39–43. IEEE (1995)Google Scholar
  19. 19.
    Tomek, I.: Two modifications of CNN. IEEE Trans. Syst. Man Cybern. 6, 769–772 (1976)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Chawla, N.V., Japkowicz, N., Kotcz, A.: Special issue on learning from imbalanced datasets. ACM Sigkdd Explor. Newsl. 6(1), 1–6 (2004)CrossRefGoogle Scholar
  21. 21.
    Zhou, Z.-H., Liu, X.-Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2006)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Seiffert, C., Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: RUSBoost: improving classification performance when training data is skewed. In: 19th International Conference on Pattern Recognition, 2008, ICPR 2008. pp. 1–4. IEEE (2008)Google Scholar
  23. 23.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007)Google Scholar
  24. 24.
    Barua, S., Islam, M.M., Yao, X., Murase, K.: MWMOTE–majority weighted minority oversampling technique for imbalanced dataset learning. IEEE Trans. Knowl. Data Eng. 26(2), 405–425 (2014)CrossRefGoogle Scholar
  25. 25.
    Susan, S., Ranjan, R., Taluja, U., Rai, S., Agarwal, P.: Neural net optimization by weight-entropy monitoring. In: Computational Intelligence: Theories, Applications and Future Directions, vol. II, pp. 201–213. Springer, Singapore (2019)Google Scholar
  26. 26.
    Susan, S., Jain, A., Sharma, A., Verma, S., Jain, S.: Fuzzy match index for scale-invariant feature transform (SIFT) features with application to face recognition with weak supervision. IET Image Proc. 9(11), 951–958 (2015)CrossRefGoogle Scholar
  27. 27.
    Susan, S., Sharawat, P., Singh, S., Meena, R., Verma, A., Kumar, M.: Fuzzy C-means with non-extensive entropy regularization. In: IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), 2015, pp. 1–5. IEEE (2015)Google Scholar
  28. 28.
    Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on Faces in Real-Life Images: Detection, Alignment, and Recognition (2008)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Information TechnologyDelhi Technological UniversityDelhiIndia

Personalised recommendations