Advertisement

Enhancing the Performance of SVM on Skewed Data Sets by Exciting Support Vectors

  • José Hernández Santiago
  • Jair Cervantes
  • Asdrúbal López-Chau
  • Farid García Lamont
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7637)

Abstract

In pattern recognition and data mining a data set is named skewed or imbalanced if it contains a large number of objects of certain type and a very small number of objects of the opposite type. The imbalance in data sets represents a challenging problem for most classification methods, this is because the generalization power achieved for classic classifiers is not good for skewed data sets. Many real data sets are imbalanced, so the development of new methods to face this problem is necessary. The SVM classifier has an exceptional performance for data sets that are not skewed, however for imbalanced sets the optimal separating hyper plane is not enough to achieve acceptable results. In this paper a novel method that improves the performance of SVM for skewed data sets is presented. The proposed method works by exciting the support vectors and displacing the separating hyper plane towards majority class. According to the results obtained in experiments with different skewed data sets, the method enhances not only the accuracy but also the sensitivity of SVM classifier on this kind of data sets.

Keywords

SVM skewed data sets imbalanced data sets SMOTE 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Akbani, R., Kwek, S., Japkowicz, N.: Applying Support Vector Machines to Imbalanced Datasets. In: Boulicaut, J.F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 39–50. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Arbach, L., Reinhardt, J., Bennett, D., Fallouh, G.: Mammographic Masses Classification – Comparison Between Backpropagation Neural Network (BNN), K Nearest Neighbors (KNN), and Human Readers. In: IEEE Canadian Conference on Electrical and Computer Engineering (CCECE 2003), vol. 3, pp. 1441–1444. IEEE, Washington, DC (2003)Google Scholar
  3. 3.
    Cervantes, J., Li, X., Yu, W.: Splice Site Detection in DNA Sequences Using a Fast Classification Algorithm. In: IEEE International Conference on Systems, Man and Cybernetics (SMC 2009), pp. 2683–2688. IEEE, Washington, DC (2009)CrossRefGoogle Scholar
  4. 4.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE – Synthetic Minority Over-Sampling Technique. Journal of Artificial Intelligence Research 16(1), 321–357 (2002)MATHGoogle Scholar
  5. 5.
    Dror, G., Sorek, R., Shamir, R.: Accurate Identification of Alternatively Spliced Exons Using Support Vector Machine. Bioinformatics 21(7), 897–901 (2005)CrossRefGoogle Scholar
  6. 6.
    Fawcett, T.: An Introduction to ROC Analysis. Pattern Recognition Letters 27(8), 861–874 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Guo, H.: Learning from Imbalanced Data Sets with Boosting and Data Generation – The DataBoost–IM Approach. ACM SIGKDD Explorations Newsletter 6(1), 30–39 (2004)CrossRefGoogle Scholar
  8. 8.
    Hu, S., Liang, Y., Ma, L., He, Y.: MSMOTE – Improving Classification Performance When Training Data is Imbalanced. In: 2nd International Workshop on Computer Science and Engineering (WCSE 2009), vol. 2, pp. 13–17. IEEE, Washington, DC (2009)CrossRefGoogle Scholar
  9. 9.
    Koknar-Tezel, S., Latecki, L.: Improving SVM Classification on Imbalanced Data Sets in Distance Spaces. In: 9th IEEE International Conference on Data Mining (ICDM 2009), pp. 259–267. IEEE, Washington, DC (2009)Google Scholar
  10. 10.
    Kononenko, I.: Machine Learning for Medical Diagnosis – History, State of the Art and Perspective. Artificial Intelligence in Medicine 23(1), 89–109 (2001)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Nguyen, H.M., Cooper, E.W., Kamei, K.: Borderline Over-Sampling for Imbalanced Data Classification. International Journal of Knowledge Engineering and Soft Data Paradigms 3(1), 4–21 (2011)CrossRefGoogle Scholar
  12. 12.
    Provost, F., Fawcett, T.: Robust Classification for Imprecise Environments. Machine Learning 42(3), 203–231 (2001)MATHCrossRefGoogle Scholar
  13. 13.
    Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys 34(1), 1–47 (2002)CrossRefGoogle Scholar
  14. 14.
    Tan, S.: Neighbor-Weighted K-Nearest Neighbor for Unbalanced Text Corpus. Expert Systems with Applications 28(4), 667–671 (2005)CrossRefGoogle Scholar
  15. 15.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag New York, Inc., New York (1995)MATHGoogle Scholar
  16. 16.
    Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)Google Scholar
  17. 17.
    Veropoulos, K., Campbell, C., Cristianini, N.: Controlling the Sensitivity of Support Vector Machines. In: 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 55–60 (1999)Google Scholar
  18. 18.
    Zeng, Z.Q., Gao, J.: Improving SVM Classification with Imbalance Data Set. In: Leung, C.S., Lee, M., Chan, J.H. (eds.) ICONIP 2009, Part I. LNCS, vol. 5863, pp. 389–398. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Zhang, Y., Chu, C.H., Chen, Y., Zha, H., Ji, X.: Splice Site Prediction Using Support Vector Machines with a Bayes Kernel. Expert Systems with Applications 30(1), 73–81 (2006)CrossRefGoogle Scholar
  20. 20.
    Zou, S., Huang, Y., Wang, Y., Wang, J., Zhou, C.: SVM Learning from Imbalanced Data by GA Sampling for Protein Domain Prediction. In: 9th International Conference for Young Computer Scientists (ICYCS 2008), pp. 982–987. IEEE, Washington, DC (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • José Hernández Santiago
    • 1
  • Jair Cervantes
    • 1
  • Asdrúbal López-Chau
    • 2
  • Farid García Lamont
    • 1
  1. 1.Posgrado e InvestigaciónUAEM-TexcocoMéxico
  2. 2.Centro Universitario UAEM ZumpangoUAEMValle Hermoso ZumpangoMéxico

Personalised recommendations