Clustering-Based Hybrid Approach for Multiclass Classification Using SVM

  • Rahul Kumar Jain
  • Girish Kumar SinghEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1025)


To develop a decision support system, various soft computing techniques are used like evolutionary computation, machine learning, deep learning, etc. To develop effective decision support system, classification and clustering is used in various applications. SVM is a binary classifier technique as it is based on separating hyperplane. Over time to enhance the applicability of SVM in multiclass classification, some methods have been discovered, but results in multiclass classification through SVM are not impressive like binary classification. This paper presents a clustering-based multiclass SVM classification. In this approach, to obtain better accuracy in multiclass problem, k-means clustering is used to partition the datasets based on properties of conditional attributes and then the SVM multiclass classifiers are developed for each partition. To make decision about testing instance, nearest cluster in each instance is searched and then the corresponding trained SVM model is used to assign the class value of instance. The simulation results of proposed approach show superior accuracy rate for both one-versus-one and one-versus-all SVM methods in comparison to normal SVM-based multiclass classification.


Data mining Classification Clustering SVM 


  1. 1.
    Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: ACM Workshop on Computational Learning Theory, pp. 144–152 (1992)Google Scholar
  2. 2.
    Knerr, S., Personnaz L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Proceeding of the Neurocomputing: Algorithms, Architectures and Applications, pp. 41–50. Springer (1990)Google Scholar
  3. 3.
    Bottou, L., et al.: Comparison of classifier methods: a case study in handwriting digit recognition. In: Proceeding of the International Conference on Pattern Recognition, pp. 77–87 (1994)Google Scholar
  4. 4.
    Platt, J., Cristianini N., Shawe-Taylor J.: Large margin DAGs for multiclass classification. In: Proceeding of the Advances in Neural Information Processing Systems, pp. 547–553. MIT Press (2000)Google Scholar
  5. 5.
    Hsu, C., Lin, C.: A comparison of methods for multiclass support vector machines. In: Proceeding of the IEEE Transactions on Neural Networks, pp. 415–425 (2002)Google Scholar
  6. 6.
    Friedman, J.: Another approach to polychotomous classification. Stanford University, Department of Statistics, Technical Report (1996)Google Scholar
  7. 7.
    Weston, J., Watkins, C.: Multi-class support vector machines. In: Proceeding of the European Symposium on Artificial Neural Networks, pp. 219–224 (1999)Google Scholar
  8. 8.
    Bredensteiner, E., Bennett, K.: Multicategory classification by support vector machines. J. Comput. Optim. Appl. 53–79 (1999)Google Scholar
  9. 9.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. J. Comput. Learn. Theory 35–46 (2000)Google Scholar
  10. 10.
    Guermeur, Y.: Combining discriminant models with new multiclass SVMs. Pattern Anal. Appl. 168–179 (2002)Google Scholar
  11. 11.
    Hsu, C., Lin, C.: A simple decomposition method for support vector machines. In: Machine Learning, pp. 291–314. Kluwer Academic Publisher (2002)Google Scholar
  12. 12.
    Mayoraz, E., Alpaydin, E.: Support vector machines for multi-class classification. In: Proceeding of the International Work Conference on Artificial Neural Networks, pp. 833–842 (1999)Google Scholar
  13. 13.
    Abe, S.: Fuzzy LP-SVMs for multiclass problems. In: Proceeding of the European Symposium on Artificial Neural Networks, pp. 429–434. Bruges, Belgium (2004)Google Scholar
  14. 14.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  15. 15.
    Lingras, P., Butz, C.: Interval set representations of 1-v-r support vector machine multi-classifiers. In: Proceeding of the IEEE International Conference on Granular Computing (2005)Google Scholar
  16. 16.
    Santosa, B.: Multiclass classification with cross entropy-support vector machines. In: Proceeding of the Third Information Systems International Conference, pp. 345–352. Elsevier (2015)Google Scholar
  17. 17.
    Ho, S., Yang, S.: The cross-entropy method and its application to inverse problems. IEEE Trans. Magn. 3401–3404 (2010)Google Scholar
  18. 18.
    Wang, J., Wu, X., Zhang, C.: Support vector machines based on K-means clustering for real-time business intelligence systems. Int. J. Bus. Intell. Data Min. 54–64 (2005)Google Scholar
  19. 19.
    Yu, H., Yanga, J., Han, J.: Classifying large data sets using SVMs with hierarchical clusters. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 306–315 (2003)Google Scholar
  20. 20.
    Hartigan, J.: Direct clustering of a data matrix. J. Am. Stat. Assoc. 123–129 (1972)Google Scholar
  21. 21.
    Jain, A., Murty M., Flynn, P.: Data clustering: a review. ACM Comput. Surv. (CSUR) 264–323 (1999)Google Scholar
  22. 22.
    Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 129–137. (1982)Google Scholar
  23. 23.
  24. 24.
  25. 25.

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.B.T.I.R.T. College Sagar, RGPV University BhoaplBhopalIndia
  2. 2.Department of Computer Science and ApplicationsDr. Harisingh Gour University SagarSagarIndia

Personalised recommendations