Multiclass Classification Based on Multi-criteria Decision-making

  • Hossein Baloochian
  • Hamid Reza GhaffaryEmail author


Lots of real-world problems require multiclass classification. Since most general classification methods are originally introduced for binary problems (including two classes), they should be extended to multiclass problems. A solution proposed for multiclass problems is to decompose such problems to several binary ones and then combine the results obtained from smaller problems as a tree-based structure to obtain the final solution. In this study, a novel method which uses VlseKriterijumska optimizacija I Kompromisno Resenje multi-criteria decision-making was proposed to build the best directed binary tree with minimum error. The proposed method is independent of classifier; nevertheless, in the current experiments, the support vector machine was employed as the base classifier. The proposed method was tested on datasets and the results were compared with other methods. It can be seen that it improves precision of predictions significantly.


Multiclass classification Multi-criteria decision-making Hierarchical decomposition Decision-making criterion 



  1. Allwein, E., Schapire, R., & Singer, Y. (2001). Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1, 113–141.MathSciNetzbMATHGoogle Scholar
  2. de Amorim, R. C. (2016). A survey on feature weighting based K-means algorithms. Journal of Classification, 33(2), 210–242. Scholar
  3. Asuncion, A., & Newman, D. UCI machine learning repository. from Accessed 4 Apr 2015
  4. Cheong, S., Oh, S. H., & Lee, S. Y. (2004). Support vector machines with binary tree architecture for multi-class classification. Neural Information Processing Letters and Reviews, 2(3), 47–50.Google Scholar
  5. Crammer, K., & Singer, Y. (2002). On the learnability and design of output codes for multiclass problems. Machine Learning, 47, 201–233.CrossRefGoogle Scholar
  6. Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  7. Díez, J., Coz, J. D., Bahamonde, A., & Luaces, O. (2009). Soft margin trees. Paper presented at the machine learning and knowledge discovery in databases, Lecture Notes in Computer Science, Berlin, Heidelberg.Google Scholar
  8. Fei, B., & Liu, J. (2006). Binary tree of svm: A new fast multiclass training and classification algorithm. IEEE Transactions on Neural Networks, 17(3), 696–704.
  9. Frank, E., & Kramer, S. (2004). Ensembles of nested dichotomies for multi-class problems. Paper presented at the Proceedings of the 21st International Conference on Machine Learning.Google Scholar
  10. Frohlich, H., Chapelle, O., & Scholkopf, B. (2003). Feature selection for support vector machines by means of genetic algorithms. Paper presented at the Proceedings of the 15th IEEE International Conference on Tools with AI.Google Scholar
  11. Fürnkranz, J. (2002). Round robin classification. Journal of Machine Learning Research, 2, 721–747.MathSciNetzbMATHGoogle Scholar
  12. Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. Paper presented at the NIPS’97, Cambridge MA, USA.Google Scholar
  13. Ho, T. K., & Basu, M. (2002). Complexity measures of supervised classification problems. Paper presented at the IEEE Transactions on Pattern Analysis and Machine Intelligence.Google Scholar
  14. Hsu, C. W., Chang, C. C., & Lin, C. J. (2016). A practical guide to support vector classification. from
  15. Jirina, M., & Jirina, M. (2013). Utilization of singularity exponent in nearest neighbor based classifier. Journal of Classification, 30(1), 3–29. Scholar
  16. Kijsirikul, B., & Ussivakul, N. (2002). Multiclass support vector machines using adaptive directed acyclic graph. Paper presented at the International Joint Conference on Neural Networks (IJCNN ‘02).Google Scholar
  17. Kumar, S., & Gosh, J. (1999). GAMLS: A generalized framework for associative modular learning systems. Paper presented at the Proceedings of the Applications and Science of Computation Intelligence II.Google Scholar
  18. Kumar, S., Gosh, J., & Crawford, M. (2002). Hierarchical fusion of multiple classifiers for hyperspectral data analysis. Pattern Analysis and Applications, 5, 210–220.MathSciNetCrossRefGoogle Scholar
  19. Lee, A., & Willcox, B. (2014). Minkowski generalizations of Ward’s method in hierarchical clustering. Journal of Classification, 31(2), 194–218. Scholar
  20. Lee, Y., Lin, Y., & Wahba, G. (2004). Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association, 99, 67–82.MathSciNetCrossRefGoogle Scholar
  21. Lei, H., & Govindaraju, V. (2005). Half-against-half multi-class support vector machines. Paper presented at the Proceedings of the Sixth International Workshop on Multiple Classifier Systems.Google Scholar
  22. Lorena, A. C., & Carvalho, A. C. P. L. F. D. (2008a). Hierarchical decomposition of multiclass problems. International Journal on Neural and Mass-Parallel Computing and Information System, 5, 407–425.Google Scholar
  23. Lorena, A. C., & Carvalho, A. C. P. L. F. D. (2008b). Tree decomposition of multiclass problems. Paper presented at the Proceedings of the Brazilian Symposium on Neural Networks (SBRN).Google Scholar
  24. Lorena, A., Carvalho, A., & Gama, J. (2008). A review on the combination of binary classifiers in multiclass problems. Artificial Intelligence Review, 30, 19–37.CrossRefGoogle Scholar
  25. Madzarov, G., Gjorgjevikj, D., & Chorbev, I. (2009). A multi-class svm classifier utilizing binary decision tree. Informatica (Slovenia), 33, 225–233.MathSciNetzbMATHGoogle Scholar
  26. Mardani, A., Zavadskas, E. K., Govindan, K., Senin, A. A., & Jusoh, A. (2016). VIKOR technique: A systematic review of the state of the art literature on methodologies and applications. Sustainability, 8(1), 37. Scholar
  27. Mitchell, T. (1997). Machine learning. New York City: McGraw-Hill.zbMATHGoogle Scholar
  28. Montañés, E., Barranquero, J., Díez, J., & Coz, J. J. D. (2013). Enhancing directed binary trees for multi-class classification. Information Sciences, 223, 42–55.MathSciNetCrossRefGoogle Scholar
  29. Oong, T., & Isa, N. (2012). One-against-all ensemble for multiclass pattern classification. Applied Soft Computing, 12, 1303–1308.CrossRefGoogle Scholar
  30. Panagiotakis, C. (2015). Point clustering via voting maximization. Journal of Classification, 32(2), 212–240. Scholar
  31. Platt, J., Cristianini, N., & Shawe-Taylor, J. (2000). Large margin DAGs for multiclass classification. Paper presented at the NIPS.Google Scholar
  32. Proença, H., & Neves, J. C. (2017). Fusing vantage point trees and linear discriminants for fast feature classification. Journal of Classification, 34(1), 85–107. Scholar
  33. Schwenker, F. (2000). Hierarchical support vector machines for multi-class pattern recognition. Paper presented at the IEEE Proceedings of the Fourth International Conference on Knowledge-Based Intelligent Systems and Allied Technologies.Google Scholar
  34. Schwenker, F., & Palm, G. (2001). Tree-structured support vector machines for multi-class pattern recognition. Paper presented at the Proceedings of the International Workshop on Multiple Classifier Systems, Springer-Verlag.Google Scholar
  35. Souto, M. C. P. D., Lorena, A. C., & Costa, I. G. (2010). Complexity measures of supervised classifications tasks: a case study for cancer gene expression data. Paper presented at the IEEE Proceedings of the International Joint Conference on Neural Networks (IJCNN).Google Scholar
  36. Steinley, D., & Brusco, M. (2007). Initializing K-means batch clustering: A critical evaluation of several techniques. Journal of Classification, 24(1), 99–121.MathSciNetCrossRefGoogle Scholar
  37. Takahashi, F., & Abe, S. (2002). Decision-tree-based multiclass support vector machines. Paper presented at the Proceedings of the Ninth International Conference on Neural Information Processing.Google Scholar
  38. Tibshirani, R., & Hastie, T. (2007). Margin trees for high-dimensional classification. Journal of Machine Learning Research, 8, 637–652.zbMATHGoogle Scholar
  39. Vrbik, I., & McNicholas, P. D. (2015). Fractionally-supervised classification. Journal of Classification, 32(3), 359–381.MathSciNetCrossRefGoogle Scholar
  40. Vural, V., & Dy, J. G. (2004). A hierarchical method for multi-class support vector machines. Paper presented at the Proceedings of the 21st International Conference on Machine Learning.Google Scholar
  41. Wang, R., Kwong, S., Chen, D., & Cao, J. (2013). A vector-valued support vector machine model for multiclass problem. Information Sciences, 235, 174–194.MathSciNetCrossRefGoogle Scholar
  42. Warrens, M. J., & Pratiwi, B. C. (2016). Kappa coefficients for circular classifications. Journal of Classification, 33(3), 507–522.MathSciNetCrossRefGoogle Scholar
  43. Windeatt, T., & Ghaderi, R. (2003). Coding and decoding strategies for multi-class learning problems. Information Fusion, 4, 11–21.CrossRefGoogle Scholar
  44. Zhang, G., & Jin, W. (2006). Automatic construction algorithm for multi-class support vector machines with binary tree architecture. International Journal of Computer Science and Network Security, 6, 119–126.Google Scholar
  45. Zhou, J., Peng, H., & Suen, C. (2008). Data-driven decomposition for multi-class classification. Pattern Recognition, 41, 67–76.CrossRefGoogle Scholar

Copyright information

© The Classification Society 2019

Authors and Affiliations

  1. 1.Department of Computer Engineering, Ferdows BranchIslamic Azad UniversityFerdowsIran

Personalised recommendations