Abstract
One of the most common approaches for handling the multi-class classification problem is to divise the original data set into binary subclasses and to use a set of binary classifiers in order to solve the binarization problem. A new method for solving multi-class classification problems is proposed, by incorporating random resampling techniques in the one-versus-all strategy. Specifically, the division used by the proposed method is based on the one-versus-all binarization technique using random resampling for handling the class-imbalance problem arising due to the one-versus-all binarization. The method has been tested extensively on several multiclass classification problems using Support Vector Machines with four different kernels. Experimental results show that the proposed method exhibits a better performance compared to the simple one-versus-all.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2000)
Cheong, S., Oh, S.H., Lee, S.Y.: Support vector machines with binary tree architecture for multi-class classification. Neural Inf. Process. Lett. Rev. 2(3), 47–51 (2004)
Chmielnicki, W., Stąpor, K.: Using the one-versus-rest strategy with samples balancing to improve pairwise coupling classification. Int. J. Appl. Math. Comput. Sci. 26(1), 191–201 (2016)
Christianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Mach. Learn. 47(2), 201–233 (2002)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Dogan, U., Glasmachers, T., Igel, C.: A unified view on multi-class support vector classification. J. Mach. Learn. Res. 17(45), 1–32 (2016)
Duan, K.-B., Keerthi, S.S.: Which is the best multiclass SVM method? an empirical study. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds.) MCS 2005. LNCS, vol. 3541, pp. 278–285. Springer, Heidelberg (2005). doi:10.1007/11494683_28
Fei, B., Liu, J.: Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans. Netw. 17(3), 696–704 (2006)
Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15, 3133–3181 (2014). bibtex: fernandez-delgado_we_2014. http://jmlr.org/papers/v15/delgado14a.html
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn. 44(8), 1761–1776 (2011)
García-Pedrajas, N., Ortiz-Boyer, D.: An empirical study of binary classifier fusion methods for multiclass classification. Inf. Fusion 12(2), 111–130 (2011)
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. Ann. Stat. 26(2), 451–471 (1998). http://dx.doi.org/10.1214/aos/1028144844
Hodges, J.L., Lehmann, E.L.: Rank methods for combination of independent experiments in analysis of variance. In: Rojo, J. (ed.) Selected Works of E.L. Lehmann, pp. 403–418. Springer, Heidelberg (2011). doi:10.1007/978-1-4614-1412-4_35
Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Netw. 13(2), 415–425 (2002)
Jian, L., Gao, C.: Binary coding SVMs for the multiclass problem of blast furnace system. IEEE Trans. Ind. Electro. 60(9), 3846–3856 (2013)
Kotsiantis, S.B.: Bagging and boosting variants for handling classifications problems: a survey. Knowl. Eng. Rev. 29(01), 78–100 (2014)
Lemaître, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18(17), 1–5 (2017). http://jmlr.org/papers/v18/16-365.html
Lichman, M.: UCI Machine Learning Repository (2013). http://archive.ics.uci.edu/ml
Liu, M., Zhang, D., Chen, S., Xue, H.: Joint binary classifier learning for ecoc-based multi-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2335–2341 (2016)
Lorena, A.C., De Carvalho, A.C., Gama, J.M.: A review on the combination of binary classifiers in multiclass problems. Artif. Intell. Rev. 30(1), 19–37 (2008)
Madjarov, G., Kocev, D., Gjorgjevikj, D., Džeroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45(9), 3084–3104 (2012)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Rocha, A., Goldenstein, S.K.: Multiclass from binary: expanding one-versus-all, one-versus-one and ecoc-based approaches. IEEE Trans. Neural Netw. Learn. Syst. 25(2), 289–302 (2014)
Santhanam, V., Morariu, V.I., Harwood, D., Davis, L.S.: A non-parametric approach to extending generic binary classifiers for multi-classification. Pattern Recogn. 58, 149–158 (2016)
Kotsiantis, S., Kanellopoulos, D., Pintelas, P.: Handling imbalanced datasets: a review. Int. Trans. Comput. Sci. Eng. 30, 25–36 (2006)
Tax, D.M., Duin, R.P.: Using two-class classifiers for multiclass classification. In: Proceedings of the 16th IEEE International Conference on Pattern Recognition, vol. 2, pp. 124–127. IEEE (2002)
Windeatt, T., Ghaderi, R.: Coding and decoding strategies for multi-class learning problems. Inf. Fusion 4(1), 11–21 (2003)
Wu, T.F., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5, 975–1005 (2004)
Zadrozny, B., Elkan, C.: Reducing multiclass to binary by coupling probability estimates. In: Advances in Neural Information Processing Systems, vol. 2, pp. 1041–1048 (2002)
Acknowledgements
Stamatios-Aggelos N. Alexandropoulos gratefully acknowledges the support of his work by the Hellenic State Scholarships Foundation (IKY), co-financed by the European Union (European Social Fund–ESF) and Greek national funds, “Reinforcement of the Human Research Potential through Doctoral Research” of the Operational Program “Development of Human Capital, Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF 2014–2020).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Aridas, C.K., Alexandropoulos, SA.N., Kotsiantis, S.B., Vrahatis, M.N. (2017). Random Resampling in the One-Versus-All Strategy for Handling Multi-class Problems. In: Boracchi, G., Iliadis, L., Jayne, C., Likas, A. (eds) Engineering Applications of Neural Networks. EANN 2017. Communications in Computer and Information Science, vol 744. Springer, Cham. https://doi.org/10.1007/978-3-319-65172-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-65172-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65171-2
Online ISBN: 978-3-319-65172-9
eBook Packages: Computer ScienceComputer Science (R0)