Advertisement

A Novel ECOC Algorithm with Centroid Distance Based Soft Coding Scheme

  • Kaijie Feng
  • Kunhong Liu
  • Beizhan Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10955)

Abstract

In ECOC framework, the ternary coding strategy is widely deployed in coding process. It relabels classes with \( \left\{ { - 1, 0, 1} \right\} \), where −1/1 means to assign the corresponding classes to the negative/positive group, and label 0 leads to ignore the corresponding classes in the training process. However, the application of hard labels may lose some information about the tendency of class distributions. Instead, we propose a Centroid distance-based Soft coding scheme to indicate such tendency, named as CSECOC. In our algorithm, Sequential Forward Floating Selection (SFFS) is applied to search an optimal class assignment by maximizing the ratio of inter-group and intra-group distance. In this way, a hard coding matrix is generated initially. Then we propose a measure, named as coverage, to describe the probability of a sample in a class falling to a correct group. The coverage of a class in a group replace the corresponding hard element, so as to form a soft coding matrix. Compared with the hard ones, such soft elements can reflect the tendency of a class belonging to positive or negative group. Instead of classifiers, regressors are used as base learners in this algorithm. To the best of our knowledge, it is the first time that soft coding scheme has been proposed. The results on five UCI datasets show that compared with some state-of-art ECOC algorithms, our algorithm can produce comparable or better classification accuracy with small scale ensembles.

Keywords

Error correcting output codes Multiclass Coverage Soft codeword 

Notes

Acknowledgement

This work is supported by National Key Technology Research and Development Program of the Ministry of Science and Technology of China (2015BAH55F05); Natural Science Foundation of Fujian Province (No. 2016J01320 and 2015J05129), and National Natural Science Foundation of China (Grant No. 61502402 and 61772023).

References

  1. 1.
    Crammer, K., Gentile, C.: Multiclass classification with bandit feedback using adaptive regularization. Mach. Learn. 90(3), 347–383 (2013)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Xue, A., Wang, X., Song, Y., Lei, L.: Discriminant error correcting output codes based on spectral clustering. Pattern Anal. Appl. 20(3), 653–671 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Zhou, J.D., Zhou, H.J., Wang, X.D., Zhang, J.M., Jia, N.: Decoding design based on posterior probabilities in ternary error-correcting output codes. Pattern Recogn. 45(4), 1802–1818 (2012)CrossRefGoogle Scholar
  4. 4.
    Procaccia, A.D., Shah, N., Zick, Y.: Voting rules as error-correcting codes. Artif. Intell. 231(2), 1–16 (2016)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2(1), 263–286 (2012)MATHGoogle Scholar
  6. 6.
    Liu, K.H., Li, B., Zhang, J., Du, J.X.: Ensemble component selection for improving ICA based microarray data prediction models. Pattern Recogn. 42(7), 1274–1283 (2009)CrossRefGoogle Scholar
  7. 7.
    Escalera, S., Pujol, O., Radeva, P.: On the decoding process in ternary error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 120–134 (2009)CrossRefGoogle Scholar
  8. 8.
    Pujol, O., Radeva, P., Vitrià, J.: Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 1007–1012 (2006)CrossRefGoogle Scholar
  9. 9.
    Liu, K.H., Zeng, Z.H., Ng, V.T.Y.: A hierarchical ensemble of ECOC for cancer classification based on multi-class microarray data. Inf. Sci. 349, 102–118 (2016)CrossRefGoogle Scholar
  10. 10.
    Escalera, S., Pujol, O., Radeva, P.: Boosted landmarks of contextual descriptors and Forest-ECOC: a novel framework to detect and classify objects in cluttered scenes. Pattern Recogn. Lett. 28(10), 1759–1768 (2007)CrossRefGoogle Scholar
  11. 11.
    Escalera, S., Pujol, O., Radeva, P.: ECOC-ONE: a novel coding and decoding strategy. In: 18th International Conference on Pattern Recognition, ICPR 2006, pp. 578–581. Institute of Electrical and Electronics Engineers Inc, Hong Kong (2006)Google Scholar
  12. 12.
    Khowaja, S.A., Yahya, B.N., Lee, S.L.: Hierarchical classification method based on selective learning of slacked hierarchy for activity recognition systems. Expert Syst. Appl. 88(11), 165–177 (2017)CrossRefGoogle Scholar
  13. 13.
    Japkowicz, N., Barnabe-Lortie, V., Horvatic, S., Zhou, J.: Multi-class learning using data driven ECOC with deep search and re-balancing. In: IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015, pp. 1–10. Institute of Electrical and Electronics Engineers Inc., Pairs (2015)Google Scholar
  14. 14.
    Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature-selection. Pattern Recogn. Lett. 15(11), 1119–1125 (1994)CrossRefGoogle Scholar
  15. 15.
    Yang, Q., Qian, Z., Zheng, G., Wei, C., Xie, L., Zhu, Y., Li, Y.: The combination approach of SVM and ECOC for powerful identification and classification of transcription factor. BMC Bioinform. 9(1), 1–8 (2008)CrossRefGoogle Scholar
  16. 16.
    Tong, M., Liu, K.H., Xu, C., Ju, W.: An ensemble of SVM classifiers based on gene pairs. Comput. Biol. Med. 43(7), 729–737 (2013)CrossRefGoogle Scholar
  17. 17.
    Pedregosa, F., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12(10), 2825–2830 (2012)MathSciNetMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Software School of Xiamen UniversityXiamenChina

Personalised recommendations