Abstract
As a generalized form of multi-class classification, multi-label classification allows each sample to be associated with multiple labels. This task becomes challenging when the number of labels bulks up, which demands a high efficiency. Many approaches have been proposed to address this problem, among which one of the main ideas is to select a subset of labels which can approximately span the original label space, and training is performed only on the selected set of labels. However, these proposed sampling algorithms either require nondeterministic number of sampling trials or are time consuming. In this paper, we propose two label selection methods for multi-label classification (i) clustering based sampling (CBS) that uses deterministic number of sampling trials; and (ii) frequency based sampling (FBS) utilizing only label frequency statistics which makes it more efficient. Moreover, neither of these two algorithms needs to perform singular value decomposition (SVD) on label matrix which is used in previously mentioned approaches. Experiments are performed on several real world multi-label data sets with the number of labels ranging from hundreds to thousands, and it is shown that the proposed approaches achieve the state-of-the-art performance among label space reduction based multi-label classification algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. Journal of Machine Learning Research 1, 113–141 (2001)
Beygelzimer, A., Langford, J., Lifshits, Y., Sorkin, G., Strehl, A.: Conditional probability tree estimation analysis and algorithm. In: Conference in Uncertainty in Artificial Intelligence (UAI) (2009)
Boutsidis, C., Mahoney, W.M., Drineas, P.: An improved approximation algorithm for the column subset selection problem. In: Proceedings of the 20th ACMSIAM Symposium on Discrete Algorithms, pp. 968–977 (2009)
Bengio, S., Weston, J. Grangier, D.: Label embedding trees for large multi-class tasks. Advances in Neural Information Processing Systems 23 (2010)
Balasubramanian, K., Lebanon, G.: The landmark selection method for multiple output prediction. In: Proceedings of the 29th International Conference on Machine Learning, pp. 983–990 (2012)
Bi, W., Kwok, J.T.: Mandatory leaf node prediction in hierarchical multilabel classification. Proceedings of the Neural Information Processing Systems (NIPS) (2012)
Bi, W., Kwok, J.T.: Efficient multi-label classification with many labels. In: Proceedings of the Thirtieth International Conference on Machine Learning (ICML) (2013)
Chen, Y.-N., Lin, H.-T.: Feature-aware label space dimension reduction for multi-label classification. Advances in Neural Information Processing Systems 25, 1538–1546 (2012)
Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)
Drineas, P., Mahoney, M.W., Muthukrishnan, S.M.: Subspace Sampling and Relative-Error Matrix Approximation: Column-Based Methods. In: D\’ıaz, J., Jansen, K., Rolim, J.D.P., Zwick, U. (eds.) APPROX 2006 and RANDOM 2006. LNCS, vol. 4110, pp. 316–326. Springer, Heidelberg (2006)
Dekel, O., Shamir, O.: Multiclass-Multilabel Learning when the Label Set Grows with the Number of Examples. In: Artificial Intelligence and Statistics (AISTATS) (2010)
Deng, J., Dong, W., Socher, R., Li, Li-Jia, Li, K., Li, Fei-Fei: Imagenet: A large-scale hierarchical image database. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009)
Hsu, D., Kakade, S.M., Langford, J., Zhang, T.: Multilabel prediction via compressed sensing. Advances in Neural Information Processing Systems 22, 772–780 (2009)
Lux, M., S.A., Chatzichristofis: Lire: lucene image retrieval: an extensible java cbir library. MM (2008)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and Efficient Multilabel Classification in Domains with Large Number of Labels. In: Proceedings of the ECML/PKDD 2008 Workshop on Mining Multidimensional Data (2008)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. Data Mining and Knowledge Discovery Handbook, pp. 667–685 (2010)
Tai, F., Lin, H.: Multilabel Classification with Principal Label Space Transformation. Neural Computation 24(9), 2508–2542 (2012)
Turnbull, D., Barrington, L., Torres, D., Lanckriet, G.: Semantic annotation and retrieval of music and sound effects. IEEE Transactions on Audio, Speech and Language Processing 16(2), 467–476 (2008)
Vens, C., Struyf, J., Schietgat, L., Dz̆eroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Machine Learning 73, 185–214 (2008)
Zhang, Y., Schneider, J.: Multi-label output codes using canonical correlation analysis. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, pp. 873–882 (2011)
Nguyen, C.-T., Zhan, D.-C., Zhou, Z.-H.: Multi-modal image annotation with multi-instance multi-label LDA. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI 2013), pp. 1558–1564 (2013)
Zhang, M.-L., Zhou, Z.-H.: A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26(8), 1819–1837 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Sun, C., Zhou, C., Jin, B., Lau, F.C.M. (2015). Efficient Methods for Multi-label Classification. In: Cao, T., Lim, EP., Zhou, ZH., Ho, TB., Cheung, D., Motoda, H. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2015. Lecture Notes in Computer Science(), vol 9077. Springer, Cham. https://doi.org/10.1007/978-3-319-18038-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-18038-0_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18037-3
Online ISBN: 978-3-319-18038-0
eBook Packages: Computer ScienceComputer Science (R0)