Abstract
Extreme multi-label text classification (XMTC), the problem of finding the most relevant label subset of each document from hundreds or even millions labels, has been a practical and important problem since the boom of big data. Significant progress has been made in recent years by the development of machine learning methods. However, although deep learning method has beaten traditional method in other related areas, it has no clear advantage in XMTC when we consider the performance of prediction. In order to improve the performance of deep learning method for Extreme multi-label text classification, we propose a novel feature extraction method to better explore the text space. Specifically, we build the model consisting of attention mechanism, convolutional neural network and recurrent neural network to extract multi-view features. Extensive experiments on four public available datasets show that our method achieves better performance than several strong baselines, including traditional methods and deep learning methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Available at http://manikvarma.org/downloads/XC/XMLRepository.html.
References
Allaouzi, I., Ahmed, M.B.: Deep neural networks and decision tree classifier for visual question answering in the medical domain. In: Working Notes of CLEF 2018 - Conference and Labs of the Evaluation Forum, Avignon, 10–14 September 2018 (2018)
Balasubramanian, K., Lebanon, G.: The landmark selection method for multiple output prediction. In: Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, 26 June–1 July 2012 (2012)
Cissé, M., Usunier, N., Artières, T., Gallinari, P.: Robust bloom filters for large multilabel classification tasks. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a Meeting Held 5–8 December 2013, Lake Tahoe, pp. 1851–1859 (2013)
Hsu, D.J., Kakade, S., Langford, J., Zhang, T.: Multi-label prediction via compressed sensing. In: Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a Meeting Held 7–10 December 2009, Vancouver, pp. 772–780 (2009)
Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification, pp. 427–431 (2016)
Kim, Y.: Convolutional neural networks for sentence classification. CoRR abs/1408.5882 (2014)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification (2015)
Liu, J., Chang, W., Wu, Y., Yang, Y.: Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, 7–11 August 2017, pp. 115–124 (2017)
Nam, J., Kim, J., Loza Mencía, E., Gurevych, I., Fürnkranz, J.: Large-scale multi-label text classification — revisiting neural networks. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part II. LNCS (LNAI), vol. 8725, pp. 437–452. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44851-9_28
Nonweiler, T.R.F.: SLEEC: a space station ambulance. Philos. Trans. Math. Phys. Eng. Sci. 357(1759), 2157–2176 (1999). http://www.jstor.org/stable/55137
Prabhu, Y., Varma, M.: FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning, August 2014
Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)
Yen, I.E.H., Huang, X., Ravikumar, P., Zhong, K., Dhillon, I.S.: PD-Sparse: a primal and dual sparse approach to extreme multiclass and multilabel classification. In: Balcan, M.F., Weinberger, K.Q. (eds.) ICML. JMLR Workshop and Conference Proceedings, vol. 48, pp. 3069–3077. JMLR.org (2016)
Yousefi-Azar, M., Hamey, L.: Text summarization using unsupervised deep learning. Expert Syst. Appl. 68, 93–105 (2017)
Yu, A.W., et al.: QANet: combining local convolution with global self-attention for reading comprehension. CoRR abs/1804.09541 (2018)
Zhang, C., et al.: Semantic sentence embeddings for paraphrasing and text summarization. CoRR abs/1809.10267 (2018)
Zhang, W., Wang, L., Yan, J., Wang, X., Zha, H.: Deep extreme multi-label learning. CoRR abs/1704.03718 (2017)
Zhang, Z., Wang, H., Liu, L., Li, J.: Multi-label relational classification via node and label correlation. Neurocomputing 292, 72–81 (2018)
Zou, Y., OuYang, J., Li, X.: Supervised topic models with weighted words: multi-label document classification. Front. IT EE 19(4), 513–523 (2018)
Acknowledgment
This work is supported by Information Science Academy of China Electronics Technology Group Corporation. The work was conducted during the first author’s internship in the Information Science Academy of China Electronics Technology Group Corporation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, S., Wang, L., Li, W., Zhang, K. (2019). Deep Learning Method with Attention for Extreme Multi-label Text Classification. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11672. Springer, Cham. https://doi.org/10.1007/978-3-030-29894-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-29894-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29893-7
Online ISBN: 978-3-030-29894-4
eBook Packages: Computer ScienceComputer Science (R0)