Abstract
Text classification (also called text categorization) is the task of automatically assigning a piece of text to one or more predefined classes or categories. According to the definition of the categories, text classification tasks include topic classification, genre classification, sentiment classification, spam detection, etc.
The early text classification methods were mainly based on rules, and thus they required experts to design elaborate rules for classification. The establishment and maintenance of rules were time-consuming and labor-intensive. Since the 1990s, with the rise of statistical machine learning, classification algorithms based on supervised machine learning have achieved great success in text classification. Commonly used text classification algorithms include naïve Bayes (NB), maximum entropy (ME) models, support vector machine (SVM), and so on. In recent years, deep learning methods represented by deep neural networks have made great progress in text classification, and this has gradually become the mainstream technology addressed in current research.
In this chapter, we will first introduce representative text classification methods based on traditional machine learning, then we will introduce the recently developed deep learning methods, and finally, we will end with the evaluation methods for text classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., et al. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of EMNLP.
Forman, G. (2003). An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research, 3, 1289–1305.
Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Proceedings of ICML (Vol. 96, pp. 148–156).
Gers, F. A., Schraudolph, N. N., & Schmidhuber, J. (2002). Learning precise timing with LSTM recurrent networks. Journal of Machine Learning Research, 3, 115–143.
Graves, A., Jaitly, N., & Mohamed, A.-R. (2013). Hybrid speech recognition with deep bidirectional LSTM. In 2013 IEEE workshop on automatic speech recognition and understanding (pp. 273–278). New York: IEEE.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of ACL (pp. 655–665).
Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of EMNLP (pp. 1746–1751).
Larkey, L. S., & Croft, W. B. (1996). Combining classifiers in text categorization. In Proceedings of SIGIR (pp. 289–297).
Li, H. (2019). Statistical machine learning (2nd ed.). Beijing: Tsinghua University Press (in Chinese).
Li, S., Xia, R., Zong, C., & Huang, C.-R. (2009a). A framework of feature selection methods for text categorization. In Proceedings of ACL-IJCNLP (pp. 692–700).
McCallum, A., & Nigam, K. (1998). A comparison of event models for naive bayes text classification. In AAAI-98 Workshop on Learning for Text Categorization (Vol. 752, pp. 41–48). Citeseer.
Ng, A. Y., & Jordan, M. I. (2002). On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In Advances in Neural Information Processing Systems (pp. 841–848).
Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2–3), 103–134.
Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of EMNLP (pp. 79–86). Stroudsburg: Association for Computational Linguistics.
Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines (pp. 212–223).
Schapire, R., & Singer, Y. (2000). Boostexter: A boosting-based system for text categorization. Machine Learning, 39(2), 135–168.
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673–2681.
Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys (CSUR), 34(1), 1–47.
Tang, D., Qin, B., Feng, X., & Liu, T. (2015a). Effective LSTMs for target-dependent sentiment classification. Proceedings of COLING (pp. 3298–3307).
Xia, R., Zong, C., & Li, S. (2011). Ensemble of feature sets and classification algorithms for sentiment classification. Information Sciences, 181(6), 1138–1152.
Yang, Y., & Liu, X. (1999). A re-examination of text categorization methods. In Proceedings of SIGIR (pp. 42–49).
Yang, Y., & Pedersen, J. O. (1997). A comparative study on feature selection in text categorization. In Proceedings of ICML, Nashville, TN, USA (Vol. 97, pp. 35).
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of NAACL (pp. 1480–1489).
Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems (pp. 649–657).
Zong, C. (2013). Statistical natural language processing (2nd ed.). Beijing: Tsinghua University Press (in Chinese).
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 Tsinghua University Press
About this chapter
Cite this chapter
Zong, C., Xia, R., Zhang, J. (2021). Text Classification. In: Text Data Mining. Springer, Singapore. https://doi.org/10.1007/978-981-16-0100-2_5
Download citation
DOI: https://doi.org/10.1007/978-981-16-0100-2_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0099-9
Online ISBN: 978-981-16-0100-2
eBook Packages: Computer ScienceComputer Science (R0)