Deep Neural Network for Short-Text Sentiment Classification
As a concise medium to describe events, short text plays an important role to convey the opinions of users. The classification of user emotions based on short text has been a significant topic in social network analysis. Neural Network can obtain good classification performance with high generalization ability. However, conventional neural networks only use a simple back-propagation algorithm to estimate the parameters, which may introduce large instabilities when training deep neural networks by random initializations. In this paper, we apply a pre-training method to deep neural networks based on restricted Boltzmann machines, which aims to gain competitive and stable classification performance of user emotions over short text. Experimental evaluations using real-world datasets validate the effectiveness of our model on the short-text sentiment classification task.
KeywordsNeural network Restricted Boltzmann machine Pre-training Short-text sentiment classification
This research was supported by the National Natural Science Foundation of China (61502545, 61472453, U1401256, U1501252), the Fundamental Research Funds for the Central Universities, and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS11/E06/14).
- 1.Bengio, Y.: Learning deep architectures for ai. Found. Trends\(\textregistered \) Mach. Learn., vol. 2(1), pp. 1–127 (2009)Google Scholar
- 3.Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Adv. Neural Inf. Process. Syst. (NIPS) 19, 153 (2007)Google Scholar
- 4.Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 79–86 (2002)Google Scholar
- 5.Katz, P., Singleton, M., Wicentowski, R.: Swat-mp: the semeval- systems for task 5 and task 14. In: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval), pp. 308–313(2007)Google Scholar
- 9.Sahami, M., Heilman, T.D.: A web-based kernel function for measuring the similarity of short text snippets. In: Proceedings of the 15th International Conference on World Wide Web (WWW), pp. 377–386 (2006)Google Scholar
- 10.Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using wikipedia. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 787–788 (2007)Google Scholar
- 14.Hinton, G.: A practical guide to training restricted boltzmann machines. Momentum 9(1), 926 (2010)Google Scholar
- 15.Strapparava, C., Mihalcea, R.: Semeval- task 14: Affective text. In: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval 2007), pp. 70–74 (2007)Google Scholar