Deep Neural Network for Short-Text Sentiment Classification

  • Xiangsheng Li
  • Jianhui Pang
  • Biyun Mo
  • Yanghui RaoEmail author
  • Fu Lee Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9645)


As a concise medium to describe events, short text plays an important role to convey the opinions of users. The classification of user emotions based on short text has been a significant topic in social network analysis. Neural Network can obtain good classification performance with high generalization ability. However, conventional neural networks only use a simple back-propagation algorithm to estimate the parameters, which may introduce large instabilities when training deep neural networks by random initializations. In this paper, we apply a pre-training method to deep neural networks based on restricted Boltzmann machines, which aims to gain competitive and stable classification performance of user emotions over short text. Experimental evaluations using real-world datasets validate the effectiveness of our model on the short-text sentiment classification task.


Neural network Restricted Boltzmann machine Pre-training Short-text sentiment classification 



This research was supported by the National Natural Science Foundation of China (61502545, 61472453, U1401256, U1501252), the Fundamental Research Funds for the Central Universities, and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS11/E06/14).


  1. 1.
    Bengio, Y.: Learning deep architectures for ai. Found. Trends\(\textregistered \) Mach. Learn., vol. 2(1), pp. 1–127 (2009)Google Scholar
  2. 2.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Sci. 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Adv. Neural Inf. Process. Syst. (NIPS) 19, 153 (2007)Google Scholar
  4. 4.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 79–86 (2002)Google Scholar
  5. 5.
    Katz, P., Singleton, M., Wicentowski, R.: Swat-mp: the semeval- systems for task 5 and task 14. In: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval), pp. 308–313(2007)Google Scholar
  6. 6.
    Bao, S., Xu, S., Zhang, L., Yan, R., Su, Z., Han, D., Yu, Y.: Mining social emotions from affective text. IEEE Trans. Knowl. Data Eng. 24(9), 1658–1670 (2012)CrossRefGoogle Scholar
  7. 7.
    Rao, Y., Li, Q., Mao, X., Wenyin, L.: Sentiment topic models for social emotion mining. Inf. Sci. 266, 90–100 (2014)CrossRefGoogle Scholar
  8. 8.
    Rao, Y., Li, Q., Wenyin, L., Wu, Q., Quan, X.: Affective topic model for social emotion detection. Neural Netw. 58, 29–37 (2014)CrossRefGoogle Scholar
  9. 9.
    Sahami, M., Heilman, T.D.: A web-based kernel function for measuring the similarity of short text snippets. In: Proceedings of the 15th International Conference on World Wide Web (WWW), pp. 377–386 (2006)Google Scholar
  10. 10.
    Banerjee, S., Ramanathan, K., Gupta, A.: Clustering short texts using wikipedia. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 787–788 (2007)Google Scholar
  11. 11.
    Tesauro, G.: Practical issues in temporal difference learning. Mach. Learn. 8(3–4), 33–53 (1992)zbMATHGoogle Scholar
  12. 12.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Hinton, G.: A practical guide to training restricted boltzmann machines. Momentum 9(1), 926 (2010)Google Scholar
  15. 15.
    Strapparava, C., Mihalcea, R.: Semeval- task 14: Affective text. In: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval 2007), pp. 70–74 (2007)Google Scholar
  16. 16.
    Scherer, K.R., Wallbott, H.G.: Evidence for universality and cultural variation of differential emotion response patterning. J. Pers. Soc. Psychol. 66(2), 310 (1994)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Xiangsheng Li
    • 1
  • Jianhui Pang
    • 1
  • Biyun Mo
    • 1
  • Yanghui Rao
    • 1
    Email author
  • Fu Lee Wang
    • 2
  1. 1.Sun Yat-sen UniversityGuangzhouChina
  2. 2.Caritas Institute of Higher EducationNew TerritoriesHong Kong

Personalised recommendations