Abstract
In recent years, the classifiers based on convolutional neural network (CNN) and word embedding achieved good performances in sentiment classification tasks. However, the CNN-based model simply uses a fully connected layer for classification and it cannot perform a non-linear classification efficiently compared to the support vector machine (SVM) classifier. Target to this problem, in this paper, we combine CNN and SVM for sentiment classification. Firstly, continuous bag of word (CBOW) model is applied to construct word embedding. CNN is then utilized to learn feature vector representation corresponding to each sentence. The learned vector representations are fed to a SVM classifier as features for sentiment classification. Evaluations on the NLPCC2014 Sentiment Classification with Deep Learning Technology Task datasets (in short, NLPCC-SCDL) show that our model outperforms the top system in the NLPCC 2014 evaluation, on both English and Chinese sides.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Baccianella, S., Esuli, A., Sebastiani, F.: Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of the International Conference on Language Resources and Evaluation, LREC 2010, 17–23 May 2010, Valletta, Malta (2010)
Bengio, Y., Ducharme, R., Vincent, P.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning (ICML), pp. 160–167. ACM (2008)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Graves, A., Mohamed, A.-R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649. IEEE (2013)
Huang, M., Ye, B., Wang, Y., Chen, H., Cheng, J., Zhu, X.: New word detection for sentiment analysis. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 531–541. Association for Computational Linguistics, Baltimore,Maryland, June 2014
Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398. Springer, Heidelberg (1998)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, 22–27 June 2014, Baltimore, MD, USA, vol. 1, pp. 655–665 (2014). Long Papers
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 655–665. Association for Computational Linguistics, Baltimore, June 2014
Kim, S.M., Hovy, E.: Automatic detection of opinion bearing words and sentences. In: Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP), pp. 61–66 (2005)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751, October 2014
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held 3–6 December 2012, Lake Tahoe, Nevada, USA, pp. 1106–1114 (2012)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held 5–8 December 2013, Lake Tahoe, Nevada, USA, pp. 3111–3119 (2013)
Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL), 21–26 July 2004, Barcelona, Spain, pp. 271–278 (2004)
Pang, B., Lee, L.: Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL), pp. 115–124. Association for Computational Linguistics (2005)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86 (2002)
Salvetti, F., Lewis, S., Reichenbach, C.: Automatic opinion polarity classification of movie. Colorado Res. Linguist. 17, 2 (2004)
Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G.: Learning semantic representations using convolutional neural networks for web search. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 373–374. International World Wide Web Conferences Steering Committee (2014)
Tang, D., Wei, F., Qin, B., Zhou, M., Liu, T.: Building large-scale twitter-specific sentiment lexicon: a representation learning approach. In: Proceedings of the 25th International Conference on Computational Linguistics (COLING), pp. 172–182 (2014)
Tekiroğlu, S.S., Özbal, G., Strapparava, C.: A computational approach to generate a sensorial lexicon. In: Proceedings of the 25th International Conference on Computational Linguistics (COLING), p. 114 (2014)
Tong, R.M.: An operational system for detecting and tracking opinions in on-line discussion. In: Working Notes of the ACM SIGIR 2001 Workshop on Operational Text Classification, vol. 1, p. 6 (2001)
Wang, Y., Li, Z., Liu, J., He, Z., Huang, Y., Li, D.: Word vector modeling for sentiment analysis of product reviews. In: Zong, C., Nie, J.-Y., Zhao, D., Feng, Y. (eds.) NLPCC 2014. CCIS, vol. 496, pp. 168–180. Springer, Heidelberg (2014)
Xu, L., Lin, H., Zhao, J.: Construction and analysis of emotional corpus. J. Chin. Inf. Process. 22(1), 116–122 (2008)
Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 42–49. ACM (1999)
Yih, W.-T., He, X., Meek, C.: Semantic parsing for single-relation question answering. In: Proceedings of the 53rd Annual Meeting on Association for Computational Linguistics (ACL) (2014)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61370165, 61203378), National 863 Program of China 2015AA015405, Natural Science Foundation of Guangdong Province S2013010014475, Shenzhen Development and Reform Commission Grant No.[2014]1507, and Shenzhen Peacock Plan Research Grant KQCX20140521144507925
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer Science+Business Media Singapore
About this paper
Cite this paper
Cao, Y., Xu, R., Chen, T. (2015). Combining Convolutional Neural Network and Support Vector Machine for Sentiment Classification. In: Zhang, X., Sun, M., Wang, Z., Huang, X. (eds) Social Media Processing. SMP 2015. Communications in Computer and Information Science, vol 568. Springer, Singapore. https://doi.org/10.1007/978-981-10-0080-5_13
Download citation
DOI: https://doi.org/10.1007/978-981-10-0080-5_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-0079-9
Online ISBN: 978-981-10-0080-5
eBook Packages: Computer ScienceComputer Science (R0)