Abstract
Word embedding has been proven to be a useful model for various natural language processing tasks. Traditional word embedding methods merely take into account word distributions independently from any specific tasks. Hence, the resulting representations could be sub-optimal for a given task. In the context of sentiment analysis, there are various types of prior knowledge available, e.g., sentiment labels of documents from available datasets or polarity values of words from sentiment lexicons. We incorporate such prior sentiment information at both word level and document level in order to investigate the influence each word has on the sentiment label of both target word and context words. By evaluating the performance of sentiment analysis in each category, we find the best way of incorporating prior sentiment information. Experimental results on real-world datasets demonstrate that the word representations learnt by DLJT2 can significantly improve the sentiment analysis performance. We prove that incorporating prior sentiment knowledge into the embedding process has the potential to learn better representations for sentiment analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For example, we extract the synonym of word ‘like’ from page http://www.urbandictionary.com/define.php?term=like
For example, we extract the synonym of word ‘like’ from the page of http://dict.youdao.com/search?q=like
References
Bansal M, Gimpel K, Livescu K. Tailoring continuous word representations for dependency parsing. In: ACL (2). 2014. p. 809–815.
Bengio Y, Schwenk H, Senécal J S, Morin F, Gauvain JL. A neural probabilistic language model. J Mach Learn Res 2003;3(6):1137–1155.
Cambria E, Das D, Bandyopadhyay S, Feraco A. A practical guide to sentiment analysis. Switzerland: Springer, Cham; 2017.
Cambria E, Poria S, Bajpai R, Björn S. SenticNet 4: A Semantic resource for sentiment analysis based on conceptual primitives. In: COLING; 2016. p. 2666–2677.
Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E. 2017. Bayesian network based extreme learning machine for subjectivity detection. J Franklin Inst. doi:1016/j.jfranklin.2017.06.007
Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In: International Conference, Helsinki, Finland, June; 2008. p. 160– 167.
Duchi J, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 2011;12(Jul):2121–2159.
Harris ZS. Distributional structure. Synthese Language Library 1954;10(2-3):146–162.
Hogenboom A, Bal D, Frasincar F, Bal M, de Jong F, Kaymak U. Exploiting emoticons in sentiment analysis Inproceedings of the 28th Annual ACM Symposium on Applied Computing; 2013. p. 703–710. ACM.
Hu X, Tang J, Gao H, Liu H. Unsupervised sentiment analysis with emotional signals. In: Proceedings of the 22nd international conference on World Wide Web; 2013. p. 607–618, ACM.
Huang EH, Socher R, Manning CD, Ng AY. Improving word representations via global context and multiple word prototypes. In: Meeting of the Association for Computational Linguistics: Long Papers; 2012. p. 873–882.
Kim Y. Convolutional neural networks for sentence classification. In: EMNLP. 2014.
Lin C -C, Ammar W, Dyer C, Levin LS. Unsupervised POS Induction with word embeddings. In: NAACL HLT 2015, The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2015. p. 1311–1316.
Liu Y, Liu Z, Chua TS, Sun M. 2015. Topical word embeddings. In: Twenty-Ninth AAAI Conference on Artificial Intelligence.
Ma Y, Cambria E, Gao S. Label embedding for zero-shot fine-grained named entity typing. In COLING; 2016. p. 171–180, Osaka.
Majumder N, Poria S, Gelbukh A, Cambria E. Deep learning-based document modeling for personality detection from text. IEEE Intelligent Systems 2017;32(2):74–79.
Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. CoRR, arXiv:1301.3781; 2013.
Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 2013;26:3111–3119.
Mnih A, Hinton G. Three new graphical models for statistical language modelling. In: International Conference on Machine Learning; 2007, p. 641–648.
Mohammad SM, Turney PD. Crowdsourcing a word-emotion association lexicon. Comput Intell 2013;29(3): 436–465.
Pang B, Lee L. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the ACL.
Bo P, Lee L. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the ACL.
Pennington J, Socher R, Manning C. 2014. Glove: Global vectors for word representation. In: proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Poria S, Cambria E, Bajpai R, Hussain A. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 2017;37:98–125.
Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst 2016;108:42–49.
Poria S, Cambria E, Hazarika D, Vij P. A deeper look into sarcastic tweets using deep convolutional neural networks. In: COLING; 2016. p. 1601–1612.
Poria S, Chaturvedi I, Cambria E, Hussain A. Convolutional MKL based multimodal emotion recognition and sentiment analysis. In: ICDM; 2016. p. 439–448, Barcelona.
Rajagopal D, Cambria E, Olsher D, Kwok K. A graph-based approach to commonsense concept extraction and semantic similarity detection. In: WWW; 2013. p. 565–570, Rio De Janeiro.
Socher R, Perelygin A, Wu JY, Chuang J, Manning CD, Ng AY, Potts C. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In: proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, p. 1642, Citeseer.
Tang D, Wei F, Qin B, Yang N, Liu T, Zhou M. Sentiment embeddings with applications to sentiment analysis. Knowledge and Data Engineering, IEEE Transactions on 2016;28(2):496–509.
Tang J, Qu M, Mei Q. Pte: Predictive text embedding through large-scale heterogeneous text networks. In: proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2015. p. 1165–1174, ACM.
Wang S, Tang J, Aggarwal C, Liu H. Linked document embedding for classification. In: CIKM. ACM; 2016.
Wang Y, Wang S, Tang J, Liu H, Li B. Unsupervised sentiment analysis for social media images. In: Proceedings of the Twenty Fourth International Joint Conference on Artificial Intelligence, IJCAI, 2015, Buenos Aires, Argentina; 2015. p. 2378– 2379.
Wilson T, Wiebe J, Hoffmann P. Recognizing contextual polarity in phrase-level sentiment analysis. International Journal of Computer Applications 2005;7(5):347–354.
Zhong X, Sun A, Cambria E. 2017. Time expression analysis and recognition using syntactic token types and general heuristic rules. In: ACL.
Zhou C, Sun C, Liu Z, Lau FCM. 2015. Category enhanced word embedding. Computer Science.
Zou WY, Socher R, Cer DM, Manning CD. Bilingual word embeddings for phrase-based machine translation. In: EMNLP; 2013. p. 1393–1398.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Informed Consent
Informed consent was not required as no human or animals were involved.
Human and Animal Rights
This article does not contain any studies with human or animal subjects performed by any of the authors.
Rights and permissions
About this article
Cite this article
Li, Y., Pan, Q., Yang, T. et al. Learning Word Representations for Sentiment Analysis. Cogn Comput 9, 843–851 (2017). https://doi.org/10.1007/s12559-017-9492-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-017-9492-2