Abstract
Sentiment analysis is an effective technique and widely employed to analyze sentiment polarity of reviews and comments on the Internet. A lot of advanced methods have been developed to solve this task. In this paper, we propose an attention aware bidirectional GRU (Bi-GRU) framework to classify the sentiment polarity from the aspects of sentential-sequence modeling and word-feature seizing. It is composed of a pre-attention Bi-GRU to incorporate the complicated interaction between words by sentence modeling, and an attention layer to capture the keywords for sentiment apprehension. Afterward, a post-attention GRU is added to imitate the function of decoder, aiming to extract the predicted features conditioned on the above parts. Experimental study on commonly used datasets has demonstrated the proposed framework’s potential for sentiment classification.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The source code is available at https://github.com/MercuryTian/Attention-Aware-Bi-directional-GRU-Based-Framework-for-Sentiment-Analysis.git.
- 2.
- 3.
- 4.
- 5.
- 6.
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)
Bengio, Y., Simard, P.Y., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. In: Proceedings of 8th Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014)
Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)
Ding, X., Liu, B., Yu, P.S.: A holistic lexicon-based approach to opinion mining. In: Proceedings of 2008 International Conference on Web Search and Web Data Mining, pp. 231–240 (2008)
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, vol. 1. MIT press, Cambridge (2016)
He, L., Lee, K., Lewis, M., Zettlemoyer, L.: Deep semantic role labeling: what works and what’s next. In: Proceedings of 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 473–483 (2017)
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Proceedings of 2015 Annual Conference on Neural Information Processing Systems, pp. 1693–1701 (2015)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kudo, T., Matsumoto, Y.: Chunking with support vector machines. In: Proceedings of 2nd Meeting of the North American Chapter of the Association for Computational Linguistics (2001)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th International Conference on Machine Learning, pp. 282–289 (2001)
Liu, J., Rong, W., Tian, C., Gao, M., Xiong, Z.: Attention aware semi-supervised framework for sentiment analysis. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10614, pp. 208–215. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68612-7_24
Lu, Y., Salem, F.M.: Simplified gating in long short-term memory (LSTM) recurrent neural networks. CoRR abs/1701.03441 (2017)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems, pp. 2204–2212 (2014)
Mousa, A.E., Schuller, B.W.: Contextual bidirectional long short-term memory recurrent neural network language models: a generative approach to sentiment analysis. In: Proceedings of 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1023–1032 (2017)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of 2002 Conference on Empirical Methods in Natural Language Processing (2002)
Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: Proceedings of 30th International Conference on Machine Learning, pp. 1310–1318 (2013)
Qiu, L., Lei, Q., Zhang, Z.: Advanced sentiment classification of tibetan microblogs on smart campuses based on multi-feature fusion. IEEE Access 6, 17896–17904 (2018)
Raza, K., Alam, M.: Recurrent neural network based hybrid model for reconstructing gene regulatory network. Comput. Biol. Chem. 64, 322–334 (2016)
Schuller, B.W., Mousa, A.E., Vryniotis, V.: Sentiment analysis and opinion mining: on optimal parameters and performances. Wiley Interdisc. Rev. Data Min. Knowl. Disc. 5(5), 255–263 (2015)
Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing, pp. 606–615 (2016)
Wang, Y., Pal, A.: Detecting emotions in social media: a constrained optimization approach. In: Proceedings of 24th International Joint Conference on Artificial Intelligence, pp. 996–1002 (2015)
Zhang, B., Xiong, D., Su, J.: A GRU-gated attention model for neural machine translation. CoRR abs/1704.08430 (2017)
Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)
Zhao, J., Liu, K., Xu, L.: Sentiment analysis: mining opinions, sentiments, and emotions. Comput. Linguist. 42(3), 595–598 (2016)
Zhou, D., Zhang, X., Zhou, Y., Zhao, Q., Geng, X.: Emotion distribution learning from texts. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 638–647 (2016)
Zhou, J., Xu, W.: End-to-end learning of semantic role labeling using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 1127–1137 (2015)
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China (No. 61332018), and the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Tian, Z., Rong, W., Shi, L., Liu, J., Xiong, Z. (2018). Attention Aware Bidirectional Gated Recurrent Unit Based Framework for Sentiment Analysis. In: Liu, W., Giunchiglia, F., Yang, B. (eds) Knowledge Science, Engineering and Management. KSEM 2018. Lecture Notes in Computer Science(), vol 11061. Springer, Cham. https://doi.org/10.1007/978-3-319-99365-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-99365-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99364-5
Online ISBN: 978-3-319-99365-2
eBook Packages: Computer ScienceComputer Science (R0)