Abstract
Aspect-based sentiment analysis (ABSA) aims at identifying sentiment polarities towards aspect in a sentence. Attention mechanism has played an important role in previous state-of-the-art neural models. However, existing attention mechanisms proposed for aspect based sentiment classification mostly focus on identifying the sentiment words, without considering the relevance of such words with respect to the given aspects in the sentence. To solve this problem, we propose a new architecture, self-attention with co-attention (SACA) for aspect-based sentiment analysis. Self-attention is capable of conducting direct connections between arbitrary two words in context from a global perspective, while co-attention can capture the word-level interaction between aspect and context. Moreover, previous works simply averaged aspect vector to learn the attention weights on the context words, which may bring information loss if the aspect has multiple words. To address the problem, we employ the pre-trained contextual word embeddings and character-level word embeddings as word representation. We evaluate the proposed approach on three datasets, experimental results demonstrate that our model outperforms the state-of-the-art on all three datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kiritchenko, S., Zhu, X., Cherry, C., Mohammad, S., Polosukhin, I.: NRC-Canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) (2014)
Wagner, J., et al.: DCU: aspect-based polarity classification for SemEval task 4. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint. arXiv:1409.0473 (2014)
Wang, Y., Huang, M., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of EMNLP (2016)
Ma, D., Li, S., Zhang, X., Wang, H.: Interactive attention networks for aspect-level sentiment classification. In: Proceedings of IJCAI (2017)
Chen, P., Sun, Z., Bing, L., Yang, W.: Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of EMNLP (2017)
He, R., Lee, W.S., Ng, H.T., Dahlmeier, D.: Effective attention modeling for aspect-level sentiment classification. In: Proceedings of the 27th International Conference on Computational Linguistics (2018)
Tang, D., Qin, B., Liu, T.: Aspect level sentiment classification with deep memory network. In: Proceedings of EMNLP (2016b)
Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of NAACL (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
Vo, D.T., Zhang, Y.: Target-dependent twitter sentiment classification with rich automatic features. In: IJCAI (2015)
Tang, D., Qin, B., Feng, X., Liu, T.: Effective LSTMs for target-dependent sentiment classification. In: COLING (2016a)
Fan, F., Feng, Y., Zhao, D.: Multi-grained attention network for aspect-level sentiment classification. In: EMNLP (2018)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems (2013)
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: EMNLP (2014)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint. arXiv:1412.3555 (2014)
Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) (2014)
Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K.: Adaptive recursive neural network for target-dependent twitter sentiment classification. In: ACL (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980 (2014)
Majumder, N., Poria, S., Gelbukh, A., Akhtar, M.S., Cambria, E., Ekbal, A.: IARM: inter-aspect relation modeling with memory networks in aspect-based sentiment analysis. In: EMNLP (2018)
Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway network. arXiv preprint. arXiv:1505.00387 (2015)
Acknowledgements
This work was supported by the Ministry of Education of Humanities and Social Science project (No. 19YJAZH128), and Guangdong Graduate Education Innovation project (No. 2018JGXM41).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, Z., Zhou, Y., Yang, A., Fu, J. (2019). Contextualized Word Representations with Effective Attention for Aspect-Based Sentiment Analysis. In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds) Chinese Computational Linguistics. CCL 2019. Lecture Notes in Computer Science(), vol 11856. Springer, Cham. https://doi.org/10.1007/978-3-030-32381-3_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-32381-3_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32380-6
Online ISBN: 978-3-030-32381-3
eBook Packages: Computer ScienceComputer Science (R0)