Abstract
Aspect extraction or feature extraction is a crucial and challenging task of opinion mining that aims to identify opinion targets from opinion text. Especially, how to explore these aspects or features from unstructured comments is a matter of concern. In this paper, we propose a novel supervised learning approach using deep learning technique for the above-mentioned aspect extraction task. Our model combines a bidirectional independently long short-term memory (Bi-IndyLSTM) with a Conditional Random Field (CRF). This integrated model is trained on labelled data to extract feature sets in opinion text. We employ a Bi-IndyLSTM with word embeddings achieved by training GloVe on the SemEval 2014 data set. There are 6086 training reviews and 1600 testing reviews on two domains, Laptop and Restaurant of the SemEval 2014 data set. Experimental results showed that our proposed Bi-IndyLSTM-CRF aspect extraction model in sentiment analysis obtained considerably better accuracy than the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, B.: Sentiment Analysis and Opinion Mining. Morgan and Claypool Publishers, San Rafael (2012)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP’14 Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543. Doha, 25–29 Oct 2014
Hu, M., Liu, B: Mining opinion features in customer reviews. In: AAAI’04 Proceedings of the 19th National Conference on Artificial Intelligence, pp. 755–760, 25–29 July 2004
Popescu, A.M., Etzioni O.: Extracting product features and opinions from reviews. In: HLT‘05 Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 339–346, 6–8 Oct 2005
Scaffidi, C., Bierhoff, K., Chang, E., Felker, M., Ng, H., Jin, C.: Red Opal: product-feature scoring from reviews. In: EC’07 Proceedings of the 8th ACM Conference on Electronic Commerce, vol. 2, pp. 182–191, 11–15 June 2007
Long, C., Zhang, Z., Zhu, X.: A review selection approach for accurate feature rating estimation. In: COLING’10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 766–774. Beijing, 23–27 Aug 2010
Kobayashi, N., Inui, K., Matsumoto, Y.: Extracting aspect–evaluation and aspect-of relations in Opinion mining. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1065–1074. Prague, June 2007
Wu, Y., Zhang, Q., Huang, X., Wu, L: Phrase dependency parsing for opinion mining. In: EMNLP’09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 3, pp. 1533–1541. Singapore, 6–7 Aug 2009
Qiu, G., Liu, B., Bu, J., Chen, C.: Opinion word expansion and target extraction through double propagation. In: 2011 Association for Computational Linguistics, vol. 37, no. 1 (2011)
Lu, Y., Zhai, C.X., Sundaresan, N.: Rated aspect summarization of short comments. In: WWW’09 Proceedings of the 18th ACM International Conference on World Wide Web, pp. 131–140. Madrid, 20–24 Apr 2009
Lin, C., He, Y.: Joint sentiment/topic model for sentiment analysis. In: CIKM’09 Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 375–384. Hong Kong, 2–6 Nov 2009
Brody, S., Elhadad, N.: An unsupervised aspect-sentiment model for online reviews. In: HLT’10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 804–812. Los Angeles, 2–4 June 2010
Jakob, N., Gurevych, I.: Extracting opinion targets in a single- and cross-domain setting with conditional random fields. In: EMNLP’10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1035–1045. Cambridge, Massachusetts, 9–11 Oct 2010
Li, F., Han, C., Huang, M., Zhu, X., Xia, Y.-J., Zhang, S., Yu, H.: Structure-aware review mining and summarization. In: COLING’10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 653–661. Beijing, 23–27 Aug 2010
Zhang, M., Zhang, Y., Vo, D.T.: Neural networks for open domain targeted sentiment. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 612–621. Lisbon, 17–21 Sept 2015
Wang, W., Pan, S.J., Dahlmeier, D., Xiao, X.: Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis. ArXiv:1603.06679v3 [cs.CL], 19 Sept 2016
Katiyar, A., Cardie, C.: Investigating LSTMs for joint extraction of opinion entities and relations. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 919–929. Berlin, Aug 2016
Irsoi, O., Cardie, C.: Opinion mining with deep recurrent neural networks. In: EMNLP’14 Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 720–728. Doha, 25–29 Oct 2014
Liu, P., Joty, S., Meng, H.: Fine-grained opinion mining with recurrent neural networks and word embeddings. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1433–1443. Lisbon, 17–21 Sept 2015
Poria, S., Cambria, E., Gelbukh, A.: Aspect extraction for opinion mining with a deep convolutional neural network. In: Knowledge-Based Systems, vol. 108, pp. 42–49, 15 Sept 2016
Giannakopoulos, A., Musat, C., Hossmann, A., Baeriswyl, M.: Unsupervised Aspect Term Extraction with B-LSTM & CRF Using Automatically Labelled Datasets. ArXiv:1709.05094v1 [cs.CL], 15 Sept 2017
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. ArXiv:1406.1078v3 [cs.CL], 3 Sept 2014
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166. New York, USA (1994)
Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently Recurrent Neural Network (IndRNN): Building a Longer and Deeper RNN. ArXiv: 1803.04831v3 [cs.CV], 22 May 2018
Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th ACM International Conference on Machine Learning, pp. 282–289, 28 June 2001
Yang, B., Cardie, C.: Extracting opinion expressions with semi-Markov conditional random fields. In: EMNLP-CoNLL’12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1335–1345. Jeju Island, Korea, 12–14 July 2012
Rubtsova, Y., Koshelnikov, S.: Aspect extraction from reviews using conditional random fields. In: International Conference on Knowledge Engineering and the Semantic Web, KESW 2015, CCIS, vol. 518, pp. 158–167, 30 Oct 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tran, T.U., Hoang, H.TT., Huynh, H.X. (2020). Bidirectional Independently Long Short-Term Memory and Conditional Random Field Integrated Model for Aspect Extraction in Sentiment Analysis. In: Satapathy, S., Bhateja, V., Nguyen, B., Nguyen, N., Le, DN. (eds) Frontiers in Intelligent Computing: Theory and Applications. Advances in Intelligent Systems and Computing, vol 1014. Springer, Singapore. https://doi.org/10.1007/978-981-13-9920-6_14
Download citation
DOI: https://doi.org/10.1007/978-981-13-9920-6_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9919-0
Online ISBN: 978-981-13-9920-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)