Advertisement

A Hybrid CNN-LSTM Model for Improving Accuracy of Movie Reviews Sentiment Analysis

  • Anwar Ur Rehman
  • Ahmad Kamran MalikEmail author
  • Basit Raza
  • Waqar Ali
Article
  • 20 Downloads

Abstract

Nowadays, social media has become a tremendous source of acquiring user’s opinions. With the advancement of technology and sophistication of the internet, a huge amount of data is generated from various sources like social blogs, websites, etc. In recent times, the blogs and websites are the real-time means of gathering product reviews. However, excessive number of blogs on the cloud has enabled the generation of huge volume of information in different forms like attitudes, opinions, and reviews. Therefore, a dire need emerges to find a method to extract meaningful information from big data, classify it into different categories and predict end user’s behaviors or sentiments. Long Short-Term Memory (LSTM) model and Convolutional Neural Network (CNN) model have been applied to different Natural Language Processing (NLP) tasks with remarkable and effective results. The CNN model efficiently extracts higher level features using convolutional layers and max-pooling layers. The LSTM model is capable to capture long-term dependencies between word sequences. In this study, we propose a hybrid model using LSTM and very deep CNN model named as Hybrid CNN-LSTM Model to overcome the sentiment analysis problem. First, we use Word to Vector (Word2Vc) approach to train initial word embeddings. The Word2Vc translates the text strings into a vector of numeric values, computes distance between words, and makes groups of similar words based on their meanings. Afterword embedding is performed in which the proposed model combines set of features that are extracted by convolution and global max-pooling layers with long term dependencies. The proposed model also uses dropout technology, normalization and a rectified linear unit for accuracy improvement. Our results show that the proposed Hybrid CNN-LSTM Model outperforms traditional deep learning and machine learning techniques in terms of precision, recall, f-measure, and accuracy. Our approach achieved competitive results using state-of-the-art techniques on the IMDB movie review dataset and Amazon movie reviews dataset.

Keywords

Natural Language Processing (NLP) Sentiment Analysis CNN LSTM 

Notes

Acknowledgements

This work is funded by the COMSATS University Islamabad (CUI), Islamabad, Pakistan, CUI/ORICPD/19.

References

  1. 1.
    Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) SA using deep learning techniques: a review. Int J Adv Comput Sci Appl, 8(6)Google Scholar
  2. 2.
    Al-Smadi M, Talafha B, Al-Ayyoub M, Jararweh Y (2018) Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews. International Journal of Machine Learning and Cybernetics, 1-13Google Scholar
  3. 3.
    Amolik A, Jivane N, Bhandari M, Venkatesan M (2016) Twitter sentiment analysis of movie reviews using machine learning techniques. Int J Eng Technol 7(6):1–7Google Scholar
  4. 4.
    Cheng Z, Ying D, Lei Z, Mohan K (2018) Aspect-aware latent factor model: Rating prediction with ratings and reviews. In: Proceedings of theWorld Wide Web Conference on World Wide Web, pp. 639-648Google Scholar
  5. 5.
    Cheng Z, Ying D, Xiangnan H, Lei Z, Xuemeng S, Mohan K (2018) A^ 3NCF: An Adaptive Aspect Attention Model for Rating Prediction. In IJCAI, pp. 3748-3754Google Scholar
  6. 6.
    Cheng Z, Xiaojun C, Lei Z, Rose C, Mohan K (2019) MMALFM: Explainable recommendation by leveraging reviews and images. ACM Trans Inf Syst 37(2):16CrossRefGoogle Scholar
  7. 7.
    Collobert R, Weston J (2008) A united architecture for natural language processing: Deep neural networks with multitask learning, in Proc. 25th Int. Conf. Mach. Learn., pp. 160-167Google Scholar
  8. 8.
    Conneau A, Schwenk H, Barrault L, Lecun Y (2017) Very deep convolutional networks for text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1107-1116Google Scholar
  9. 9.
    Elghazaly T, Mahmoud A, Hefny HA (2016) Political sentiment analysis using twitter data. In Proceedings of the ACM International Conference on Internet of things and Cloud Computing, pp.11Google Scholar
  10. 10.
    Fang X, Zhan J (2015) Sentiment analysis using product review data. J Big Data 2(1):5CrossRefGoogle Scholar
  11. 11.
    Govindarajan M (2013) Sentiment analysis of movie reviews using hybrid method of naive bayes and genetic algorithm. Inte J Adv Comput Res 3(4):139Google Scholar
  12. 12.
    Hao H (2014) Recursive deep learning for sentiment analysis over social data. In Proceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), pp. 180-185Google Scholar
  13. 13.
    Hassan A, Mahmood A (2018) Convolutional Recurrent Deep Learning Model for Sentence Classification. IEEE Access 6:13949–13957CrossRefGoogle Scholar
  14. 14.
    Himelboim I, Smith MA, Rainie L, Shneiderman B, Espina C (2017) Classifying twitter topic-networks using social network analysis. Social Media+ Society, 1-13Google Scholar
  15. 15.
    Islam, J, Zhang Y (2016) Visual sentiment analysis for social images using transfer learning approach. In IEEE International Conferences on Big Data and Cloud Computing (BDCloud) pp. 124-130Google Scholar
  16. 16.
    Kaur A, Gupta V (2013) A survey on SA and opinion mining techniques. J Emerg Technol Web Intell 5(4):367–371Google Scholar
  17. 17.
    Li G, Hoi SC, Chang K, Jain R (2010) Micro-blogging sentiment detection by collaborative online learning. In IEEE 10th International Conference on Data Mining (ICDM), pp. 893-898Google Scholar
  18. 18.
    Liao S, Wang J, Yu R, Sato K, & Cheng Z (2017) CNN for situations understanding based on SA of twitter data. Procedia Computer Science, 376-381Google Scholar
  19. 19.
    Manek AS, Shenoy PD, Mohan MC, Venugopal KR (2017) Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM. World wide web 20(2):135–154CrossRefGoogle Scholar
  20. 20.
    McCallum A, Nigam K (1998) A comparison of event models for naïve Bayes text classi-cation, in Proc. AAAI Workshop Learn. Text Catego-rization, pp. 41-48Google Scholar
  21. 21.
    Ouyang X, Pan Z, Cheng H, Lijun L (2015) Sentiment analysis using convolutional neural network. In IEEE 2015 International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, , pp. 2359-2364Google Scholar
  22. 22.
    Pang B, Lee L (2005) Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd annual meeting on association for computational linguistics, pp. 115-124Google Scholar
  23. 23.
    Ruangkanokmas P, Achalakul T, Akkarajitsakul K (2016) Deep belief networks with feature selection for sentiment classification. In IEEE 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS), pp. 9-14Google Scholar
  24. 24.
    Sanguansat P (2016) Paragraph2vec-based sentiment analysis on social media for business in thailand. In IEEE 8th International Conference on Knowledge and Smart Technology (KST), pp. 175-178Google Scholar
  25. 25.
    Severyn A, & Moschitti A (2015) Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval pp. 959-962. ACMGoogle Scholar
  26. 26.
    Singh J, Singh G, Singh R (2017) Optimization of sentiment analysis using machine learning classifiers. Human-centric Comput Inf Sci 7(1):32CrossRefGoogle Scholar
  27. 27.
    Srivastava A, Singh MP, Kumar P (2014) Supervised semantic analysis of product reviews using weighted k-NN classifier. In 11th IEEE International Conference on Information Technology: New Generations (ITNG), pp. 502-507Google Scholar
  28. 28.
    Syed AZ, Aslam M, Martinez-Enriquez AM (2010) Lexicon based SA of Urdu text using SentiUnits. In Mexican International Conference on Artificial Intelligence Springer, Berlin, Heidelberg pp. 32-43Google Scholar
  29. 29.
    Tripathy A, Agrawal A, Rath SK (2015) Classification of Sentimental Reviews Using Machine Learning Techniques. Procedia Comput Sci 57:821–829CrossRefGoogle Scholar
  30. 30.
    Wang S, Manning CD (2012) Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational LinguisticsGoogle Scholar
  31. 31.
    Yanmei L, Yuda C (2015) Research on Chinese Micro-Blog Sentiment Analysis Based on Deep Learning. In IEEE 8th International Symposium on Computational Intelligence and Design (ISCID), pp. 358-361Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceCOMSATS University Islamabad (CUI)IslamabadPakistan

Personalised recommendations