Effect of Activation Functions on Deep Learning Algorithms Performance for IMDB Movie Review Analysis

  • Achin Jain
  • Vanita JainEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1164)


Huge amount of data is generated every moment over the Internet on various platforms such as social networking sites, blogs, customer reviews on various sites where individuals express their views or thoughts about different subjects. Users’ sentiments expressed over the Web influence the readers, product vendors, and politicians greatly. This unstructured form of data needs to be analyzed and converted into a well-structured form and for this purpose, we require Sentiment Analysis. Sentiment Analysis is the process of contextual mining of text that is used to identify and extract the expressed mindset or feelings in different manners such as negative, positive, favorable, unfavorable, thumbs up, thumbs down, etc. In this paper, we have used Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and a hybrid approach of CNN and LSTM to perform sentiment classification of IMDB Movie Review dataset. We have applied the trained model on the dataset using various activation functions and compared the accuracy achieved. Maximum accuracy (88.35%) is achieved with CNN with ReLU Activation Function whereas minimum accuracy (48.19%) is achieved with LSTM when used with Linear Activation Function.


Sentiment analysis Deep learning Activation function Long Short-Term memory (LSTM) Convolutional neural network (CNN) 


  1. 1.
    A.V. Yeole, P.V. Chavan, M.C. Nikose, Opinion mining for emotions determination. in 2015 IEEE International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS), (2015)Google Scholar
  2. 2.
    B. Heredia, T.M. Khoshgoftaar, J. Prusa, M. Crawford, Cross domain sentiment analysis: an empirical investigation, in 2016 IEEE 17th International Conference Information Reuse and Integration, 2016, pp. 160–165Google Scholar
  3. 3.
    F. Luo, C. Li, Z. Cao, Affective-feature-based sentiment analysis using SVM classifier, in 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design, 2016, pp. 276–281Google Scholar
  4. 4.
    M. Haenlein, A.M. Kaplan, An empirical analysis of attitudinal and behavioral reactions toward the abandonment of unprofitable customer relationships. J. Relatsh. Mark. 9(4), 200–228 (2010)Google Scholar
  5. 5.
    E. Aydogan, M.A. Akcayol, A comprehensive survey for sentiment analysis tasks using machine learning techniques, in 2016 International Symposium on Innovations in Intelligent Systems and Applications, (2016), p. 17Google Scholar
  6. 6.
    J. Singh, G. Singh, R. Singh, A review of sentiment analysis techniques for opinionated web text (CSI Trans, ICT, 2016)CrossRefGoogle Scholar
  7. 7.
    K. Schouten, F. Frasincar, Survey on aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng. 28(3), 813–830 (2015)CrossRefGoogle Scholar
  8. 8.
    R. Moraes, J.F. Valiati, W.P.G. Neto, Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Syst. Appl. 40(2), 621–633 (2013)CrossRefGoogle Scholar
  9. 9.
    D.M.E.D.M. Hussein, A survey on sentiment analysis challenges. J. King Saud Univ. Eng. Sci. 30(4), 330–338 (2018)Google Scholar
  10. 10.
    Y. Wang, K. Kim, B. Lee, H.Y. Youn, Word clustering based on POS feature for efficient twitter sentiment analysis. Human-Centric Comput. Inf. Sci. 8(1), 17 (2018)CrossRefGoogle Scholar
  11. 11.
    S. Pandey, S. Sagnika, B.S.P Mishra, A technique to handle negation in sentiment analysis on movie reviews. in 2018 International Conference on Communication and Signal Processing (ICCSP) 2018, (pp. 0737–0743). IEEEGoogle Scholar
  12. 12.
    Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  13. 13.
    S. Wen, H. Wei, Y. Yang, Z. Guo, Z. Zeng, T. Huang, Y. Chen. Memristive LSTM network for sentiment analysis. in IEEE Transactions on Systems, Man, and Cybernetics: Systems (2019)Google Scholar
  14. 14.
    H.H. Do, P.W.C. Prasad, A. Maag, A. Alsadoon, Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst. Appl. 118, 272–299 (2019)CrossRefGoogle Scholar
  15. 15.
    J. Islam, Y. Zhang, Visual sentiment analysis for social images using transfer learning approach, in 2016 IEEE International Conference on Innovations in Information, Embedded and Communication Systems (BDCloud), Soc. Comput. Netw. (SocialCom), Sustain. Comput. Commun., 2016 (pp. 124–130)Google Scholar
  16. 16.
    A. Severyn, A. Moschitti, Twitter sentiment analysis with deep convolutional neural networks. in Proceedings 38th International ACM SIGIR Conference Research Development, Information Retrieval SIGIR, vol. 15, pp. 959–962 (2015)Google Scholar
  17. 17.
    L. Yanmei, C. Yuda, Research on Chinese micro-blog sentiment analysis based on deep learning, in 2015 8th International Symposium on Computational Intelligence and Design, 2015 (pp. 358–361)Google Scholar
  18. 18.
    X. Ouyang, P. Zhou, C.H. Li, L. Liu, Sentiment analysis using convolutional neural network, Comput. in 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing (CIT/IUCC/DASC/PICOM), 2015, (pp. 2359–2364)Google Scholar
  19. 19.
    P. Vateekul, T. Koomsubha, A study of sentiment analysis using deep learning techniques on thai twitter data, 2016Google Scholar
  20. 20.
    R. Ghosh, K. Ravi, V. Ravi, A novel deep learning architecture for sentiment classification, in 3rd IEEE International Conference on Recent Advances in Information Technology (2016) pp. 511–516Google Scholar
  21. 21.
    C. Nwankpa, W. Ijomah, A. Gachagan, S. Marshall, Activation functions: comparison of trends in practice and research for deep learning. arXiv preprint arXiv:1811.03378, pp. 1–20 (Nov 2018)
  22. 22.
    B. Xu, N. Wang, T. Chen, M. Li, Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853, pp. 1–5 (May 2015)
  23. 23.
    K. Alrawashdeh, C. Purdy, Fast activation function approach for deep learning based online anomaly intrusion detection. in International Conference on Big Data Security on Cloud, Omaha, pp. 5–13 (May 2018)Google Scholar
  24. 24.
    A.L. Maas, R.E. Daly, P.T. Pham, D. Huang, A.Y. Ng, C. Potts, Learning word vectors for sentiment analysis. in Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies vol. 1, Association for Computational Linguistics, 2011 (pp. 142–150)Google Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021

Authors and Affiliations

  1. 1.University School of Information, Communication and TechnologyGGSIPUDwarkaIndia
  2. 2.Bharati Vidyapeeth’s College of EngineeringNew DelhiIndia

Personalised recommendations