Advertisement

A Deep Recursive Neural Network Model for Fine-Grained Opinion Classification

  • Ramesh S. WadawadagiEmail author
  • Veerappa B. Pagi
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1037)

Abstract

In recent times, deep neural networks (DNN) have acquired greater significance in providing solutions to many deep learning tasks. Particularly, recursive neural networks (RNN) have been efficiently utilized in exploring semantic compositions for natural language content represented with structured formats (e.g. parse-trees). Despite the fact that RNN are deep in structure, yet they fail to exhibit hierarchical representations observed in traditional deep feed-forward networks (DFNN) and also in revolutionary deep recurrent neural networks (DRcNN). However, the notion of depth can be incorporated through stacking multiple recursive layers, which results in deep recursive neural networks (DRNN). On the other hand, enhanced word spaces offer added benefits in capturing fine-grained semantic regularities. In this paper, we address the problem of fine-grained opinion classification using DRNN and word embeddings. Furthermore, the efficiency of DRNN model is estimated through the conduction of a series of experiments over several opinion datasets. The results report that the proposed DRNN architecture achieves better prediction rate for fine-grained classification when compared with conventional shallow counterparts that employ similar parameters.

Keywords

Deep learning Recursive neural networks Fine-grained opinion classification Compositional semantics 

References

  1. 1.
    Wang, H., Can, D., Kazemzadeh, A., Bar, F., Narayanan, S.: A system for real-time Twitter sentiment analysis of 2012 US presidential election cycle. In: Proceedings of the ACL 2012 System Demonstrations, pp. 115–120 (2012)Google Scholar
  2. 2.
    Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E.: A survey of deep neural network architectures and their applications. Neurocomputing 234, 11–26 (2017)CrossRefGoogle Scholar
  3. 3.
    Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2013 (2013)Google Scholar
  4. 4.
    Irsoy, O., Cardie, C.: Deep recursive neural networks for compositionality in language. In: Advances in Neural Information Processing Systems, pp. 1–4 (2014)Google Scholar
  5. 5.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)CrossRefGoogle Scholar
  6. 6.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  7. 7.
    Gori, M., Maggini, M., Sarti, L.: A recursive neural network model for processing directed acyclic graphs with labeled edges. In: Proceedings of the International Joint Conference on Neural Networks (2003)Google Scholar
  8. 8.
    Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–221 (1990)CrossRefGoogle Scholar
  9. 9.
    Pascanu, R., Gulcehre, C., Cho, K., Bengio, Y.: How to construct deep recurrent neural networks. arXiv preprint arXiv:1312.6026 (2013)
  10. 10.
    Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: ICML-2013 (2013)Google Scholar
  11. 11.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  12. 12.
    Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR-15), pp. 959–962 (2015)Google Scholar
  13. 13.
    Poria, S., Cambria, E., Gelbukh, A.: Aspect extraction for opinion mining with a deep convolutional neural network. Knowl.-Based Syst. 108, 42–49 (2016)CrossRefGoogle Scholar
  14. 14.
    Ruder, S., Ghaffari, P., Breslin, J.G.: INSIGHT-1 at SemEval-2016 Task 5: deep learning for multilingual aspect-based sentiment analysis. In: Proceedings of SemEval-2016, pp. 330–336 (2016)Google Scholar
  15. 15.
    Poria, S., Cambria, E., Hazarika, D., Vij, P.: A deeper look into sarcastic tweets using deep convolutional neural networks. In: Proceedings of COLING 2016 (2016)Google Scholar
  16. 16.
    Ukil, S., Ghosh, S., Obaidullah, Sk.Md., Santosh, K.C., Roy, K., Das, N.: Deep learning for word-level handwritten Indic script identification. Cornell University Library (arXiv:1801.01627) (2018)
  17. 17.
    Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)CrossRefGoogle Scholar
  18. 18.
    Pascanu, R., Gulcehre, C., Cho, K., Bengio, Y.: How to construct deep recurrent neural networks. In: Proceedings of ICLR 2014, pp. 1–10 (2014)Google Scholar
  19. 19.
    İrsoy, O., Cardie, C.: Modeling compositionality with multiplicative recurrent neural networks. In: Proceedings of ICLR 2015 (2015)Google Scholar
  20. 20.
    Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. Comput. Intell. Mag. 13, 55–75 (2018)CrossRefGoogle Scholar
  21. 21.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  22. 22.
    Wang, X., Liu, Y., Sun, C., Wang, B., Wang, X.: Predicting polarities of tweets by composing word-embeddings with long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 1343–1353 (2015)Google Scholar
  23. 23.
    Graves, A., Mohamed, A., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (2013)Google Scholar
  24. 24.
    Bowman, S.R., Potts, C., Manning, C.D.: Recursive neural networks can learn logical semantics. In: Proceedings of the third Workshop on Continuous Vector Space Models and their Compositionality (2015)Google Scholar
  25. 25.
    Tang, D., Qin, B., Liu, T.: Aspect level sentiment classification with deep memory network. In: Proceeding of EMNLP 2016 (2016)Google Scholar
  26. 26.
    Liu, S., Yang, N., Li, M., Zhou, M.: A recursive recurrent neural network for statistical machine translation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1491–1500 (2014)Google Scholar
  27. 27.
    Hassan, A., Mahmood, A.: Convolutional recurrent deep learning model for sentence classification. IEEE Access 6, 13949–13957 (2018)CrossRefGoogle Scholar
  28. 28.
    Li, Z., Huang, J., Zhou, Z., Zhang, H., Chang, S., Huang, Z.: LSTM-based deep learning models for answer ranking. In: 2016 IEEE First International Conference on Data Science in Cyberspace (DSC) (2016)Google Scholar
  29. 29.
    Van, V.D., Thai, T., Nghiem, M.-Q.: Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the Eighth International Symposium on Information and Communication Technology 2017. ACM (2018)Google Scholar
  30. 30.
    Socher, R., Lin, C.C.-Y., Ng, A.Y., Manning, C.D.: Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML 2011), pp. 129–136 (2011)Google Scholar
  31. 31.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, vol. 15, pp. 315–323 (2011)Google Scholar
  32. 32.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of Words and Phrases and their compositionality. arXiv:1310.4546, pp. 1–9 (2013)
  33. 33.
    Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  34. 34.
    Liu, P., Qiu, X., Huang, X.: Learning context-sensitive word-embeddings with neural tensor skip-gram model. In: Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI 2015), pp. 1284–1290 (2015)Google Scholar
  35. 35.
    Melamud, O., Goldberger, J., Dagan, I.: Context2vec: learning generic context embedding with bidirectional LSTM. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp 51–61 (2016)Google Scholar
  36. 36.
    Srivastava, N.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2012), pp. 1201–1211 (2012)Google Scholar
  39. 39.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014)

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Computer ApplicationsBasaveshwar Engineering CollegeBagalkotIndia
  2. 2.Department of Computer Science and EngineeringBasaveshwar Engineering CollegeBagalkotIndia

Personalised recommendations