Advertisement

Role of Data Properties on Sentiment Analysis of Texts via Convolutions

  • Erion Çano
  • Maurizio Morisio
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 745)

Abstract

Dense and low dimensional word embeddings opened up the possibility to analyze text polarity with highly successful deep learning techniques like Convolution Neural Networks. In this paper we utilize pretrained word vectors in combination with simple neural networks of stacked convolution and max-pooling layers, to explore the role of dataset size and document length in sentiment polarity prediction. We experiment with song lyrics and reviews of products or movies and see that convolution-pooling combination is very fast and yet quiet effective. We also find interesting relations between dataset size, text length and length of feature maps with classification accuracy. Our next goal is the design of a generic neural architecture for analyzing polarity of various text types, with high accuracy and few hyper-parameter changes.

Keywords

Textual sentiment analysis Convolution Neural Networks Text dataset properties 

Notes

Acknowledgments

This work was supported by a fellowship from TIM (https://www.tim.it). Part of computational resources was provided by HPC@POLITO (http://hpc.polito.it), a project of Academic Computing within the Department of Control and Computer Engineering at Politecnico di Torino.

References

  1. 1.
    Bengio, Y., Ducharme, R., Vincent, P., Janvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)zbMATHGoogle Scholar
  2. 2.
    Çano, E., Morisio, M.: Quality of word embeddings on sentiment analysis tasks. In: 22nd International Conference on Applications of Natural Language to Information Systems, NLDB 2017, pp. 332–338, Liège, Belgium. Springer International Publishing, 21–23 June 2017CrossRefGoogle Scholar
  3. 3.
    Çano, E., Morisio, M.: MoodyLyrics: a sentiment annotated lyrics dataset. In: 2017 International Conference on Intelligent Systems, Metaheuristics and Swarm Intelligence, pp. 118–124, Hong Kong. ACM, March 2017Google Scholar
  4. 4.
    Çano, E., Morisio, M.: Music mood dataset creation based on last.fm tags. Computer Science & Information Technology (CS & IT), vol. 7, pp. 15–26. AIRCC Publishing Corporation, Vienna (2017)CrossRefGoogle Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR, abs/1512.03385 (2015)Google Scholar
  6. 6.
    Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Machine Learning, ECML 1998, pp. 137–142 (1998)CrossRefGoogle Scholar
  7. 7.
    Johnson, R., Zhang, T.: Effective use of word order for text categorization with convolutional neural networks. CoRR, abs/1412.1058 (2014)Google Scholar
  8. 8.
    Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 August, vol. 1, Long Papers, pp. 562–570 (2017)Google Scholar
  9. 9.
    Kim, Y.: Convolutional neural networks for sentence classification. CoRR, abs/1408.5882 (2014)Google Scholar
  10. 10.
    Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI, vol. 333, pp. 2267–2273 (2015)Google Scholar
  11. 11.
    Lau, J.H., Baldwin, T.: An empirical evaluation of doc2vec with practical insights into document embedding generation. CoRR, abs/1607.05368 (2016)Google Scholar
  12. 12.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  13. 13.
    Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT 2011, pp. 142–150, Stroudsburg, PA, USA. Association for Computational Linguistics (2011)Google Scholar
  14. 14.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: 27th Annual Conference on Neural Information Processing Systems, pp. 3111–3119, December 2013Google Scholar
  15. 15.
    Szegedy, C., Ioffe, S., Vanhoucke, V.: Inception-v4, inception-resnet and the impact of residual connections on learning. CoRR, abs/1602.07261 (2016)Google Scholar
  16. 16.
    Wang, S., Manning, C.D.: Baselines and bigrams: simple, good sentiment and topic classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, ACL 2012, Stroudsburg, PA, USA, pp. 90–94. Association for Computational Linguistics (2012)Google Scholar
  17. 17.
    Zhang, M., Zhang, Y., Vo, D.-T.: Gated neural networks for targeted sentiment analysis. In: AAAI, pp. 3087–3093 (2016)Google Scholar
  18. 18.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS 2015, Cambridge, MA, USA, pp. 649–657. MIT Press (2015)Google Scholar
  19. 19.
    Zhang, X., Zhao, J.J., LeCun, Y.: Character-level convolutional networks for text classification. CoRR, abs/1509.01626 (2015)Google Scholar
  20. 20.
    Zhao, H., Lu, Z., Poupart, P.: Self-adaptive hierarchical sentence model. CoRR, abs/1504.05070 (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Politecnico di TorinoTorinoItaly

Personalised recommendations