Abstract
Analyzing human reactions from text is an important step towards automated modeling of affective content. The variance in human perceptions and experiences leads to a lack of uniform, well-labeled, ground-truth datasets, hence, limiting the scope of neural supervised learning approaches. Recurrent and convolutional networks are popular for text classification and generation tasks, specifically, where large datasets are available; but are inefficient when dealing with unlabeled corpora. We propose a gated sequence-to-sequence, convolutional-deconvolutional autoencoding (GCNN-DCNN) framework for affect classification with limited labeled data. We show that compared to a vanilla CNN-DCNN network, gated networks improve performance for affect prediction as well as text reconstruction. We present a regression analysis comparing outputs of traditional learning models with information captured by hidden variables in the proposed network. Quantitative evaluation with joint, pre-trained networks, augmented with psycholinguistic features, reports highest accuracies for affect prediction, namely frustration, formality, and politeness in text.
K. Chawla and S. Khosla—denotes equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Link to the annotated ENRON-FFP dataset: https://bit.ly/2IAxPab.
- 2.
References
Burgoon, J.K., Hale, J.L.: The fundamental topoi of relational communication. Commun. Monogr. 51(3), 193–214 (1984). https://doi.org/10.1080/03637758409390195
Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)
Cohen, W.W.: Enron email dataset (2009)
Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. In: Advances in Neural Information Processing Systems, pp. 3079–3087 (2015)
Danescu-Niculescu-Mizil, C., Sudhof, M., Jurafsky, D., Leskovec, J., Potts, C.: A computational approach to politeness with application to social factors. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (2013)
Dauphin, Y.N., Fan, A., Auli, M., Grangier, D.: Language modeling with gated convolutional networks. CoRR abs/1612.08083 (2016)
Dieng, A.B., Wang, C., Gao, J., Paisley, J.W.: TopicRNN: a recurrent neural network with long-range semantic dependency. CoRR abs/1611.01702 (2016)
Ghosh, S., Chollet, M., Laksana, E., Morency, L.P., Scherer, S.: Affect-LM: a neural language model for customizable affective text generation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (2017)
Kalchbrenner, N., Espeholt, L., Simonyan, K., van den Oord, A., Graves, A., Kavukcuoglu, K.: Neural machine translation in linear time. arXiv preprint arXiv:1610.10099 (2016)
Ke, Y., Hagiwara, M.: Alleviating overfitting for polysemous words for word representation estimation using lexicons. In: International Joint Conference on Neural Networks (2017)
Khosla, S., Chhaya, N., Chawla, K.: Aff2Vec: affect-enriched distributional word representations. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 2204–2218. Association for Computational Linguistics (2018)
Lee, D.H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML (2013)
Li, J., Luong, M., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. CoRR abs/1506.01057 (2015)
Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: Proceedings of ACL Workshop on Text Summarization Branches Out (2004)
Mairesse, F., Walker, M.A.: Trainable generation of big-five personality styles through data-driven parameter estimation. In: ACL, pp. 165–173 (2008)
Majumder, N., Poria, S., Gelbukh, A., Cambria, E.: Deep learning-based document modeling for personality detection from text. IEEE Intell. Syst. 32(2), 74–79 (2017)
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., Khudanpur, S.: Recurrent neural network based language model. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association, vol. 2010, pp. 1045–1048 (2010)
Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., et al.: Conditional image generation with pixelCNN decoders. In: NIPS (2016)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL 2002, pp. 311–318 (2002)
Pavlick, E., Tetreault, J.: An empirical analysis of formality in online communication. Trans. Assoc. Comput. Linguist. 4, 61–74 (2016)
Pennebaker, J.W.: The secret life of pronouns. New Scientist 211(2828), 42–45 (2011)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)
Preotiuc-Pietro, D., Liu, Y., Hopkins, D.J., Ungar, L.: Personality driven differences in paraphrase preference. In: Proceedings of the Workshop on Natural Language Processing and Computational Social Science (NLP+CSS). ACL (2017)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
dos Santos, C., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 69–78 (2014)
Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2015, pp. 959–962 (2015)
Shen, Y., Lin, Z., Huang, C., Courville, A.C.: Neural language modeling by jointly learning syntax and lexicon. CoRR abs/1711.02013 (2017)
Subramanian, S., Baldwin, T., Cohn, T.: Content-based popularity prediction of online petitions using a deep regression model. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 182–188 (2018)
Titov, I., Klementiev, A.: Semi-supervised semantic role labeling: approaching from an unsupervised perspective. In: Proceedings of COLING 2012, pp. 2635–2652 (2012)
Yin, W., Kann, K., Yu, M., Schütze, H.: Comparative study of CNN and RNN for natural language processing. CoRR abs/1702.01923 (2017)
Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems, pp. 649–657 (2015)
Zhang, Y., Shen, D., Wang, G., Gan, Z., Henao, R., Carin, L.: Deconvolutional paragraph representation learning. CoRR abs/1708.04729 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Chawla, K., Khosla, S., Chhaya, N. (2019). Gated Convolutional Encoder-Decoder for Semi-supervised Affect Prediction. In: Yang, Q., Zhou, ZH., Gong, Z., Zhang, ML., Huang, SJ. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2019. Lecture Notes in Computer Science(), vol 11439. Springer, Cham. https://doi.org/10.1007/978-3-030-16148-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-16148-4_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-16147-7
Online ISBN: 978-3-030-16148-4
eBook Packages: Computer ScienceComputer Science (R0)