Complaint Classification Using Hybrid-Attention GRU Neural Network

  • Shuyang Wang
  • Bin WuEmail author
  • Bai Wang
  • Xuesong Tong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)


Recently, a growing number of customers tend to complain about the services of different enterprises on the Internet to express their dissatisfaction. The correct classification of complaint texts is fairly important for enterprises to improve the efficiency of transaction processing. However, the existing literature lacks research on complaint texts. Most previous approaches of text classification fail to take advantage of the information of specific characters and negative emotions in complaint texts. Besides, some grammatical and semantic errors caused by violent mood swings of customers are another challenge. To address the problems, a novel model based on hybrid-attention GRU neural network (HATT-GRU) is proposed for complaint classification. The model constructs text vectors at character level, and it is able to extract sentiment features in complaint texts. Then a hybrid-attention mechanism is proposed to learn the importance of each character and sentiment feature, so that the model can focus on the features that contribute more to text classification. Finally, experiments are conducted on two complaint datasets from different industries. Experiments show that our model can achieve state-of-the-art results on both Chinese and English datasets compared to several text classification baselines.


Text classification Recurrent neural network Attention 



This work is supported by the National Key R&D Program of China (No. 2018YFC0831500) and the National Social Science Foundation of China under Grant 16ZDA055.


  1. 1.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(8), 2493–2537 (2011)zbMATHGoogle Scholar
  2. 2.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  3. 3.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
  4. 4.
    Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)Google Scholar
  5. 5.
    Wang, M., Chen, S., He, L.: Sentiment classification using neural networks with sentiment centroids. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 56–67 (2018)Google Scholar
  6. 6.
    Xia, W., Zhu, W., Liao, B., Chen, M., Cai, L., Huang, L.: Novel architecture for long short-term memory used in question classification. Neurocomputing 299, 20–31 (2018)CrossRefGoogle Scholar
  7. 7.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems, pp. 649–657 (2015)Google Scholar
  8. 8.
    Yang, J., et al.: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5216–5225 (2017)Google Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    Wang, L., Niu, J., Song, H., Atiquzzaman, M.: SentiRelated: a cross-domain sentiment classification algorithm for short texts through sentiment related index. J. Netw. Comput. Appl. 101, 111–119 (2018)CrossRefGoogle Scholar
  11. 11.
    Yang, M., Qu, Q., Chen, X., Guo, C., Shen, Y., Lei, K.: Feature-enhanced attention network for target-dependent sentiment classification. Neurocomputing 307, 91–97 (2018)CrossRefGoogle Scholar
  12. 12.
    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
  13. 13.
    Xu, J., Chen, D., Qiu, X., Huang, X.: Cached long short-term memory neural networks for document-level sentiment classification. arXiv preprint arXiv:1610.04989 (2016)
  14. 14.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: NAACL-HLT, pp. 1480–1489 (2016)Google Scholar
  15. 15.
    Conneau, A., Schwenk, H., Barrault, L., Lecun, Y.: Very deep convolutional networks for text classification. arXiv preprint arXiv:1606.01781 (2016)
  16. 16.
    Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI, pp. 2267–2273 (2015)Google Scholar
  17. 17.
    Zhou, C., Sun, C., Liu, Z., Lau, F.: A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630 (2015)
  18. 18.
    Rios, A., Kavuluru, R.: Convolutional neural networks for biomedical text classification: application in indexing biomedical articles. In: Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 258–267 (2015)Google Scholar
  19. 19.
    Seo, S., Cho, S.B.: Offensive sentence classification using character-level CNN and transfer learning with fake sentences. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017, vol. 10635, pp. 532–539. LNCS. Springer, Cham (2017). Scholar
  20. 20.
    Shirai, K., Sornlertlamvanich, V., Marukata, S.: Recurrent neural network with word embedding for complaint classification. In: Proceedings of the Third International Workshop on Worldwide Language Service Infrastructure and Second Workshop on Open Infrastructures and Analysis Frameworks for Human Language Technologies, pp. 36–43 (2016)Google Scholar
  21. 21.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  22. 22.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Kinga, D., Adam, J.B.: A method for stochastic optimization. In: International Conference on Learning Representations (2015)Google Scholar
  24. 24.
    Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Beijing Key Laboratory of Intelligent Telecommunications Software and MultimediaBeijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations