Advertisement

Dialogue intent classification with character-CNN-BGRU networks

  • Yufan Wang
  • Jiawei Huang
  • Tingting HeEmail author
  • Xinhui Tu
Article
  • 20 Downloads

Abstract

Dialogue intent classification plays a significant role in human-computer interaction systems. In this paper, we present a hybrid convolutional neural network and bidirectional gated recurrent unit neural network (CNN-BGRU) architecture to classify the intent of a dialogue utterance. First, character embeddings are trained and used as the inputs of the proposed model. Second, a CNN is used to extract local features from each utterance, and a maximum pooling layer is applied to select the most crucial latent semantic factors. A bidirectional gated recurrent unit (BGRU) layer architecture is used to capture the contextual semantic information. Then, two feature maps, which are the outputs of the two architectures, are integrated into the final utterance representation. The proposed model can utilize local semantic and contextual information to recognize and classify the user dialogue intent in an efficient way. The proposed model is evaluated based on a social media processing (SMP) data set and a real conversational data set. The experimental results show that the proposed model outperforms the corresponding traditional methods. In addition, compared to the CNN and BGRU methods, the classification accuracy of the proposed model is 1.4% higher for the SMP data set.

Keywords

Dialogue intent classification CNN BGRU Character neural embeddings 

Notes

Acknowledgements

This research is supported by the Fundamental Research Funds for Central Universities (CCNU18JCK05), the National Natural Science Foundation of China (61532008), the National Science Foundation of China (61572223), and the National Key Research and Development Program of China (2017YFC0909502).

References

  1. 1.
    Ali SA, Sulaiman N, Mustapha A, Mustapha N (2009) Improving Accuracy of Intention-Based Response Classification using Decision Tree. Inf Technol J 8(6)CrossRefGoogle Scholar
  2. 2.
    Becerra A, Rosa JIDL, González E (2017) Speech recognition in a dialog system: from conventional to deep processing. Multimed Tools Appl 78(2):1–37Google Scholar
  3. 3.
    Celikyilmaz A, Hakkanitur D, Tur G, Fidler A, Hillard D (2011) Exploiting Distance Based Similarity in Topic Models for User Intent Detection. IEEE Automatic Speech Recognition & Understanding Workshop:425–430Google Scholar
  4. 4.
    Chen H, Liu X, Yin D, Tang J (2017) A Survey on Dialogue Systems: Recent Advances and New Frontiers. Acm Sigkdd Explorations Newsletter 19(2)CrossRefGoogle Scholar
  5. 5.
    Cho K, Van Merrienboer B, Bahdanau D, Bengio Y (2014) On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. Computer ScienceGoogle Scholar
  6. 6.
    Chung, J., Gulcehre, C., Cho, K.H., Bengio, Y (2014) Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. Eprint ArxivGoogle Scholar
  7. 7.
    Deep Learning for Natural Language Processing - Communication Between Neural Network Systems and Human [EB/OL]: Deep Learning for Natural Language Processing - Communication Between Neural Network Systems and Human [EB/OL]. http://www.noahlab.com.hk/topics/DeepLearning4NLP.
  8. 8.
    Ding, X., Liu, T., Duan, J., Nie, J.Y.: Mining User Consumption Intention from Social Media Using Domain Adaptive Convolutional Neural Network (2015)Google Scholar
  9. 9.
    Er MJ, Zhang Y, Wang N, Pratama M (2016) Attention pooling-based convolutional neural network for sentence modelling. Inf Sci 373:388–403.  https://doi.org/10.1016/j.ins.2016.08.084 CrossRefGoogle Scholar
  10. 10.
    Gallicchio C (2018) Short-term Memory of Deep RNNGoogle Scholar
  11. 11.
    Graves A, Jaitly N, Mohamed AR (2014) Hybrid speech recognition with Deep Bidirectional LSTMAutomatic Speech Recognition and Understanding, 2014273-278Google Scholar
  12. 12.
    H., Y.L.Y.B (2015) Deep learning. Nature.Google Scholar
  13. 13.
    Hashem IAT, Yaqoob I, Anuar NB, Mokhtar S, Gani A, Ullah Khan S (2015) The rise of "big data" on cloud computing: Review and open research issues. Inf Syst 47(0):98–115.  https://doi.org/10.1016/j.is.2014.07.006 CrossRefGoogle Scholar
  14. 14.
    Jingxue Liu FMYZ (2017) Character-Level neural networks for short text classification. Paper presented at the International Smart Cities ConferenceGoogle Scholar
  15. 15.
    Karahan M, Hakkani-Tur D, Riccardi G, Tur G (2003) Combining classifiers for spoken language understandingAutomatic Speech Recognition and Understanding, 2003. ASRU ‘03. 2003 IEEE Workshop on, 2003589-594Google Scholar
  16. 16.
    Keizer S (2001) Dialogue act modelling using Bayesian networksGoogle Scholar
  17. 17.
    Kim Y (2014) Convolutional Neural Networks for Sentence Classification. Eprint ArxivGoogle Scholar
  18. 18.
    Lai S, Xu L, Liu K, Zhao J (2015) Recurrent Convolutional Neural Networks for Text ClassificationGoogle Scholar
  19. 19.
    Lian Meng MH (2017) Dialogue Intent Classification with Long Short-Term Memory Networks. Paper presented at the The Sixth Conference on Natural Language Processing and Chinese Computing (NLPCC), Dalian, ChinaGoogle Scholar
  20. 20.
    Liu T, F.B.C.Y (2015) Detecting consumption intention based on graph ranking in social media. Sci Sin InformGoogle Scholar
  21. 21.
    Liu B, Lane I (2016) Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot FillingGoogle Scholar
  22. 22.
    Luong T, Socher R, Manning CD (2013) Better Word Representations with Recursive Neural Networks for Morphology Conference, 2013104-113Google Scholar
  23. 23.
    Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic modelsGoogle Scholar
  24. 24.
    Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed Representations of Words and Phrases and their Compositionality. Adv Neural Inf Proces Syst 26:3111–3119Google Scholar
  25. 25.
    Niimi Y, Oku T, Nishimoto T, Araki M (2001) A rule based approach to extraction of topics and dialog acts in a spoken dialog system Euro speech 2001 Scandinavia, European Conference on Speech Communication and Technology, Interspeech Event, Aalborg, 20012185-2188Google Scholar
  26. 26.
    Pearson correlation coefficient: Pearson correlation coefficient. https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
  27. 27.
    Qian Y, X.D.T.L. (2017) Identification method of user’s travel consumption intention in chatting robot. Sci Sin InformGoogle Scholar
  28. 28.
    Ravuri S, Stoicke A (2016) A comparative study of neural network models for lexical intent classification Automatic Speech Recognition and Understanding, 2016368-374Google Scholar
  29. 29.
    Socher R, Bauer J, Manning CD, Ng AY (2013) Parsing with Compositional Vector GrammarsMeeting of the Association for Computational Linguistics, 2013455-465Google Scholar
  30. 30.
    Social Media Processing Homepage: Social Media Processing Homepage. http://www.cips-smp.org/smp2017/
  31. 31.
    Surendran D, Levow GA (2006) Dialog act tagging with support vector machines and hidden markov modelsIn Proceedings of Interspeech/ICSLP, 20061-28Google Scholar
  32. 32.
    Wang J, Wang Z, Zhang D, Yan J (2017) Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification Twenty-Sixth International Joint Conference on Artificial Intelligence, 20172915-2921Google Scholar
  33. 33.
    Word2vec Homepage (2014) Word2vec Homepage. http://code.google.com/archive/p/word2vec/
  34. 34.
    Xiao, Y., Cho, K (2016) Efficient Character-level Document Classification by Combining Convolution and Recurrent LayersGoogle Scholar
  35. 35.
    Xu X, Li W, Ran Q, Du Q, Gao L, Zhang B (2018) Multisource Remote Sensing Data Classification Based on Convolutional Neural Network. IEEE Transactions on Geoscience & Remote Sensing PP(99):1–13Google Scholar
  36. 36.
    Yao K, Peng B, Zhang Y, Yu D, Zweig G, Shi Y (2014) Spoken Language Understanding using Long Short-Term Memory Neural Networks. IEEE – Institute of Electrical & Electronics Engineers:189–194Google Scholar
  37. 37.
    Yin J, J. X. L. Z.: Neural Generative Question Answering. Paper presented at the International Joint Conference on Artificial IntelligenceGoogle Scholar
  38. 38.
    Zhang Y, Marshall I, Wallace BC (2016) Rationale-Augmented Convolutional Neural Networks for Text Classification, 2016795Google Scholar
  39. 39.
    Zhang X, Zhao J, Lecun Y (2015) Character-level Convolutional Networks for Text Classification, 649-657Google Scholar
  40. 40.
    Zheng X, Chen H, Xu T (2013) Deep learning for Chinese word segmentation and POS taggingConference on Empirical Methods in Natural Language Processing, 2013Google Scholar
  41. 41.
    Zhou C, Sun C, Liu Z, Lau FCM (2015) A C-LSTM Neural Network for Text Classification. Comput Therm Sci 1(4):39–44Google Scholar
  42. 42.
    Zhou C, Sun C, Liu Z, Lau FCM (2015) A C-LSTM Neural Network for Text ClassificationGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yufan Wang
    • 1
  • Jiawei Huang
    • 1
  • Tingting He
    • 1
    Email author
  • Xinhui Tu
    • 1
  1. 1.School of Computer ScienceCentral China Normal UniversityWuhanChina

Personalised recommendations