Skip to main content
SpringerLink
Log in

A Knowledge-Gated Mechanism for Utterance Domain Classification

  • Conference paper
  • First Online:
Natural Language Processing and Chinese Computing (NLPCC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11839))

Abstract

Utterance domain classification (UDC) is a critical pre-processing step for many speech understanding and dialogue systems. Recently neural models have shown promising results on text classification. Meanwhile, the background information and knowledge beyond the utterance plays crucial roles in utterance comprehension. However, some improper background information and knowledge are easily introduced due to the ambiguity of entities or the noise in knowledge bases (KBs), UDC task remains a great challenge. To address this issue, this paper proposes a knowledge-gated (K-Gated) mechanism that leverages domain knowledge from external sources to control the path through which information flows in the neural network. We employ it with pre-trained token embedding from Bidirectional Encoder Representation from Transformers (BERT) into a wide spectrum of state-of-the-art neural text classification models. Experiments on the SMP-ECDT benchmark corpus show that the proposed method achieves a strong and robust performance regardless of the quality of the encoder models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://shuyantech.com/api/entitylinking/cutsegment.

  2. 2.

    https://github.com/hanxiao/bert-as-service.

  3. 3.

    https://github.com/fxsjy/jieba.

References

  1. Sarikaya, R.: The technology behind personal digital assistants: an overview of the system architecture and key components. IEEE Signal Process. Mag. 34(1), 67–81 (2017)

    Article  Google Scholar 

  2. Yu, K., Chen, R., Chen, B., et al.: Cognitive technology in task-oriented dialogue systems - concepts, advances and future. Chin. J. Comput. 38(12), 2333–2348 (2015). (in Chinese)

    MathSciNet  Google Scholar 

  3. Kim, Y., Kim, D., Kumar, A.: Efficient large-scale neural domain classification with personalized attention. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 2214–2224 (2018)

    Google Scholar 

  4. Tür, G., Mori, R.: Spoken Language Understanding: Systems for Extracting Semantic Information from Speech. Wiley, Chichester (2011)

    Book  Google Scholar 

  5. Xu, P., Sarikaya, R.: Contextual domain classification in spoken language understanding systems using recurrent neural network. In: Proceedings of the 39th International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014), pp. 136–140 (2014)

    Google Scholar 

  6. Ke, Z., Huang, P., Zeng, Z.: Domain classification based on undefined utterances detection optimization. J. Chin. Inf. Process. 32(4), 105–113 (2018). (in Chinese)

    Google Scholar 

  7. Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. In: Proceedings of the 17th Annual Conference of the International Speech Communication Association (INTERSPEECH 2016), pp. 685–689 (2016)

    Google Scholar 

  8. Devlin, J., Chang, M., Lee, K., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), pp. 4171–4186 (2019)

    Google Scholar 

  9. Deng, Y., Shen, Y., Yang, M., et al.: Knowledge as a bridge: improving cross-domain answer selection with external knowledge. In: Proceedings of the 27th International Conference on Computational Linguistics (COLING 2018), pp. 3295–3305 (2018)

    Google Scholar 

  10. Dauphin, Y.N., Fan, A., Auli, M., et al.: Language modeling with gated convolutional networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML 2017), pp. 933–941 (2017)

    Google Scholar 

  11. Shi, C., Liu, S., Ren, S., et al.: Knowledge-based semantic embedding for machine translation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016), pp. 2245–2254 (2016)

    Google Scholar 

  12. Wang, J., Wang, Z., Zhang, D., Yan, J.: Combining knowledge with deep convolutional neural networks for short text classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI 2017), pp. 2915–2921 (2017)

    Google Scholar 

  13. Oord, A., Kalchbrenner, N., Espeholt, L., et al.: Conditional image generation with PixelCNN decoders. In: Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), pp. 4790–4798 (2016)

    Google Scholar 

  14. Xue, W., Li, T.: Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), pp. 2514–2523 (2018)

    Google Scholar 

  15. Heck, L., Tür, D., Tür, G.: Leveraging knowledge graphs for web-scale unsupervised semantic parsing. In: Proceedings of the 14th Annual Conference of the International Speech Communication Association (INTERSPEECH 2013), pp. 1594–1598 (2013)

    Google Scholar 

  16. Sarikaya, R., Hinton, G., Ramabhadran, B.: Deep belief nets for natural language call-routing. In: Proceedings of the 36th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011), pp. 5680–5683 (2011)

    Google Scholar 

  17. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 1292–1302 (2014)

    Google Scholar 

  18. Ravuri, S., Stolcke, S.: A comparative study of recurrent neural network models for lexical domain classification. In: Proceedings of the 41st IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2016), pp. 6075–6079 (2016)

    Google Scholar 

  19. Xiao, Y., Cho, K.: Efficient character-level document classification by combining convolution and recurrent layers. Computing Research Repository, arXiv:1602.00367. Version 1 (2016)

  20. Cheng, J., Dong, L., Lapata, M.: Long short-term memory-networks for machine reading. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP 2016), pp. 551–561 (2016)

    Google Scholar 

  21. Vu, N.T., Gupta, P., Adel, H., et al.: Bi-directional recurrent neural network with ranking loss for spoken language understanding. In: Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2016), pp. 6060–6064 (2016)

    Google Scholar 

  22. Yang, Z., Yang, D., Dyer, C., et al.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2016), pp. 1480–1489 (2016)

    Google Scholar 

  23. Chen, Y., Tur, D., Tür, G., et al.: Syntax or semantics? Knowledge-guided joint semantic frame parsing. In: Proceedings of 2016 IEEE Spoken Language Technology Workshop (SLT 2016), pp. 348–355 (2016)

    Google Scholar 

  24. Chen, J., Wang, A., Chen, J., et al.: CN-Probase: a data-driven approach for large-scale Chinese taxonomy construction. In: Proceedings of the 35th IEEE International Conference on Data Engineering (ICDE 2019) (2019)

    Google Scholar 

  25. Moro, A., Raganato, A., Navigli, R.: Entity linking meets word sense disambiguation: a unified approach. Trans. Assoc. Comput. Linguist. 2, 231–244 (2013)

    Article  Google Scholar 

  26. Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Proceedings of the 41st Annual Conference on Neural Information Processing Systems (NIPS 2017), pp. 6000–6010 (2017)

    Google Scholar 

  27. Boureau, Y., Ponce, J., LeCun, Y.: A theoretical analysis of feature pooling in visual recognition. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 111–118 (2010)

    Google Scholar 

  28. Zhang, W., Chen, Z., Che, W., et al.: The first evaluation of Chinese human-computer dialogue technology. Computing Research Repository, arXiv:1709.10217. Version 1 (2017)

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 71472068), National Innovation Training Project for College Students of China (No. 201710564154), and Innovation Training Project for College Students of Guangdong Province (No. 201810564094). We also thank the SCIR Lab of Harbin Institute of Technology and the iFLYTEK Co. Ltd. for providing the SMP-ECDT benchmark corpus.

Author information

Authors and Affiliations

  1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou, China

    Zefeng Du, Peijie Huang, Yuhong He, Wei Liu & Jiankai Zhu

Authors
  1. Zefeng Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peijie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiankai Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peijie Huang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Jie Tang

  2. National University of Singapore, Singapore, Singapore

    Min-Yen Kan

  3. Peking University, Beijing, China

    Dongyan Zhao

  4. Peking University, Beijing, China

    Sujian Li

  5. Zhengzhou University, Zhengzhou, China

    Hongying Zan

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, Z., Huang, P., He, Y., Liu, W., Zhu, J. (2019). A Knowledge-Gated Mechanism for Utterance Domain Classification. In: Tang, J., Kan, MY., Zhao, D., Li, S., Zan, H. (eds) Natural Language Processing and Chinese Computing. NLPCC 2019. Lecture Notes in Computer Science(), vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-32236-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32235-9

  • Online ISBN: 978-3-030-32236-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation