Abstract
Spoken language understanding (SLU) topic has seen a lot of progress these last three years, with the emergence of end-to-end neural approaches. Spoken language understanding refers to natural language processing tasks related to semantic extraction from speech signal, like named entity recognition from speech or slot filling task in a context of human-machine dialogue. Classically, SLU tasks were processed through a cascade approach that consists in applying, firstly, an automatic speech recognition process, followed by a natural language processing module applied to the automatic transcriptions. These three last years, end-to-end neural approaches, based on deep neural networks, have been proposed in order to directly extract the semantics from speech signal, by using a single neural model. More recent works on self-supervised training with unlabeled data open new perspectives in term of performance for automatic speech recognition and natural language processing. In this paper, we present a brief overview of the recent advances on the French MEDIA benchmark dataset for SLU, with or without the use of additional data. We also present our last results that significantly outperform the current state-of-the-art with a Concept Error Rate (CER) of 11.2%, instead of 13.6% for the last state-of-the-art system presented this year.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amodei, D., et al.: Deep speech 2: end-to-end speech recognition in English and mandarin. In: International Conference on Machine Learning, pp. 173–182. PMLR (2016)
Baevski, A., Zhou, H., Mohamed, A., Auli, M.: wav2vec 2.0: a framework for self-supervised learning of speech representations. arXiv preprint arXiv:2006.11477 (2020)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Béchet, F., Raymond, C.: Benchmarking benchmarks: introducing new automatic indicators for benchmarking spoken language understanding corpora. In: Interspeech, Graz, Austria (2019)
Bonneau-Maynard, H., Rosset, S., Ayache, C., Kuhn, A., Mostefa, D.: Semantic annotation of the French media dialog corpus. In: INTERSPEECH (2005)
Caubrière, A., Tomashenko, N., Laurent, A., Morin, E., Camelin, N., Estève, Y.: Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability. In: Proceedings Interspeech 2019, pp. 1198–1202 (2019). https://doi.org/10.21437/Interspeech.2019-1832
Chan, W., Jaitly, N., Le, Q., Vinyals, O.: Listen, attend and spell: a neural network for large vocabulary conversational speech recognition. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4960–4964. IEEE (2016)
Chiu, C.C., et al.: State-of-the-art speech recognition with sequence-to-sequence models. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4774–4778. IEEE (2018)
Chorowski, J., Bahdanau, D., Cho, K., Bengio, Y.: End-to-end continuous speech recognition using attention-based recurrent NN: first results. arXiv preprint arXiv:1412.1602 (2014)
Chung, Y.A., Zhu, C., Zeng, M.: SPLAT: speech-language joint pre-training for spoken language understanding. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1897–1907 (2021)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190 (2018)
Dahl, D.A., et al.: Expanding the scope of the ATIS task: The ATIS-3 corpus. In: Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, 8–11 March 1994 (1994)
De Mori, R.: Spoken language understanding: a survey. In: 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), pp. 365–376. IEEE (2007)
Desot, T., Portet, F., Vacher, M.: Towards end-to-end spoken intent recognition in smart home. In: 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 1–8. IEEE (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423
Evain, S., et al.: Lebenchmark: a reproducible framework for assessing self-supervised representation learning from speech. In: Interspeech, Brno, Czechia (2021)
Ghannay, S., et al.: End-to-end named entity and semantic concept extraction from speech. In: 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 692–699. IEEE (2018)
Ghannay, S., Servan, C., Rosset, S.: Neural networks approaches focused on French spoken language understanding: application to the MEDIA evaluation task. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 2722–2727. International Committee on Computational Linguistics, Barcelona, Spain (Online), December 2020. https://doi.org/10.18653/v1/2020.coling-main.245. https://www.aclweb.org/anthology/2020.coling-main.245
Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 369–376 (2006)
Hahn, S., et al.: Comparing stochastic approaches to spoken language understanding in multiple languages. IEEE Trans. Audio Speech Lang. Process. 19(6), 1569–1583 (2010)
Hakkani-Tü, D., Béchet, F., Riccardi, G., Tur, G.: Beyond ASR 1-best: using word confusion networks in spoken language understanding. Comput. Speech Lang. (2005). https://doi.org/10.1016/j.csl.2005.07.005. https://hal.archives-ouvertes.fr/hal-01314993
Jabaian, B., Lefèvre, F.: Error-corrective discriminative joint decoding of automatic spoken language transcription and understanding. In: Bimbot, F., et al. (eds.) INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, 25–29 August 2013, pp. 2718–2722. ISCA (2013). http://www.isca-speech.org/archive/interspeech_2013/i13_2718.html
Kudo, T., Richardson, J.: SentencePiece: a simple and language independent subword tokenizer and Detokenizer for neural text processing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 66–71. Association for Computational Linguistics, Brussels, Belgium, November 2018. https://doi.org/10.18653/v1/D18-2012. https://www.aclweb.org/anthology/D18-2012
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Martin, L., et al.: CamemBERT: a tasty French language model. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020)
Mesnil, G., He, X., Deng, L., Bengio, Y.: Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. In: Interspeech, pp. 3771–3775 (2013)
Pelloin, V., et al.: End2end acoustic to semantic transduction. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7448–7452 (2021). https://doi.org/10.1109/ICASSP39728.2021.9413581
Serdyuk, D., Wang, Y., Fuegen, C., Kumar, A., Liu, B., Bengio, Y.: Towards end-to-end spoken language understanding. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5754–5758. IEEE (2018)
Servan, C., Raymond, C., Béchet, F., Nocera, P.: Conceptual decoding from word lattices: application to the spoken dialogue corpus MEDIA. In: The Ninth International Conference on Spoken Language Processing (Interspeech 2006 - ICSLP), Pittsburgh, United States, September 2006. https://hal.archives-ouvertes.fr/hal-01160181
Shah, P., et al.: Building a conversational agent overnight with dialogue self-play. arXiv preprint arXiv:1801.04871 (2018)
Simonnet, E., Ghannay, S., Camelin, N., Estève, Y.: Simulating ASR errors for training SLU systems. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan, May 2018. https://www.aclweb.org/anthology/L18-1499
Simonnet, E., Ghannay, S., Camelin, N., Estève, Y., De Mori, R.: ASR error management for improving spoken language understanding. In: Interspeech 2017, Stockholm, Sweden, August 2017
Tur, G., De Mori, R.: Spoken Language Understanding: Systems for Extracting Semantic Information From Speech. Wiley (2011)
Vaswani, A., et al.: Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)
Acknowledgments
This work was granted access to the HPC resources of IDRIS under the allocation 2020-AD011011838 made by GENCI.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ghannay, S., Caubrière, A., Mdhaffar, S., Laperrière, G., Jabaian, B., Estève, Y. (2021). Where Are We in Semantic Concept Extraction for Spoken Language Understanding?. In: Karpov, A., Potapova, R. (eds) Speech and Computer. SPECOM 2021. Lecture Notes in Computer Science(), vol 12997. Springer, Cham. https://doi.org/10.1007/978-3-030-87802-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-87802-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87801-6
Online ISBN: 978-3-030-87802-3
eBook Packages: Computer ScienceComputer Science (R0)