Abstract
This paper proposed improved automatic scoring methods for L2 English speaking tests based on acoustic models with lattice-free Maximum Mutual Information (MMI). Deep Neural Network (DNN) acoustic modeling with lattice-free MMI is the state-of-the-art technology in speech recognition because of its effectiveness in sequential discriminative training. Novel Goodness of Pronunciation (GOP) implementations based on lattice free MMI were proposed to improve the performance of automatic scoring for L2 English speech tests. Sequential acoustic weights during forced-alignment and posteriors based on Forward-Backward Algorithm with lattice free MMI acoustic models were used to improved GOP based automatic scoring. Experimental results show that our proposed lattice free MMI based methods outperform conventional regular DNN based automatic scoring methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tsubota, Y., et al.: Practical use of english pronunciation system for Japanese students in the CALL classroom. In: Proceedings of ICSLP 2004, pp. 1689–1692 (2004)
Zhang, et al.: Generalized segment posterior probability for automatic Mandarin pronunciation evaluation. In: Proceedings of the ICASSP, pp. 201–204 (2007)
Neri, A., et al.: Automatic Speech Recognition for second language learning: How and why it actually works. In: Proceedings of International Congresses of Phonetic Sciences, pp. 1157–1160 (2003)
Cardenoso-Payo, V., et al.: Assessment of Non-native Prosody for Spanish as L2 using quantitative scores. In: Proceedings of LREC, pp. 3967–3972 (2014)
Luo, D., et al.: Automatic pronunciation evaluation of lan-guage learners’ utterances generated through shadowing. In: Proceedings of the INTERSPEECH, pp. 2807–2810 (2008)
Dahl, G.E.,. et al.: Large-vocabulary continuous speech recognition with context-dependent DBN-HMMs. In: Proceedings of the ICASSP (2011)
Hu, W., et al.: A new DNN-based high quality pronunciation evaluation for computer-aided language learning (CALL). In: Proceedings of the INTERSPEECH 2012, pp. 1886–1890 (2012)
Luo, D., et al.: Improvement of segmental mispronunciation detection with prior knowledge extracted from large L2 speech corpus. In: Proceedings of the INTERSPEECH 2011, pp. 1593–1596 (2011)
Luo, D., et al.: Naturalness judgement of L2 english through dubbing practice. In: Proceedings of the INTERSPEECH (2016)
Luo, D., et al.: Factorized deep neural network adaptation for automatic scoring of L2 speech in english speaking tests. In: Proceedings of INTERSPEECH 2018, pp. 1656–1660 (2018)
Witt, S.M., Young, S.J.: Phone-level pronunciation scoring and assessment for interactive language learning. Speech Commun. 30(2–3), 95–108 (2000)
Dahl, G.E., et al.: Context-Dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEE Trans. Audio, Speech Lang. Process. 20(1), 30–42 (2012)
Hadian, H., Sameti, H., Povey, D., et al.: End-to-end speech recognition using lattice-free MMI. In:. Conference of the International Speech Communication Association, pp. 12–16 (2018)
Povey, D., et al.: The Kaldi speech recognition toolkit. In: Proceedings of the ASRU (2011)
Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: an ASR corpus based on public domain audio books. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5206–5210 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Luo, D., Guan, M., Xia, L. (2021). Automatic Scoring of L2 English Speech Based on DNN Acoustic Models with Lattice-Free MMI. In: Guan, M., Na, Z. (eds) Machine Learning and Intelligent Communications. MLICOM 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 342. Springer, Cham. https://doi.org/10.1007/978-3-030-66785-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-66785-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66784-9
Online ISBN: 978-3-030-66785-6
eBook Packages: Computer ScienceComputer Science (R0)