Abstract
Modern speech recognition applications are becoming very complex program packages. To understand the error behaviour of the ASR systems, a special diagnosis–a procedure or a tool—is needed. Many ASR users and developers have developed their own expert diagnostic rules that can be successfully applied to a system. There are also several explicit approaches in the literature for determining the problems related to application errors. The approaches are based on error and ablative analyses of the ASR components, with a blame assignment to a problematic component. The disadvantage of those methods is that they are either quite time-consuming to acquire expert diagnostic knowledge, or that they offer very coarse-grained localization of a problematic ASR part. This paper proposes fine-grained diagnostics for debugging ASR by applying a program-spectra based failure localization, and it localizes directly a part of ASR implementation. We designed a toy experiment with diagnostic database OLLO to show that our method is very easy to use and that it provides a good localization accuracy. Because it is not able to localize all the errors, an issue that we discuss in the discussion, we recommend to use it with other coarse-grained localization methods for a complex ASR diagnosis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chase L. L.: Error-Responsive Feedback Mechanisms for Speech Recognizers. Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh (April 1997); Also available as Robotics Institute Tech. Report # CMU-RI-TR-97-18
Nanjo, H., Lee, A., Kawahara, T.: Automatic Diagnosis of Recognition Errors in LVCSR Systems. In: ICSLP 2000, vol. 2, pp. 1027–1030 (2000)
Cerňak, M.: A Comparison of Decision Tree Classifiers for Automatic Diagnosis of Speech Recognition Errors. Computing and Informatics 3 (July 2010)
Zheng, A.X., Lloyd, J., Brewer, E.: Failure Diagnosis using Decision Trees. In: Proceedings of the First International Conference on Autonomic Computing (ICAC 2004), Washington, DC, pp. 36–43. IEEE Computer Society Press, Los Alamitos (2004)
Picheny, M., Nahamoo, D.: Towards Superhuman Speech Recognition. In: Benesty, J., Sondhi, M.M., Huang, Y. (eds.) Springer Handbook of Speech Proccesing, pp. 597–616. Springer, Heidelberg (2008)
Zoeteweij, P., Abreu, R., van Gemund, A.J.C.: Software Fault Diagnosis. In: A Tutorial in TESTCOM/FATES, Tallinn, Estonia, June 26-29 (2007)
Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: On the Accuracy of Spectrum-based Fault Localization. In: Proceedings of TAIC PART (2007)
Steeneken, H.J.M., Varga, A.: Assessment for Automatic Speech Recognition. Comparison of Assessment Methods, Speech Communication 12(3), 241–246 (1993)
Wesker, T., Meyer, B., Wagener, J., Anemuller, J., Mertins, A., Kollmeier, B.: Oldenburg Logatome Speech Corpus (OLLO) for Speech Recognition Experiments with Humans and Machines. In: Interspeech, pp. 1273–1276 (2005)
Collobert, R., Bengio, S., Mariéthoz, J.: Torch: a Modular Machine Learning Software Library. Technical Report IDIAP-RR 02-46, IDIAP (2002)
Santelices, R., Jones, J.A., Yu, Y., Harrold, M.J.: Lightweight Fault-Localization Using Multiple Coverage Types. In: Proc. of the ICSE, pp. 56–66 (2009)
Vandewalle, P., Kovačević, J., Vetterli, M.: Reproducible Research in Signal Processing: What, Why, and How. IEEE Signal Processing Magazine (37) (May 2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cerňak, M. (2010). Diagnostics for Debugging Speech Recognition Systems. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech and Dialogue. TSD 2010. Lecture Notes in Computer Science(), vol 6231. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15760-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-15760-8_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15759-2
Online ISBN: 978-3-642-15760-8
eBook Packages: Computer ScienceComputer Science (R0)