Abstract
An audiovisual automatic speech recognition (ASR) system is described. It is composed of two independent subsystems, both of which are complete speech recognition systems, plus a final integration stage. One of the subsystems is an audio ASR system which uses well-known audio processing techniques. The other subsystem performs lipreading, using a VQ-related method and hidden Markov models. The integration stage uses a product rule. The system is tested with a 500 word vocabulary. It is demonstrated that the combined system has superior performance to either of the subsystems acting alone.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
K.-R Lee, Automatic Speech Recognition: the Development of the SPHINX System, Kluwer Academic Publishers, Boston, 1989.
K. Mase and A. Pentland, “Automatic lipreading by computer,” Proc. IEICE Image Understanding Symposium, pp. 65–70, Apr. 1989.
E. D. Petajan, “Automatic lipreading to enhance speech recognition,” Proc. IEEE Global Telecom. Conf., pp. 265–272, Nov. 1984.
D. G. Stork, G. Wolff, and E. Levine, “Neural network lipreading system for improved speech recognition,” in Intl. Joint Conf. on Neural Networks, pp. 285–295, 1992.
H. Hermansky, “Perceptual linear predictive (PLP) analysis of speech,” J. Acoust. Soc. Am., vol. 87, no. 4, pp. 1738–1752, Apr. 1990.
P. L. Silsbee, Computer Lipreading for Improved Accuracy in Automatic Speech Recognition, PhD thesis, University of Texas, 1993.
D. W. Massaro, “Speech perception by ear and eye,” in B. Dodd and R. Campbell, eds., Hearing by Eye: the Psychology of Lip-reading, pp. 53–83. Lawrence Erlbaum Associates, London, 1987.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Silsbee, P.L., Bovik, A.C. (1995). Medium Vocabulary Audiovisual Speech Recognition. In: Ayuso, A.J.R., Soler, J.M.L. (eds) Speech Recognition and Coding. NATO ASI Series, vol 147. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-57745-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-57745-1_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-63344-7
Online ISBN: 978-3-642-57745-1
eBook Packages: Springer Book Archive