Abstract
This paper describes a novel method for hands-free speech recognition and in particular for simultaneous recognition of distant-talking speech of multiple sound sources (talkers or noise sources). Our method is based on the 3-D Viterbi search extended to a 3-D N-best search method to allow simultaneous speech recognition of multiple talkers. The baseline system integrates two existing technologies—3-D Viterbi search and conventional N-best search—into a complete system. However, initial evaluation of the 3-D N-best search-based system showed that new ideas were needed in order to build a system to simultaneously recognize multiple sound sources. Two factors were found to have an important role in system performance. Those two factors are the different likelihood ranges of the talkers and the direction-based separation of the hypotheses. More specifically, since we have to compare hypotheses originating from different talkers, an accurate comparison of these hypotheses cannot be made due to the different likelihood dynamic range of the talkers. Moreover, the hypotheses originated from talkers are located in different directions and therefore separating them based on their direction provides an efficient method for accurate recognition. To solve these problems, we implemented a likelihood normalization technique and a path distance-based clustering technique into the baseline 3-D N-best search-based system. The performance of our system was evaluated by experiments for recognizing the distant-talking speech of two talkers. The experiments were carried out on simulated (with only time delay) data and on reverberated (simulated and real) data. In this paper, we evaluated the proposed method in reverberant environments, and we introduced results obtained by experiments at several reverberation times and results obtained in a real environment. The experiments showed that implementing the two techniques described above produced significant improvements. Best results for simulated data were obtained by implementing the two techniques and using a microphone array composed of 32 channels. In that case in particular, the Simultaneous Word Accuracy (where both talkers are correctly recognized simultaneously) in the ‘top l’ hypothesis was 72.49%, and in the ‘top 3’ hypotheses was 86.25%, which were very promising results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
S. Nakamura, T. Yamada,T. Takiguchi, and K. Shikano, “Hands-Free Speech Recognition by a Microphone Array and HMM Composition,” in Proc. of International Workshop on Human Interface Technology,1995, pp. 33–38
M. Inoue, S. Nakamura, T. Yamada, and K. Shikano, “Microphone Array Design Measures for Hands-Free Speech Recognition,” in Proc. of European Conference on Speech Communication and Technology, 1997, pp. 331–334.
S. Nakamura, T. Yamada, R. Heracleous, and K. Shikano, “Recognition of Distant-Talking Speech Based on 3-D Trellis Search Using a Microphone Array and Adaptive Beamforming, in Proc. of Workshop on Robust Methods for Speech Recognition, 1999, pp. 219–222.
T. Takiguchi, S. Nakamura, and K. Shikano, “Speech Recognition for a Distant Moving Speaker Based on HMM Composition and Separation,” in Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing,2000, pp. 1403–1406.
G.W. Elko, Superdirectional Microphone Arrays, Acoustic Signal Processing for Telecommunications, Kluwer Academic Publishers, 2000.
D.H. Johnson and D.E. Dudgeon, Array Signal Processing, Concepts and Techniques PTR Prentice-Hall, Inc., 1993.
J.L. Flanagan, D.A. Berkley, G.W. Elko, J.E. West, and M.M. Sondhi, “Autodirective Microphone Systems,” Acoustica vol. 75, 1991.
M. Omologo and P. Svaizer, “Talker Localization and Speech Recognition Using a Microphone Array and a Cross-Power Spectrum Phase Analysis”, Pro-c. ICSLP,1994, pp. 1243–1246.
M. Omologo and P. Svaizer, “Acoustic Source Location in Noisy and Reverberant Environment Using CSP Analysis,” Proc. ICASSP, 1996, pp. 921–924.
T. Yamada, S. Nakamura, and K. Shikano, “Robust Speech Recognition with Speaker Localization by a Microphone Array,” Proc. ICSLP, 1996, pp. 1317–1320.
P. Svaizer, M. Matassoni, and M. Omologo, “Acoustic Source Location in a Three-Dimensional Space Using Crosspower Spectrum Phase,” Proc. ICASSP, 1997, pp. 231–234.
T. Hughes, H. Kim, J. DiBiase, and H. Silverman, “Using a Real Time, Tracking Microphone Array as Input to an HMM Speech Recognizer,” Proc. ICASSP, 1998, pp. 249–252.
T. Nishiura, T. Yamada, S. Nakamura, and K. Shikano, “Localization of Multiple Sound Sources Based on a CSP Analysis with a Microphone Array,” Proc. ICASSP, 2000, pp. 1053–1056.
T. Yamada, S. Nakamura, K. Shikano, “Hands-free Speech Recognition Based on 3-D Viterbi Search Using a Microphone Array,” Proc. ICASSP, 1998, pp. 245–248.
T. Yamada, S. Nakamura, and K. Shikano, “An Effect of Adaptive Beamforming on Hands-free Speech Recognition Based on 3-D Viterbi Search,” Proc. ICSLP, 1998, pp. 381–384.
P. Heracleous, T. Yamada, S. Nakamura, and K. Shikano, “Simultaneous Recognition of Multiple Sound Sources based on 3-D N-best Search using Microphone Array,” Proc. Eurospeech99, 1999, pp. 69–72.
P. Heracleous, S. Nakamura, and K. Shikano, “Multiple Sound Sources Recognition by a Microphone Array-based 3-D N-best Search with Likelihood Normalization,” in Proc. International Workshop on Hands-free Speech Communication, 2001, pp. 103–107.
T. Matsui and S. Fumi, “Likelihood Normalization for Speaker Verification using a Phoneme-and Speaker-independent Model,” Speech Communication, vol. 17, 1995, pp. 109–116.
J.B. Allen and D.A. Berkley. Image Method for Efficiently Simulating Small-Room Acoustics. Journal of Acoustical Society of America, vol. 65, no 4, 1979, pp. 943–950.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer Science+Business Media New York
About this chapter
Cite this chapter
Heracleous, P., Nakamura, S., Shikano, K. (2004). Simultaneous Recognition of Distant-Talking Speech of Multiple Talkers Based on the 3-D N-Best Search Method. In: Wang, JF., Furui, S., Juang, BH. (eds) Real World Speech Processing. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-6363-8_3
Download citation
DOI: https://doi.org/10.1007/978-1-4757-6363-8_3
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4419-5439-8
Online ISBN: 978-1-4757-6363-8
eBook Packages: Springer Book Archive