Topic-Specific Language Model Based on Graph Spectral Approach for Speech Recognition

  • Shinya Takahashi
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 6)

Large vocabulary continuous speech recognition techniques have greatly advanced in recent years due to the remarkable advances of computers. Even personal computers today have extraordinary computation powers so that we can perform automatic speech recognition with high performance in a small computer. This is due to not only the evolution of the computers but also the development of some efficient recognition algorithms and the utilization of statistical acoustic and language models with a large speech database.

In addition, rapid development of the WWW makes it possible to utilize enormous textual data resources for creating excellent language models. Especially, the topic-specific language model can give high performance for speech recognition if the large amount of appropriate topic-related documents can be collected.

Under these circumstances, we have been developing the broadcast news search system with the language model adaptation using the information on the WWW. The basic idea is that broadcast news has similar Web documents on the Internet news site, so the performance of news speech recognition can be improved with the adapted language model by collecting a similar article via Web crawling [1, 2].


Speech Recognition Spectral Cluster Recognition Result Index Term Broadcast News 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Takai, D., Morimoto, T., Takahashi, S., “Extraction of index terms for retrieving multimedia news documents from World Wide Web (in Japanese),” Proceedings of the 56th JCEEE Kyushu, Kumamoto, Japan, August 2003Google Scholar
  2. 2.
    Takhashi, S., Morimoto, T., Irie, Y., “Adaptation of language model with iterative web crawling for speech recognition of broadcast news (in Japanese),” Proceedings of FIT2006, Fukuoka, Japan, pp. 381–384, September 2006Google Scholar
  3. 3.
    Zhu, X., Rosenfield, R., “Improving trigram language modeling with the World Wide Web,” Proceedings of ICASSP’01, Salt Lake City, UUT, May 2001Google Scholar
  4. 4.
    Berger, A., Miller, R., “Just-in-time language modeling,” Proceedings of ICASSP’98, Seattle, pp. 705–708, December 1998Google Scholar
  5. 5.
    Bulyko, I., Ostendorf, M., Stolcke, A., “Getting more mileage from Web text sources for conversational speech language modeling using class-dependent mixtures,” Proceedings of HLT-ACL, Edmonton Canada, pp. 7–9, May 2003Google Scholar
  6. 6.
    Nishimura, R., et al., “Automatic N-gram language model creation from Web resources,” Proceedings of EUROSPEECH-2001, Aolbarg, Denmark, pp. 2127–2130, September 2001Google Scholar
  7. 7.
    Sethy, A., Georgiou, P.G., Narayanan, S., “Building topic specific language models from webdata using competitive models,” Proceedings of INTERSPEECH’06, Lisboa, Portugal, pp. 1293–1296, September 2005Google Scholar
  8. 8.
    Suzuki, M., Kajiura, Y., Ito, A., Makino, S., “Unsupervised language model adaptation based on automatic text collection from WWW,” Proceedings of INTERSPEECH’06, Pittsburgh, pp. 2202–2205, September 2006Google Scholar
  9. 9.
  10. 10.
    Salton, G., et al., “A vector space model for automatic indexing,” Communications of the ACM, V18, N11, pp. 613-620, 1975. Reprinted in Readings in Information Retrieval, Jones, K.S. and Willett, P. (Eds.), Morgan Kaufmann, San Mateo, CA, pp. 273–280, 1997MathSciNetGoogle Scholar
  11. 11.
    Nagatomo, K., et al., “Complemental back-off algorithm for merging language models (in Japanese),” IPSJ Journal, V43, N9, pp. 2884–2893, September 2002Google Scholar
  12. 12.
    Pothen, A., Simon, H., Liou, K., “Partitioning sparse matrices with eigenvecotors of graphs,” SIAM Journal on Matrix Analysis, V11, N3, pp. 430–452, 1990CrossRefMathSciNetGoogle Scholar
  13. 13.
    Dhillon, I.S., “Co-clustering documents and words using bipartite spectral graph partitioning,” ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, pp. 269–274, August 2001Google Scholar
  14. 14.
    Tsuruta, N., Aly, S.K.H., Maeda, S., Takahashi, S., Morimoto, T., “Self-organizing map vs. spectral clustering on visual feature extraction for human interface,” Proceedings of International Forum on Strategic Technology (IFOST) 2006, Ulsan, Korea, pp. 55–58, October 2006Google Scholar
  15. 15.
    Ding, C., He, X., “Linearized cluster assignment via spectral ordering,” Proceedings of ACM International Conference on Machine Learning, Banff, Canada, pp. 30–37, July 2004Google Scholar
  16. 16.
    Clarkson, P.R., Rosenfeld, R., “Statistical language modeling using the CMU-Cambridge toolkit,” Proceedings of ESCA Eurospeech, Rhodes, Greek, pp. 2707–2710, September 1997Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Shinya Takahashi
    • 1
  1. 1.Electronics and Computer Science DepartmentFukuoka UniversityFukuokaJapan

Personalised recommendations