The Rich Transcription 2005 Spring Meeting Recognition Evaluation
- 1.4k Downloads
This paper presents the design and results of the Rich Transcription Spring 2005 (RT-05S) Meeting Recognition Evaluation. This evaluation is the third in a series of community-wide evaluations of language technologies in the meeting domain. For 2005, four evaluation tasks were supported. These included a speech-to-text (STT) transcription task and three diarization tasks: “Who Spoke When”, “Speech Activity Detection”, and “Source Localization.” The latter two were first-time experimental proof-of-concept tasks and were treated as “dry runs”. For the STT task, the lowest word error rate for the multiple distant microphone condition was 30.0% which represented an impressive 33% relative reduction from the best result obtained in the last such evaluation – the Rich Transcription Spring 2004 Meeting Recognition Evaluation. For the diarization “Who Spoke When” task, the lowest diarization error rate was 18.56% which represented a 19% relative reduction from that of RT-04S.
KeywordsWord Error Rate Microphone Array Meeting Participant Linguistic Data Consortium Reference Word
Unable to display preview. Download preview PDF.
- 1.Fiscus, et al.: Results of the Fall 2004 STT and MDE Evaluation. In: RT-04F Evaluation Workshop Proceedings, November 7-10 (2004) Google Scholar
- 2.Garofolo, et al.: The Rich Transcription 2004 Spring Meeting Recognition Evaluation. In: ICASSP 2004 Meeting Recognition Workshop, May 17 (2004)Google Scholar
- 3.Spring 2005 (RT-05S) Rich Transcription Meeting Recognition Evaluation Plan (2005), http://www.nist.gov/speech/tests/rt/rt2005/spring/rt05s-meeting-eval-plan-V1.pdf
- 4.Speaker Localization and Tracking – Evaluation Criteria, http://www.nist.gov/speech/tests/rt/t2005/spring/sloc/CHIL-IRST_SpeakerLocEval-V5.0-2005-01-18.pdf
- 5.LDC Meeting Recording Transcription, http://www.ldc.upenn.edu/Projects/Transcription/NISTMeet
- 6.SCTK toolkit, http://www.nist.gov/speech/tools/index.htm
- 7.Janin, A., Ang, J., Bhagat, S., Dhillon, R., Edwards, J., Macias-Guarasa, J., Morgan, N., Peskin, B., Shriberg, E., Stolcke, A., Wooters, C., Wrede, B.: The ICSI Meeting Project: Resources and Research. In: NIST ICASSP 2004 Meeting Recognition Workshop, Montreal (2004)Google Scholar
- 8.Garofolo, J.S., Laprun, C.D., Michel, M., Stanford, V.M., Tabassi, E.: The NIST Meeting Room Pilot Corpus. In: LREC 2004 (2004)Google Scholar
- 9.The ISL Meeting Corpus: The Impact of Meeting Type on Speech Style, Susanne Burger, Victoria MacLaren, Hua Yu, ICSLP 2002 (2002) Google Scholar
- 10.Huang, Z., Harper, M.P.: Speech Activity Detection on Multichannels of Meeting Recordings. In: Proceedings from the RT 2005 Workshop at MLML 2005 (2005)Google Scholar