Abstract
Systematic evaluation is essential to drive the rapid progress of a broad range of audiovisual perceptual technologies. Within the CHIL project, such evaluations were undertaken on an annual basis, so that improvements could be measured objectively, and different approaches compared and assessed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
The AGTK Annotation Tool, http://agtk.sourceforge.net.
AMI – Augmented Multiparty Interaction, http://www.amiproject.org.
S. Burger. The CHIL RT07 evaluation data. In Multimodal Technologies for Perception of Humans, Proceedings of the International Evaluation Workshops CLEAR 2007 and RT 2007, LNCS 4625, Baltimore, MD, 2007.
S. Burger, V. McLaren, and H. Yu. The ISL meeting corpus: The impact of meeting type on speech style. In Proceedings of the International Conference on Spoken Language Processing, Denver, CO, 2002.
Classification of Events, Activities, and Relationships Evaluation and Workshop: http://www.clear-evaluation.org.
The CLEF Website, http://www.clef-campaign.org/.
ELRA Catalogue of Language Resources, http://catalog.elra.info.
A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, A. Stolcke, and C.Wooters. The ICSI meeting corpus. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, 2003.
D. Mostefa, K. Bernardin, H. K. Ekenel, N. Gourier, D. Macho, R. Malkin, F. Marques, R. Morros, K. Nickel, M. Omologo, G. Potamianos, R. Stiefelhagen, and J. Turmo. CHIL Public Deliverable D7.6: Exploitation Material for CHIL Evaluation Campaign 1. http://chil.server.de/servlet/is/8063/.
D. Mostefa, M.-N. Garcia, and K. Choukri. Evaluation of multimodal components within CHIL. In Proceedings of the 5th International Language Resources and Evaluations Conference (LREC), Genoa, Italy, 2006.
The NIST MarkIII Microphone Array, http://www.nist.gov/smartspace/cmaiii.html.
The Rich Transcription 2006 Spring Meeting Recognition Evaluation website, http://www.nist.gov/speech/tests/rt/rt2006/spring.
The NIST Smart Space project. http://www.nist.gov/smartspace/.
R. Stiefelhagen, K. Bernardin, R. Bowers, J. Garofolo, D. Mostefa, and P. Soundararajan. The CLEAR 2006 evaluation. In Multimodal Technologies for Perception of Humans, Proceedings of the First International CLEAR Evaluation Workshop, CLEAR 2006, LNCS 4122, pages 1–45, Southampton, UK, Apr. 6-7 2006.
R. Stiefelhagen, K. Bernardin, R. Bowers, R. T. Rose, M. Michel, and J. Garofolo. The CLEAR 2007 evaluation. In Multimodal Technologies for Perception of Humans, Proceedings of the International Evaluation Workshops CLEAR 2007 and RT 2007, LNCS 4625, pages 3–34, Baltimore, MD, May 8-11 2007.
R. Stiefelhagen, R. Bowers, and J. Fiscus, editors. Multimodal Technologies for Perception of Humans, Proceedings of the International Evaluation Workshops CLEAR 2007 and RT 2007. LNCS 4625. Springer, Baltimore, MD, May 8-11 2007.
R. Stiefelhagen and J. Garofolo, editors. Multimodal Technologies for Perception of Humans, First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR'06. LNCS 4122. Springer, Southampton, UK, Apr. 6-7 2006.
The Transcriber Tool home page, http://trans.sourceforge.net.
VACE – Video Analysis and Content Extraction. https://control.nist.gov/dto/twiki/bin/view/Main/WebHome.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag London Limited
About this chapter
Cite this chapter
Moreau, N., Mostefa, D., Choukri, K., Stiefelhagen, R., Burger, S. (2009). Perceptual Component Evaluation and Data Collection. In: Waibel, A., Stiefelhagen, R. (eds) Computers in the Human Interaction Loop. Human–Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84882-054-8_15
Download citation
DOI: https://doi.org/10.1007/978-1-84882-054-8_15
Publisher Name: Springer, London
Print ISBN: 978-1-84882-053-1
Online ISBN: 978-1-84882-054-8
eBook Packages: Computer ScienceComputer Science (R0)