Abstract
This paper is a summary of the 2007 CLEAR Evaluation on the Classification of Events, Activities, and Relationships which took place in early 2007 and culminated with a two-day workshop held in May 2007. CLEAR is an international effort to evaluate systems for the perception of people, their activities, and interactions. In its second year, CLEAR has developed a following from the computer vision and speech communities, spawning a more multimodal perspective of research evaluation. This paper describes the evaluation tasks, including metrics and databases used, and discusses the results achieved. The CLEAR 2007 tasks comprise person, face, and vehicle tracking, head pose estimation, as well as acoustic scene analysis. These include subtasks performed in the visual, acoustic and audio-visual domains for meeting room and surveillance data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
CHIL - Computers In the Human Interaction Loop, http://chil.server.de
AMI - Augmented Multiparty Interaction, http://www.amiproject.org
VACE - Video Analysis and Content Extraction, https://control.nist.gov/dto/twiki/bin/view/Main/WebHome
CALO - Cognitive Agent that Learns and Organizes, http://caloproject.sri.com/
NIST Rich Transcription Meeting Recognition Evaluations, http://www.nist.gov/speech/tests/rt/rt2006/spring/
The i-LIDS dataset, http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-imaging-technology/video-based-detection-systems/i-lids/
DARPA VIVID, http://www.vividevaluation.ri.cmu.edu/datasets/PETS2005/PkTest02/index.html
CLEAR evaluation webpage, http://www.clear-evaluation.org
ELRA/ELDA’s Catalogue of Language Resources, http://catalog.elda.org/
Ba, S.O., Odobez, J.M.: Evaluation of head pose tracking algorithm in indoor environments. In: International Conference on Multimedia & Expo, ICME 2005, Amsterdam, Netherlands (2005)
Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. In: 6th IEEE Int. Workshop on Visual Surveillance, VS 2006 (May 2006)
Bernardin, K., Gehrig, T., Stiefelhagen, R.: Multi- and single view multiperson tracking for smart room environments. In: Stiefelhagen, R., Garofolo, J.S. (eds.) CLEAR 2006. LNCS, vol. 4122, Springer, Heidelberg (2007)
Bernardin, K., Gehrig, T., Stiefelhagen, R.: Multi-level particle filter fusion of features and cues for audio-visual person tracking. In: Multimodal Technologies for Perception of Humans, Joint Proceedings of the CLEAR 2007 and RT 2007 Evaluation Workshops, Baltimore, MD, USA. LNCS, vol. 4625, Springer, Heidelberg (2007)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: The CLEAR MOT Metrics. EURASIP Journal on Image and Video Processing, Special Issue on Video Tracking in Complex Scenes for Surveillance Applications (submitted)
Canton-Ferrer, C., Salvador, J., Casas, J.R., Pardas, M.: Multi-person tracking strategies based on voxel analysis. In: Multimodal Technologies for Perception of Humans, Joint Proceedings of the CLEAR 2007 and RT 2007 Evaluation Workshops, Baltimore, MD, USA. LNCS, vol. 4625, Springer, Heidelberg (2007)
Doermann, D., Mihalcik, D.: Tools and techniques for video performances evaluation. In: International Conference on Pattern Recognition, pp. 167–170 (2000)
Ekenel, H.K., Jin, Q., Fischer, M., Stiefelhagen, R.: ISL Person Identification Systems in the CLEAR 2007 Evaluations. In: Multimodal Technologies for Perception of Humans, Joint Proceedings of the CLEAR 2007 and RT 2007 Evaluation Workshops, Baltimore, MD, USA. LNCS, vol. 4625, Springer, Heidelberg (2007)
Mariano, V.Y., Min, J., Park, J.-H., Kasturi, R., Mihalcik, D., Doermann, D., Drayer, T.: Performance evaluation of object detection algorithms. In: International Conference on Pattern Recognition, pp. 965–969 (2002)
Mostefa, D., Moreau, N., Choukri, K., Potamianos, G., Chu, S.M., Tyagi, A., Casas, J.R., Turmo, J., Christoforetti, L., Tobia, F., Pnevmatikakis, A., Mylonakis, V., Talantzis, F., Burger, S., Stiefelhagen, R., Bernardin, K., Rochet, C.: The chil audiovisual corpus for lecture and meeting analysis inside smart rooms. In: Language Resources and Evaluation, vol. 41, Springer, Heidelberg (2007)
Nechyba, M.C., Brandy, L., Schneiderman, H.: Pittpatt face detection and tracking for the clear 2007 evaluation. In: Multimodal Technologies for Perception of Humans, Joint Proceedings of the CLEAR 2007 and RT 2007 Evaluation Workshops, Baltimore, MD, USA. LNCS, vol. 4625, Springer, Heidelberg (2007)
Raju, H., Prasad, S.: Annotation guidelines for video analysis and content extraction (VACE-II). In: CLEAR Evaluation Workshop (2006)
Stiefelhagen, R., Bernardin, K., Bowers, R., Garofolo, J., Mostefa, D., Soundararajan, P.: The CLEAR 2006 Evaluation. In: Stiefelhagen, R., Garofolo, J.S. (eds.) CLEAR 2006. LNCS, vol. 4122, Springer, Heidelberg (2007)
Stiefelhagen, R., Garofolo, J.S. (eds.): CLEAR 2006. LNCS, vol. 4122. Springer, Heidelberg (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stiefelhagen, R., Bernardin, K., Bowers, R., Rose, R.T., Michel, M., Garofolo, J. (2008). The CLEAR 2007 Evaluation. In: Stiefelhagen, R., Bowers, R., Fiscus, J. (eds) Multimodal Technologies for Perception of Humans. RT CLEAR 2007 2007. Lecture Notes in Computer Science, vol 4625. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68585-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-68585-2_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68584-5
Online ISBN: 978-3-540-68585-2
eBook Packages: Computer ScienceComputer Science (R0)