Advertisement

The CLEAR 2006 Evaluation

  • Rainer Stiefelhagen
  • Keni Bernardin
  • Rachel Bowers
  • John Garofolo
  • Djamel Mostefa
  • Padmanabhan Soundararajan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4122)

Abstract

This paper is a summary of the first CLEAR evaluation on CLassification of Events, Activities and Relationships - which took place in early 2006 and concluded with a two day evaluation workshop in April 2006. CLEAR is an international effort to evaluate systems for the multimodal perception of people, their activities and interactions. It provides a new international evaluation framework for such technologies. It aims to support the definition of common evaluation tasks and metrics, to coordinate and leverage the production of necessary multimodal corpora and to provide a possibility for comparing different algorithms and approaches on common benchmarks, which will result in faster progress in the research community. This paper describes the evaluation tasks, including metrics and databases used, that were conducted in CLEAR 2006, and provides an overview of the results. The evaluation tasks in CLEAR 2006 included person tracking, face detection and tracking, person identification, head pose estimation, vehicle tracking as well as acoustic scene analysis. Overall, more than 20 subtasks were conducted, which included acoustic, visual and audio-visual analysis for many of the main tasks, as well as different data domains and evaluation conditions.

Keywords

Face Detection Evaluation Task Visual Tracking Tracking Task Multiple Object Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    CHIL - Computers in the Human Interaction Loop. http://chil.server.de
  2. 2.
    AMI - Augmented Multiparty Interaction. http://www.amiproject.org
  3. 3.
    VACE - Video Analysis and Content Extraction. https://control.nist.gov/dto/twiki/bin/view/Main/WebHome
  4. 4.
    CALO - Cognitive Agent that Learns and Organizes, http://caloproject.sri.com/
  5. 5.
    NIST Rich Transcription Meeting Recognition Evaluations. http://www.nist.gov/speech/tests/rt/rt2006/spring/
  6. 6.
    PETS - Performance Evaluation of Tracking and Surveillance. http://www.cbsr.ia.ac.cn/conferences/VS-PETS-2005
  7. 7.
    TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid/
  8. 8.
    ETISEO Video Understanding Evaluation. http://www.silogic.fr/etiseo/
  9. 9.
    Mostefa, D., Garcia, M.N., Bernardin, K., Stiefelhagen, R., McDonough, J., Voit, M., Omologo, M., Marques, F., Ekenel, H.K., Pnevmatikakis, A.: Clear evaluation plan. Technical report (2006), http://www.clear-evaluation.org/downloads/chil-clear-v1.1-2006-02-21.pdf
  10. 10.
  11. 11.
    CLEAR evaluation webpage. http://www.clear-evaluation.org
  12. 12.
    Transcriber Labeling Tool. http://trans.sourceforge.net/
  13. 13.
    AGTK: Annotation Graph Toolkit. http://agtk.sourceforge.net/
  14. 14.
    Fiscus, J.G., Ajot, J., Michel, M., Garofolo, J.S.: The rich transcription 2006 spring meeting recognition evaluation. In: Renals, S., Bengio, S., Fiscus, J.G. (eds.) MLMI 2006. LNCS, vol. 4299, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
  16. 16.
    Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. In: Sixth IEEE International Workshop on Visual Surveillance, in conjunction with ECCV2006, Graz, Austria (2006)Google Scholar
  17. 17.
    ViPER: The Video Performance Evaluation Resource. http://viper-toolkit.sourceforge.net/
  18. 18.
    Danninger, M., Flaherty, G., Bernadin, K., Ekenel, H., Kohler, T., Malkin, R., Stiefelhagen, R., Waibel, A.: The Connector — facilitating context-aware communication. In: Proceedings of the International Conference on Multimodal Interfaces (2005)Google Scholar
  19. 19.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. 511–518. IEEE, Los Alamitos (2001)Google Scholar
  20. 20.
    ELRA/ELDA’s Catalogue of Language Resources: http://catalog.elda.org/

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Rainer Stiefelhagen
    • 1
  • Keni Bernardin
    • 1
  • Rachel Bowers
    • 2
  • John Garofolo
    • 2
  • Djamel Mostefa
    • 3
  • Padmanabhan Soundararajan
    • 4
  1. 1.Interactive Systems Lab, Universität Karlsruhe, 76131 KarlsruheGermany
  2. 2.National Institute of Standards and Technology (NIST), Information Technology Lab - Information Access Division, Speech Group 
  3. 3.Evaluations and Language Resources Distribution Agency (ELDA), ParisFrance
  4. 4.Computer Science and Engineering, University of South Florida, Tampa, FLUSA

Personalised recommendations