Multi-and Single View Multiperson Tracking for Smart Room Environments

  • Keni Bernardin
  • Tobias Gehrig
  • Rainer Stiefelhagen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4122)


Simultaneous tracking of multiple persons in real world environments is an active research field and several approaches have been proposed, based on a variety of features and algorithms. In this work, we present 2 multimodal systems for tracking multiple users in a smart room environment. One is a multi-view tracker based on color histogram tracking and special person region detectors. The other is a wide angle overhead view person tracker relying on foreground segmentation and model-based tracking. Both systems are completed by a joint probabilistic data association filter-based source localization framework using input from several microphone arrays.

We also very briefly present two intuitive metrics to allow for objective comparison of tracker characteristics, focusing on their precision in estimating object locations, their accuracy in recognizing object configurations and their ability to consistently label objects over time.

The trackers are extensively tested and compared, for each modality separately, and for the combined modalities, on the CLEAR 2006 Evaluation Database.


Color Histogram Visual Tracker Microphone Array Person Model Mismatch Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Khalaf, R.Y., Intille, S.S.: Improving Multiple People Tracking using Temporal Consistency. MIT Dept. of Architecture House n Project Technical Report (2001)Google Scholar
  2. 2.
    Niu, W., Jiao, L., Han, D., Wang, Y.-F.: Real-Time Multi-Person Tracking in Video Surveillance. In: Pacific Rim Multimedia Conference, Singapore (2003)Google Scholar
  3. 3.
    Mittal, A., Davis, L.S.: M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 18–33. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Checka, N., Wilson, K., Rangarajan, V., Darrell, T.: A Probabilistic Framework for Multi-modal Multi-Person Tracking. In: Workshop on Multi-Object Tracking (CVPR) (2003)Google Scholar
  5. 5.
    Comaniciu, D., Meer, P.: Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE PAMI 24 (May 2002)Google Scholar
  6. 6.
    Haritaoglu, I., Harwood, D., Davis, L.S.: W4: Who? When? Where? What? A Real Time System for Detecting and Tracking People. In: Third Face and Gesture Recognition Conference, pp. 222–227 (1998)Google Scholar
  7. 7.
    Raja, Y., McKenna, S.J., Gong, S.: Tracking and Segmenting People in Varying Lighting Conditions using Colour. In: 3rd. Int. Conference on Face & Gesture Recognition, p. 228 (1998)Google Scholar
  8. 8.
    Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: IEEE CVPR (2001)Google Scholar
  9. 9.
    Lienhart, R., Maydt, J.: An Extended Set of Haar-like Features for Rapid Object Detection. In: IEEE ICIP 2002, vol. 1, pp. 900–903 (Sept. 2002)Google Scholar
  10. 10.
    Gehrig, T., McDonough, J.: Tracking of Multiple Speakers with Probabilistic Data Association Filters. In: CLEAR Workshop, Southampton, UK, April (2006)Google Scholar
  11. 11.
    Bernardin, K., Elbs, A., Stiefelhagen, R.: Detection-Assisted Initialization, Adaptation and Fusion of Body Region Trackers for Robust Multiperson Tracking. In: IEEE International Conference on Pattern Recognition, 20-24 August 2006, Hong Kong (2006)Google Scholar
  12. 12.
    Nickel, K., Stiefelhagen, R.: Pointing Gesture Recognition based on 3Dtracking of Face, Hands and Head Orientation. In: 5th International Conference on Multimodal Interfaces, Vancouver, Canada (Nov. 2003)Google Scholar
  13. 13.
    Focken, D., Stiefelhagen, R.: Towards Vision-Based 3-D People Tracking in a Smart Room. In: IEEE International Conference on Multimodal Interfaces, Pittsburgh, PA, USA, October 14-16, pp. 400–405 (2002)Google Scholar
  14. 14.
    Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple Object Tracking Performance Metrics and Evaluation in a Smart Room Environment. In: Sixth IEEE International Workshop on Visual Surveillance, in conjunction with ECCV2006, May 13th 2006, Graz, Austria (2006)Google Scholar
  15. 15.
    Tao, H., Sawhney, H., Kumar, R.: A Sampling Algorithm for Tracking Multiple Objects. In: International Workshop on Vision Algorithms: Theory and Practice, pp. 53–68 (1999)Google Scholar
  16. 16.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-Time Tracking of the Human Body. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 780–785 (1997)CrossRefGoogle Scholar
  17. 17.
    CHIL - Computer in the Human Interaction Loop.
  18. 18.
    AMI - Augmented Multiparty Interaction.
  19. 19.
    VACE - Video Analysis and Content Extraction.
  20. 20.
    OpenCV - Computer Vision Library.

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Keni Bernardin
    • 1
  • Tobias Gehrig
    • 1
  • Rainer Stiefelhagen
    • 1
  1. 1.Interactive Systems Lab, Institut für Theoretische Informatik, Universität Karlsruhe, 76131 KarlsruheGermany

Personalised recommendations