Integrated Detection, Tracking, and Recognition of Faces with Omnivideo Array in Intelligent Environments

  • KohsiaS Huang
  • MohanM Trivedi
Open Access
Research Article
Part of the following topical collections:
  1. Anthropocentric Video Analysis: Tools and Applications


We present a multilevel system architecture for intelligent environments equipped with omnivideo arrays. In order to gain unobtrusive human awareness, real-time 3D human tracking as well as robust video-based face detection and tracking and face recognition algorithms are needed. We first propose a multiprimitive face detection and tracking loop to crop face videos as the front end of our face recognition algorithm. Both skin-tone and elliptical detections are used for robust face searching, and view-based face classification is applied to the candidates before updating the Kalman filters for face tracking. For video-based face recognition, we propose three decision rules on the facial video segments. The majority rule and discrete HMM (DHMM) rule accumulate single-frame face recognition results, while continuous density HMM (CDHMM) works directly with the PCA facial features of the video segment for accumulated maximum likelihood (ML) decision. The experiments demonstrate the robustness of the proposed face detection and tracking scheme and the three streaming face recognition schemes with 99% accuracy of the CDHMM rule. We then experiment on the system interactions with single person and group people by the integrated layers of activity awareness. We also discuss the speech-aided incremental learning of new faces.


Face Recognition Face Detection Video Segment Incremental Learning Tracking Loop 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publisher note

To access the full article, please see PDF.

Copyright information

© K. S. Huang and M. M. Trivedi. 2008

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Authors and Affiliations

  1. 1.Computer Vision and Robotics Research (CVRR) LaboratoryUniversity of CaliforniaSan Diego, La JollaUSA

Personalised recommendations