Probabilistic Human Recognition from Video

  • Shaohua Zhou
  • Rama Chellappa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


This paper presents a method for incorporating temporal information in a video sequence for the task of human recognition. A time series state space model, parameterized by a tracking state vector and a recognizing identity variable, is proposed to simultaneously characterize the kinematics and identity. Two sequential importance sampling (SIS) methods, a brute-force version and an efficient version, are developed to provide numerical solutions to the model. The joint distribution of both state vector and identity variable is estimated at each time instant and then propagated to the next time instant. Marginalization over the state vector yields a robust estimate of the posterior distribution of the identity variable. Due to the propagation of identity and kinematics, a degeneracy in posterior probability of the identity variable is achieved to give improved recognition. This evolving behavior is characterized using changes in entropy. The effectiveness of this approach is illustrated using experimental results on low resolution face data and upper body data.


Posterior Probability State Vector Face Recognition Video Sequence Conditional Entropy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    M. J. Black and A. D. Jepson. A probabilistic frameworkfor matching temporal trajectories. Proc. of ICCV, pages 176–181, 1999.Google Scholar
  2. 2.
    T. Choudhury, B. Clarkson, T. Jebara, and A. Pentland. Multimodal person recognition using unconstrained audio and video. Proc. of Intl. Conf. on Audio-and Video-Based Person Authentication, pages 176–181, 1999.Google Scholar
  3. 3.
    T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, 1991.Google Scholar
  4. 4.
    A. Doucet, S. J. Godsill, and C. Andrieu. On sequential monte carlo sampling methods for bayesian filtering. Statistics and Computing, 10(3):197–209, 2000.CrossRefGoogle Scholar
  5. 5.
    T. J. Edwards, C. J. Taylor, and T. F. Cootes. Improving identification performance by intergrating evidence from sequences. Proc. of CVPR, pages 486–491, 1999.Google Scholar
  6. 6.
    M. Isard and A. Blake. Contour tracking by stochatic propagation of conditional density. Proc. of ECCV, 1996.Google Scholar
  7. 7.
    A. K. Jain, R. Duin, and J. Mao. Statistical pattern recognition: A review. IEEE Trans. PAMI, 22:4–37, 2000.Google Scholar
  8. 8.
    G. Kitagawa. Monte carlo filter and smoother for non-gaussian nonlinear state space models. J. Computational and Graphical Statistics, 5:1–25, 1996.CrossRefMathSciNetGoogle Scholar
  9. 9.
    Baoxin Li and R. Chellappa. Simultaneous tracking and verification via sequential posterior estimation. Proc. of CVPR, pages 110–117, 2000.Google Scholar
  10. 10.
    Yongmin Li, Shaogang Gong, and H. Liddell. Constucting structures of facial identities on the view sphere using kernel discriminant analysis. Proc. of the 2rd Intl. Workshop on SCTV, 2001.Google Scholar
  11. 11.
    J. S. Liu and R. Chen. Sequential monte carlo for dynamic systems. Journal of the American Statistical Association, 93:1031–1041, 1998.Google Scholar
  12. 12.
    P. J. Philipps, H. Moon, S. Rivzi, and P. Ross. The feret testing protocol. Face Recognition: From Theory to Applications, 83:244–261, 1998.Google Scholar
  13. 13.
    L. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proc. of IEEE, 77(2), 1989.Google Scholar
  14. 14.
    De la Torre, F. Y. Yacoob, and L. Davis. A probabilistic framework for rigid and non-rigid appearance-based tracking and recognition. Proc. of 4th Intl. Conf. on Auto. Face and Gesture Recognition, 2000.Google Scholar
  15. 15.
    M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neutoscience, 3:72–86, 1991.Google Scholar
  16. 16.
    W. Y. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phillips. Face recognition: A literature survey. UMD CfAR Technical Report CAR-TR-948, 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Shaohua Zhou
    • 1
  • Rama Chellappa
    • 1
  1. 1.Center for Automation Research (CfAR) Department of Electrical and Computer EngineeringUniversity of MarylandCollege Park

Personalised recommendations