Abstract
This paper addresses the problem of how to learn an appropriate feature representation from video to benefit video-based face recognition. By simultaneously exploiting the spatial and temporal information, the problem is posed as learning Spatio-Temporal Embedding (STE) from raw video. STE of a video sequence is defined as its condensed version capturing the essence of space-time characteristics of the video. Relying on the co-occurrence statistics and supervised signatures provided by training videos, STE preserves the intrinsic temporal structures hidden in video volume, meanwhile encodes the discriminative cues into the spatial domain. To conduct STE, we propose two novel techniques, Bayesian keyframe learning and nonparametric discriminant embedding (NDE), for temporal and spatial learning, respectively. In terms of learned STEs, we derive a statistical formulation to the recognition problem with a probabilistic fusion model. On a large face video database containing more than 200 training and testing sequences, our approach consistently outperforms state-of-the-art methods, achieving a perfect recognition accuracy.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Baker, S., Kanade, T.: Limits on Super-Resolution and How to Break Them. IEEE Trans. PAMI 24(9), 1167–1183 (2002)
Bar-hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning a mahalanobis metric from equivalence constraints. J. of Machine Learning Research 6, 937–965 (2005)
Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Trans. PAMI 19(7), 711–720 (1997)
Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley, New York (2000)
Fukunaga, K.: Statistical Pattern Recognition. Academic Press, London (1990)
Krüger, V., Zhou, S.: Exemplar-Based Face Recognition from Video. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 732–746. Springer, Heidelberg (2002)
Lee, K., Ho, J., Yang, M., Kriegman, D.: Video-Based Face Recognition Using Probabilistic Appearance Manifolds. In: Proc. IEEE Conf. CVPR, pp. 313–320 (2003)
Liu, C., Shum, H., Zhang, C.: A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model. In: Proc. IEEE Conf. CVPR, pp. 192–198 (2001)
Liu, X., Chen, T.: Video-Based Face Recognition Using Adaptive Hidden Markov Models. In: Proc. IEEE Conf. CVPR, pp. 340–345 (2003)
Liu, W., Lin, D., Tang, X.: TensorPatch Super-Resolution and Coupled Residue Compensation. In: Proc. IEEE Conf. CVPR, pp. 478–484 (2005)
Messer, K., Matas, J., Kittler, J., Luettin, J., Matitre, G.: XM2VTSDB: The Extended M2VTS Database. In: Proc. 2nd Int. Conf. Audio- and Video-Based Biometric Person Authentication, pp. 72–77 (1999)
Satoh, S.: Comparative Evaluation of Face Sequence Matching for Content-based Video Access. In: Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, pp. 163–168 (2000)
Tang, X., Li, Z.: Frame Synchronization and Multi-Level Subspace Analysis for Video Based Face Recognition. In: Proc. IEEE Conf. CVPR, pp. 902–907 (2004)
Wang, X., Tang, X.: A Unified Framework for Subspace Face Recognition. IEEE Trans. PAMI 26(9), 1222–1228 (2004)
Yamaguchi, O., Fukui, K., Maeda, K.: Face Recognition Using Temporal Image Sequence. In: Proc. Int. Conf. Face and Gesture Recognition, pp. 318–323 (1998)
Zhou, S., Krueger, V., Chellappa, R.: Probabilistic Recognition of Human Faces from Video. Computer Vision and Image Understanding 91(1), 214–245 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, W., Li, Z., Tang, X. (2006). Spatio-temporal Embedding for Statistical Face Recognition from Video. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3952. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744047_29
Download citation
DOI: https://doi.org/10.1007/11744047_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33834-5
Online ISBN: 978-3-540-33835-2
eBook Packages: Computer ScienceComputer Science (R0)