Abstract
3D tracking of faces in video streams is a difficult problem that can be assisted with the use of a priori knowledge of the structure and appearance of the subject’s face at predefined poses (keyframes). This paper provides an extensive analysis of a state-of-the-art keyframe-based tracker: quantitatively demonstrating the dependence of tracking performance on underlying mesh accuracy, number and coverage of reliably matched feature points, and initial keyframe alignment.
Tracking with a generic face mesh can introduce an erroneous bias that leads to degraded tracking performance when the subject’s out-of-plane motion is far from the set of keyframes. To reduce this bias, we show how online refinement of a rough estimate of face geometry may be used to re-estimate the 3d keyframe features, thereby mitigating sensitivities to initial keyframe inaccuracies in pose and geometry. An in-depth analysis is performed on sequences of faces with synthesized rigid head motion.
Subsequent trials on real video sequences demonstrate that tracking performance is more sensitive to initial model alignment and geometry errors when fewer feature points are matched and/or do not adequately span the face. The analysis suggests several indications for most effective 3D tracking of faces in real environments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fua, P.: Using model-driven bundle-adjustment to model heads from raw video sequences. In: Proceedings of the 7th International Conference on Computer Vision, Corfu, Greece, September 1999, p. 4653 (1999)
Vacchetti, L., Lepetit, V., Fua, P.: Stable Real-Time 3D Tracking Using Online and Offline Information. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1385–1391 (2004)
Shan, Y., Liu, Z., Zhang, Z.: Model-Based Bundle Adjustment with Application to Face Modeling. In: International Conference on Computer Vision, Vancouver, Canada (July 2001)
Lepetit, V., Vacchetti, L., Thalmann, D., Fua, P.: Fully Automated and Stable Registration for Augmented Reality Applications. In: International Symposium on Mixed and Augmented Reality, Tokyo, Japan (September 2003)
Cascia, M., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(4) (April 2000)
Basu, S., Essa, I., Pentland, A.: Motion regularization for model-based head tracking. In: International Conference on Pattern Recognition (1996)
De Carlo, D., Metaxas, D.: The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation. In: Computer Vision and Pattern Recognition (1996)
Schodl, A., Haro, A., Essa, I.: Head Tracking using a Textured Polygonal Model. In: Proceedings of Perceptual User Interfaces Workshop (held in Conjunction with ACM UIST 1998), San Francisco, CA (November 1998)
Azarbayejani, A., Starner, T., Horowitz, B., Pentland, A.: Visually controlled graphics. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(6) (1993)
Azarbayejani, A., Pentland, A.: Recursive Estimation of Motion, Structure, and Focal Length. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(6) (1995)
Strom, J., Jebara, T., Basu, S., Pentland, A.: Real time Tracking and Modeling of Faces: An EKF-based Analysis by Synthesis Approach. In: Proceedings of the Modeling People Workshop at ICCV 1999 (1999)
Jebara, T., Pentland, A.: Parameterized Structure from Motion for 3D Adaptive Feedback Tracking of Faces. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 1997 (1997)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Geometrix, http://www.geometrix.com
EoS Systems Inc., http://www.photomodeler.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fidaleo, D., Medioni, G., Fua, P., Lepetit, V. (2005). An Investigation of Model Bias in 3D Face Tracking. In: Zhao, W., Gong, S., Tang, X. (eds) Analysis and Modelling of Faces and Gestures. AMFG 2005. Lecture Notes in Computer Science, vol 3723. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564386_11
Download citation
DOI: https://doi.org/10.1007/11564386_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29229-6
Online ISBN: 978-3-540-32074-6
eBook Packages: Computer ScienceComputer Science (R0)