Abstract
An automatic facial occlusion reconstruction system based upon a novel learning algorithm called the direct combined model (DCM) approach is presented. The system comprises two basic DCM modules, namely a shape reconstruction module and a texture reconstruction module. Each module models the occluded and non-occluded regions of the facial image in a single, combined eigenspace, thus preserving the correlations between the geometry of the facial features and the pixel grayvalues, respectively, in the two regions. As a result, when shape or texture information is available only for the non-occluded region of the facial image, the optimal shape and texture of the occluded region can be reconstructed via a process of Bayesian inference within the respective eigenspaces. To enhance the quality of the reconstructed results, the shape reconstruction module is rendered robust to facial feature point labeling errors by suppressing the effects of biased noises. Furthermore, the texture reconstruction module recovers the texture of the occluded facial image by synthesizing the global texture image and the local detailed texture image. The experimental results demonstrate that compared to existing facial reconstruction systems, the reconstruction results obtained using the proposed DCM-based scheme are quantitatively closer to the ground truth.
Chapter PDF
Similar content being viewed by others
References
Blanz, V., Romdhani, S., Vetter, T.: Face Identification across Different Poses and Illuminations with a 3D Morphable Model. In: IEEE Intl. Conf. on FG, pp. 202–207 (2002)
Chen, H., Xu, Y.Q., Shum, H.Y., Zhu, S.C., Zhen, N.N.: Example-Based Facial Sketch Generation with Non-Parametric Sampling. In: Proceedings of ICCV, pp. 433–438 (2001)
Cootes, T.F., Taylor, C.J.: Statistical Models of Appearance for Computer Vision. Technical Report, Univ. of Manchester (2000)
Fisker, R.: Making Deformable Template Models Operational. PhD Thesis, Informatics and Mathematical Modelling, Technical University of Denmark (2000)
Freeman, W.T., Pasztor, E.C.: Learning Low-Level Vision. In: ICCV, pp. 1182–1189 (1999)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)
Hwang, B.W., Blanz, V., Vetter, T., Lee, S.W.: Face Reconstruction from a Small Number of Feature Points. In: Proceedings of ICPR, pp. 842–845 (2000)
Hwang, B.W., Lee, S.W.: Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model. IEEE Trans. on PAMI 25(3), 365–372 (2003)
Jones, M.J., Poggio, T.: Multidimensional Morphable Models: A Framework for Representing and Matching Object Classes. IJCV 29(2), 107–131 (1998)
Lanitis, A.: Person Identification from Heavily Occluded Face Images. In: Handschuh, H., Hasan, M.A. (eds.) SAC 2004. LNCS, vol. 3357, pp. 5–9. Springer, Heidelberg (2004)
Liu, C., Shum, H.Y., Zhang, C.S.: A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model. In: Proc. of CVPR, pp. 192–198 (2001)
Mo, Z., Lewis, J.P., Neumann, U.: Face Inpainting with Local Linear Representations. In: Proceedings of British Machine Vision Conference, vol. 1, pp. 347–356 (2004)
Park, J.S., Oh, Y.H., Ahn, S.C., Lee, S.W.: Glasses Removal from Facial Image Using Recursive PCA Reconstruction. IEEE Trans. on PAMI 27(5), 805–811 (2005)
Saito, Y., Kenmochi, Y., Kotani, K.: Estimation of Eyeglassless Facial Images Using Principal Component Analysis. In: Proceedings of ICIP, vol. 4, pp. 197–201 (1999)
Vetter, T., Poggio, T.: Linear Object Classes and Image Synthesis from a Single Example Image. IEEE Trans. on PAMI 19(7), 733–742 (1997)
Wu, C., Liu, C., Shum, H.Y., Xu, Y.Q., Zhang, Z.: Automatic Eyeglasses Removal from Face Images. IEEE Trans. PAMI 26, 322–336 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tu, CT., Lien, JJ.J. (2007). Facial Occlusion Reconstruction: Recovering Both the Global Structure and the Local Detailed Texture Components. In: Mery, D., Rueda, L. (eds) Advances in Image and Video Technology. PSIVT 2007. Lecture Notes in Computer Science, vol 4872. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77129-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-77129-6_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77128-9
Online ISBN: 978-3-540-77129-6
eBook Packages: Computer ScienceComputer Science (R0)