Facial Contour Labeling via Congealing

  • Xiaoming Liu
  • Yan Tong
  • Frederick W. Wheeler
  • Peter H. Tu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


It is a challenging vision problem to discover non-rigid shape deformation for an image ensemble belonging to a single object class, in an automatic or semi-supervised fashion. The conventional semi- supervised approach [1] uses a congealing-like process to propagate manual landmark labels from a few images to a large ensemble. Although effective on an inter-person database with a large population, there is potential for increased labeling accuracy. With the goal of providing highly accurate labels, in this paper we present a parametric curve representation for each of the seven major facial contours. The appearance information along the curve, named curve descriptor, is extracted and used for congealing. Furthermore, we demonstrate that advanced features such as Histogram of Oriented Gradient (HOG) can be utilized in the proposed congealing framework, which operates in a dual-curve congealing manner for the case of a closed contour. With extensive experiments on a 300-image ensemble that exhibits moderate variation in facial pose and shape, we show that substantial progress has been achieved in the labeling accuracy compared to the previous state-of-the-art approach.

Supplementary material

978-3-642-15549-9_26_MOESM1_ESM.avi (8.3 mb)
Electronic Supplementary Material (8,503 KB)


  1. 1.
    Tong, Y., Liu, X., Wheeler, F.W., Tu, P.: Automatic facial landmark labeling with minimal supervision. In: CVPR (2009)Google Scholar
  2. 2.
    Miller, E., Matsakis, N., Viola, P.: Learning from one example through shared densities on transforms. In: CVPR, vol. 1, pp. 464–471 (2000)Google Scholar
  3. 3.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE T-PAMI 23, 681–685 (2001)Google Scholar
  4. 4.
    Matthews, I., Baker, S.: Active appearance models revisited. IJCV 60, 135–164 (2004)CrossRefGoogle Scholar
  5. 5.
    Liu, X.: Discriminative face alignment. IEEE T-PAMI 31, 1941–1954 (2009)Google Scholar
  6. 6.
    Dalal, N., Triggs, W.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)Google Scholar
  7. 7.
    Vetter, T., Jones, M.J., Poggio, T.: A bootstrapping algorithm for learning linear models of object classes. In: CVPR, pp. 40–46 (1997)Google Scholar
  8. 8.
    Learned-Miller, E.: Data driven image models through continuous joint alignment. IEEE T-PAMI 28, 236–250 (2006)Google Scholar
  9. 9.
    Cox, M., Sridharan, S., Lucey, S., Cohn, J.: Least squares congealing for unsupervised alignment of images. In: CVPR (2008)Google Scholar
  10. 10.
    Balci, S., Golland, P., Shenton, M., Wells, W.: Free-form B-spline deformation model for groupwise registration. In: MICCAI, pp. 23–30 (2007)Google Scholar
  11. 11.
    Baker, S., Matthews, I., Schneider, J.: Automatic construction of active appearance models as an image coding problem. IEEE T-PAMI 26, 1380–1384 (2004)Google Scholar
  12. 12.
    Kokkinos, I., Yuille, A.: Unsupervised learning of object deformation models. In: ICCV (2007)Google Scholar
  13. 13.
    Cootes, T., Twining, C., Petrovic, V., Schestowitz, R., Taylor, C.: Groupwise construction of appearance models using piece-wise affine deformations. In: BMVC, vol. 2, pp. 879–888 (2005)Google Scholar
  14. 14.
    Cristinacce, D., Cootes, T.: Facial motion analysis using clustered shortest path tree registration. In: Proc. of the 1st Int. Workshop on Machine Learning for Vision-based Motion Analysis with ECCV (2008)Google Scholar
  15. 15.
    Torre, F., Nguyen, M.: Parameterized kernel principal component analysis: Theory and applications to supervised and unsupervised image alignment. In: CVPR (2008)Google Scholar
  16. 16.
    Langs, G., Donner, R., Peloschek, P., Horst, B.: Robust autonomous model learning from 2D and 3D data sets. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part I. LNCS, vol. 4791, pp. 968–976. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Saragih, J., Goecke, R.: A nonlinear discriminative approach to AAM fitting. In: ICCV (2007)Google Scholar
  18. 18.
    Sidorov, K., Richmond, S., Marshall, D.: An efficient stochastic approach to groupwise non-rigid image registration. In: CVPR (2009)Google Scholar
  19. 19.
    Meltzer, J., Soatto, S.: Edge descriptors for robust wide-baseline correspondence. In: CVPR (2008)Google Scholar
  20. 20.
    Baker, S., Matthews, I.: Lucas-Kanade 20 years on: A unifying framework. IJCV 56, 221–255 (2004)CrossRefGoogle Scholar
  21. 21.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models — their training and application. CVIU 61, 38–59 (1995)Google Scholar
  22. 22.
    Liu, X., Yu, T., Sebastian, T., Tu, P.: Boosted deformable model for human body alignment. In: CVPR (2008)Google Scholar
  23. 23.
    Liu, X., Tong, Y., Wheeler, F.W.: Simultaneous alignment and clustering for an image ensemble. In: ICCV (2009)Google Scholar
  24. 24.
    Kasinski, A., Florek, A., Schmidt, A.: The PUT face database. Technical report, Poznan University of Technology, Poznan, Poland (2009)Google Scholar
  25. 25.
    Schneiderman, H.: Learning a restricted Bayesian network for object detection. In: CVPR (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Xiaoming Liu
    • 1
  • Yan Tong
    • 1
  • Frederick W. Wheeler
    • 1
  • Peter H. Tu
    • 1
  1. 1.Visualization and Computer Vision LabGE Global ResearchNiskayuna

Personalised recommendations