Face Alignment Models

  • Phil Tresadern
  • Tim CootesEmail author
  • Chris Taylor
  • Vladimir Petrović


In order to interpret images of faces (e.g., for recognition), it is important to have a model of the different ways that a face may appear. Though faces vary widely, changes can be broken down into two categories—changes in shape and changes in the texture (patterns of pixel values) across the face—that are largely due to differences between individuals, but also due to changes in expression, viewpoint and lighting conditions. In this chapter, we describe a powerful method of generating compact models of shape and texture variation, and describe two methods—the Active Shape Model (ASM) and Active Appearance Model (AAM)—that fit an appearance model to an unseen image of the face so that we can interpret its underlying properties (e.g., identity).


Feature Point Training Image Shape Model Texture Variation Texture Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors would like to thank their numerous colleagues who have contributed to the research summarised in this chapter, including C. Beeston, F. Bettinger, D. Cooper, D. Cristinacce, G. Edwards, A. Hill, J. Graham, H. Kang, P. Kittipanya-ngam and M. Roberts.


  1. 1.
    Baker, S., Matthews, I.: Lucas–Kanade 20 years on: A unifying framework. Part I: The quantity approximated, the warp update rule and the gradient descent approximation. Int. J. Comput. Vis. (2004) Google Scholar
  2. 2.
    Batur, A.U., Hayes, M.H.: Adaptive active appearance models. IEEE Trans. Med. Imaging 14(11), 1707–1721 (2005) Google Scholar
  3. 3.
    Belkin, M., Nigoyi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003) CrossRefzbMATHGoogle Scholar
  4. 4.
    Benson, P.J., Perrett, D.I.: Synthesizing continuous-tone caricatures. Image Vis. Comput. 9, 123–129 (1991) CrossRefGoogle Scholar
  5. 5.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. (2003) Google Scholar
  6. 6.
    Bookstein, F.L.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989) CrossRefzbMATHGoogle Scholar
  7. 7.
    Cootes, T.F., Kittipanya-ngam, P.: Comparing variations on the active appearance model algorithm. In: 13th British Machine Vision Conf., vol. 2, pp. 837–846, September 2002 Google Scholar
  8. 8.
    Cootes, T., Taylor, C.J.: A mixture model for representing shape variation. Image Vis. Comput. 17(8), 567–574 (1999) CrossRefGoogle Scholar
  9. 9.
    Cootes, T.F., Taylor, C.J.: Constrained active appearance models. In: 8th Int’l Conf. on Comp. Vis., vol. 1, pp. 748–754, July 2001. IEEE Computer Society Press, Los Alamitos (2001) Google Scholar
  10. 10.
    Cootes, T.F., Taylor, C.J.: On representing edge structure for model matching. Comput. Vis. Pattern Recognit. 1, 1114–1119 (2001) Google Scholar
  11. 11.
    Cootes, T.F., Taylor, C.J., Cooper, D., Graham, J.: Active shape models—their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995) CrossRefGoogle Scholar
  12. 12.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) 5th European Conf. on Comp. Vis., vol. 2, pp. 484–498. Springer, Berlin (1998) Google Scholar
  13. 13.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: A comparative evaluation of active appearance model algorithms. In: British Machine Vision Conf., vol. 2, pp. 680–689, September 1998 Google Scholar
  14. 14.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001) CrossRefGoogle Scholar
  15. 15.
    Cootes, T.F., Wheeler, G.V., Walker, K.N., Taylor, C.J.: View-based active appearance models. Image Vis. Comput. 20, 657–664 (2002) CrossRefGoogle Scholar
  16. 16.
    Costen, N., Cootes, T.F., Taylor, C.J.: Compensating for ensemble-specificity effects when building facial models. Image Vis. Comput. 20, 673–682 (2002) CrossRefGoogle Scholar
  17. 17.
    Crandall, D., Felzenszwalb, P., Huttenlocher, D.: Spatial priors for part-based recognition using statistical models. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog., vol. 1 (2005) Google Scholar
  18. 18.
    Craw, I., Cameron, P.: Parameterising images for recognition and reconstruction. In: 2nd British Machine Vision Conf., pp. 367–370. Springer, London (1991) Google Scholar
  19. 19.
    Craw, I., Cameron, P.: Face recognition by computer. In: Hogg, D., Boyle, R. (eds.) 3rd British Machine Vision Conf., pp. 489–507. Springer, London (1992) Google Scholar
  20. 20.
    Cristinacce, D., Cootes, T.: Facial feature detection using AdaBoost with shape constraints. In: Proc. British Machine Vision Conf. (2003) Google Scholar
  21. 21.
    Cristinacce, D., Cootes, T.F.: Automatic feature localisation with constrained local models. Pattern Recognit. 41, 3054–3067 (2008) CrossRefzbMATHGoogle Scholar
  22. 22.
    Donner, R., Reitner, M., Langs, G., Peloschek, P., Bischof, H.: Fast active appearance model search using canonical correlation analysis. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1690–1694 (2006) CrossRefGoogle Scholar
  23. 23.
    Dryden, I., Mardia, K.V.: The Statistical Analysis of Shape. Wiley, London (1998) Google Scholar
  24. 24.
    Edwards, G.J., Lanitis, A., Taylor, C.J., Cootes, T.F.: Statistical models of face images—improving specificity. Image Vis. Comput. 16(3), 203–211 (1998) CrossRefGoogle Scholar
  25. 25.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. Int. J. Comput. Vis. 61(1), 55–79 (2005) CrossRefGoogle Scholar
  26. 26.
    Gao, X., Su, Y., Li, X., Tao, D.: A review of active appearance models. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 40(2), 145–158 (2010) CrossRefGoogle Scholar
  27. 27.
    Goodall, C.: Procrustes methods in the statistical analysis of shape. J. R. Stat. Soc. B 53(2), 285–339 (1991) MathSciNetzbMATHGoogle Scholar
  28. 28.
    Gu, L., Kanade, T.: A generative shape regularization model for robust face alignment. In: Proc. European Conf. on Computer Vision (2008) Google Scholar
  29. 29.
    Gu, L., Xing, E.P., Kanade, T.: Learning GMRF structures for spatial priors. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2007) Google Scholar
  30. 30.
    Hill, A., Cootes, T.F., Taylor, C.J.: Active shape models and the shape approximation problem. Image Vis. Comput. 14, 601–607 (1996) CrossRefGoogle Scholar
  31. 31.
    Hou, X., Li, S., Zhang, H., Cheng, Q.: Direct appearance models. In: Computer Vision and Pattern Recognition Conf. 2001, vol. 1, pp. 828–833 (2001) Google Scholar
  32. 32.
    Huang, Y., Liu, Q., Metaxas, D.N.: A component based deformable model for generalized face alignment. In: Proc. IEEE Int’l Conf. on Comp. Vis., pp. 1–8 (2007) Google Scholar
  33. 33.
    Jones, M.J., Poggio, T.: Multidimensional morphable models: A framework for representing and matching object classes. Int. J. Comput. Vis. 2(29), 107–131 (1998) CrossRefzbMATHGoogle Scholar
  34. 34.
    Kirby, M., Sirovich, L.: Application of the Karhumen–Loeve procedure for the characterization of human faces. IEEE Trans. Pattern Anal. Mach. Intell. 12(1), 103–108 (1990) CrossRefGoogle Scholar
  35. 35.
    la Torre, F.D., Collet, A., Quero, M., Cohn, J.F., Kanade, T.: Filtered component analysis to increase robustness to local minima in appearance models. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2007) Google Scholar
  36. 36.
    Lee, H.-S., Kim, D.: Tensor-based AAM with continuous variation estimation: Application to variation-robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1102–1116 (2009) CrossRefGoogle Scholar
  37. 37.
    Liang, L., Wen, F., Xu, Y.-Q., Tang, X., Shum, H.-Y.: Accurate face alignment using shape constrained Markov network. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2006) Google Scholar
  38. 38.
    Liang, L., Xiao, R., Wen, F., Sun, J.: Face alignment via component-based discriminative search. In: Proc. European Conf. on Computer Vision (2008) Google Scholar
  39. 39.
    Liu, X.: Discriminative face alignment. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1941–1954 (2009) CrossRefGoogle Scholar
  40. 40.
    Lu, H.-M., Fainman, Y., Hecht-Nelson, R.: Image manifolds. In: Proc. SPIE Symposium on Electronic Imaging: Science and Technology (1998) Google Scholar
  41. 41.
    Lucey, S., Wang, Y., Saragih, J., Cohn, J.F.: Non-rigid face tracking with enforced convexity and local appearance consistency constraint. Image Vis. Comput. 28(5), 781–789 (2010) CrossRefGoogle Scholar
  42. 42.
    Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 26(10), 135–164 (2004) CrossRefGoogle Scholar
  43. 43.
    Matthews, I., Xiao, J., Baker, S.: 2D vs. 3D deformable face models: Representational power, construction, and real-time fitting. Int. J. Comput. Vis. 75(1), 93–113 (2007) CrossRefGoogle Scholar
  44. 44.
    Milborrow, S., Nicolls, F.: Locating facial features with an extended active shape model. In: Proc. European Conf. on Computer Vision (2008) Google Scholar
  45. 45.
    Paquet, U.: Convexity and Bayesian constrained local models. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2009) Google Scholar
  46. 46.
    Romdhani, S., Gong, S., Psarrou, A.: A multi-view non-linear active shape model using kernel PCA. In: 10th British Machine Vision Conf., vol. 2, pp. 483–492, September 1999 Google Scholar
  47. 47.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science (2000) Google Scholar
  48. 48.
    Saragih, J., Goecke, R.: A nonlinear discriminative approach to AAM fitting. In: Proc. IEEE Int’l Conf. on Comp. Vis. (2007) Google Scholar
  49. 49.
    Saragih, J., Goecke, R.: Learning AAM fitting through simulation. Pattern Recognit. 42(11), 2628–2636 (2009) CrossRefzbMATHGoogle Scholar
  50. 50.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting with a mixture of local experts. In: Proc. IEEE Int’l Conf. on Comp. Vis. (2009) Google Scholar
  51. 51.
    Saragih, J.M., Lucey, S., Cohn, J.F.: Face alignment through subspace constrained mean-shifts. In: Proc. IEEE Int’l Conf. on Comp. Vis. (2009) Google Scholar
  52. 52.
    Sclaroff, S., Isidoro, J.: Active blobs. In: 6th Int’l Conf. on Comp. Vis., pp. 1146–1153 (1998) Google Scholar
  53. 53.
    Scott, I.M., Cootes, T.F., Taylor, C.J.: Improving appearance model matching using local image structure. In: Information Processing in Medical Imaging, pp. 258–269. Springer, Berlin (2003) CrossRefGoogle Scholar
  54. 54.
    Sozou, P.D., Cootes, T.F., Taylor, C.J., Mauro, E.C.D.: Non-linear generalization of point distribution models using polynomial regression. Image Vis. Comput. 13(5), 451–457 (1995) CrossRefGoogle Scholar
  55. 55.
    Stegmann, M.B., Ersbøll, B.K., Larsen, R.: FAME—a flexible appearance modelling environment. IEEE Trans. Med. Imaging 22(10), 1319–1331 (2003) CrossRefGoogle Scholar
  56. 56.
    Stegmann, M.B., Larsen, R.: Multi-band modelling of appearance. Image Vis. Comput. 21(1), 66–67 (2003) CrossRefGoogle Scholar
  57. 57.
    Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000) CrossRefGoogle Scholar
  58. 58.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991) CrossRefGoogle Scholar
  59. 59.
    van Ginneken, B., Frangi, A.F., Stall, J.J., ter Haar Romeny, B.M.: Active shape model segmentation with optimal features. IEEE Trans. Med. Imaging 21, 924–933 (2002) CrossRefGoogle Scholar
  60. 60.
    Vasilescu, M.A.O., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Proc. European Conf. on Computer Vision (2002) Google Scholar
  61. 61.
    Vetter, T.: Learning novel views to a single face image. In: 2nd Int’l Conf. on Automatic Face and Gesture Recognition 1996, pp. 22–27, October 1996 CrossRefGoogle Scholar
  62. 62.
    Wu, H., Liu, X., Doretto, G.: Face alignment via boosted ranking model. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2008) Google Scholar
  63. 63.
    Zhou, S.K., Comaniciu, D.: Shape regression machine. In: Proc. Int’l Conf. on Information Processing in Medical Imaging (2007) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Phil Tresadern
    • 1
  • Tim Cootes
    • 1
    Email author
  • Chris Taylor
    • 1
  • Vladimir Petrović
    • 1
  1. 1.Imaging Science and Biomedical EngineeringUniversity of ManchesterManchesterUK

Personalised recommendations