Morphable Displacement Field Based Image Matching for Face Recognition across Pose

  • Shaoxin Li
  • Xin Liu
  • Xiujuan Chai
  • Haihong Zhang
  • Shihong Lao
  • Shiguang Shan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


Fully automatic Face Recognition Across Pose (FRAP) is one of the most desirable techniques, however, also one of the most challenging tasks in face recognition field. Matching a pair of face images in different poses can be converted into matching their pixels corresponding to the same semantic facial point. Following this idea, given two images G and P in different poses, we propose a novel method, named Morphable Displacement Field (MDF), to match G with P’s virtual view under G’s pose. By formulating MDF as a convex combination of a number of template displacement fields generated from a 3D face database, our model satisfies both global conformity and local consistency. We further present an approximate but effective solution of the proposed MDF model, named implicit Morphable Displacement Field (iMDF), which synthesizes virtual view implicitly via an MDF by minimizing matching residual. This formulation not only avoids intractable optimization of the high-dimensional displacement field but also facilitates a constrained quadratic optimization. The proposed method can work well even when only 2 facial landmarks are labeled, which makes it especially suitable for fully automatic FRAP system. Extensive evaluations on FERET, PIE and Multi-PIE databases show considerable improvement over state-of-the-art FRAP algorithms in both semi-automatic and fully automatic evaluation protocols.


Face Recognition Normalize Root Mean Square Error Virtual Image Virtual View Local Consistency 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zhang, X., Gao, Y.: Face recognition across pose: A review. PR (2009)Google Scholar
  2. 2.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. TPAMI (2003)Google Scholar
  3. 3.
    Asthana, A., Marks, T., Jones, M., Tieu, K., Mv, R.: Fully automatic pose-invariant face recognition via 3d pose normalization. In: ICCV (2011)Google Scholar
  4. 4.
    Cootes, T., Wheeler, G., Walker, K., Taylor, C.: View-based active appearance models. In: IVC (2002)Google Scholar
  5. 5.
    Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H.: Local gabor binary pattern histogram sequence (lgbphs): A novel non-statistical model for face representation and recognition. In: ICCV (2005)Google Scholar
  6. 6.
    Gross, R., Matthews, I., Baker, S.: Eigen light-fields and face recognition across pose. FG (2002)Google Scholar
  7. 7.
    Chai, X., Shan, S., Chen, X., Gao, W.: Locally linear regression for pose-invariant face recognition. TIP (2007)Google Scholar
  8. 8.
    Prince, S., Warrell, J., Elder, J., Felisberti, F.: Tied factor analysis for face recognition across large pose differences. TPAMI (2008)Google Scholar
  9. 9.
    Ashraf, A., Lucey, S., Chen, T.: Learning patch correspondences for improved viewpoint invariant face recognition. In: CVPR (2008)Google Scholar
  10. 10.
    Arashloo, S., Kittler, J.: Energy normalization for pose-invariant face recognition based on mrf model image matching. TPAMI (2011)Google Scholar
  11. 11.
    Castillo, C., Jacobs, D.: Wide-baseline stereo for face recognition with large pose variation. In: CVPR (2011)Google Scholar
  12. 12.
    Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The feret evaluation methodology for face-recognition algorithms. TPAMI (2000)Google Scholar
  13. 13.
    Sim, T., Baker, S., Bsat, M.: The cmu pose, illumination, and expression (pie) database. FG (2002)Google Scholar
  14. 14.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. In: IVC (2010)Google Scholar
  15. 15.
    Su, Y., Shan, S., Chen, X., Gao, W.: Hierarchical ensemble of gabor fisher classifier for face recognition. FG (2006)Google Scholar
  16. 16.
    Baocai, Y., Yanfeng, S., Chengzhang, W., Yun, G.: Bjut-3d large scale 3d face database and information processing. JCRD (2009)Google Scholar
  17. 17.
    Vetter, T., Poggio, T.: Linear object classes and image synthesis from a single example image. TPAMI (1997)Google Scholar
  18. 18.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. IJCAI (1981)Google Scholar
  19. 19.
    Shewchuk, J.: An introduction to the conjugate gradient method without the agonizing pain. In: CMUCS-TR (1994)Google Scholar
  20. 20.
    Li, A., Shan, S., Chen, X., Gao, W.: Maximizing intra-individual correlations for face recognition across pose differences. In: CVPR (2009)Google Scholar
  21. 21.
    Gross, R., Matthews, I., Baker, S.: Generic vs. person specific active appearance models. In: IVC (2005)Google Scholar
  22. 22.
    Murphy-Chutorian, E., Doshi, A., Trivedi, M.: Head pose estimation for driver assistance systems: A robust algorithm and experimental evaluation. In: ITSC (2007)Google Scholar
  23. 23.
    Zhao, X., Chai, X., Niu, Z., Heng, C., Shan, S.: Context constrained facial landmark localization based on discontinuous haar-like feature. FG (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Shaoxin Li
    • 1
    • 2
  • Xin Liu
    • 1
    • 2
  • Xiujuan Chai
    • 1
  • Haihong Zhang
    • 3
  • Shihong Lao
    • 3
  • Shiguang Shan
    • 1
  1. 1.Institute of Computing Technology, CASKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS)BeijingChina
  2. 2.Graduate University of Chinese Academy of SciencesBeijingChina
  3. 3.Omron Social Solutions Co., LTD.KyotoJapan

Personalised recommendations