Efficient Misalignment-Robust Representation for Real-Time Face Recognition

  • Meng Yang
  • Lei Zhang
  • David Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


Sparse representation techniques for robust face recognition have been widely studied in the past several years. Recently face recognition with simultaneous misalignment, occlusion and other variations has achieved interesting results via robust alignment by sparse representation (RASR). In RASR, the best alignment of a testing sample is sought subject by subject in the database. However, such an exhaustive search strategy can make the time complexity of RASR prohibitive in large-scale face databases. In this paper, we propose a novel scheme, namely misalignment robust representation (MRR), by representing the misaligned testing sample in the transformed face space spanned by all subjects. The MRR seeks the best alignment via a two-step optimization with a coarse-to-fine search strategy, which needs only two deformation-recovery operations. Extensive experiments on representative face databases show that MRR has almost the same accuracy as RASR in various face recognition and verification tasks but it runs tens to hundreds of times faster than RASR. The running time of MRR is less than 1 second in the large-scale Multi-PIE face database, demonstrating its great potential for real-time face recognition.


  1. 1.
    Zhao, W., Chellppa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. ACM Computing Survey 35, 399–458 (2003)CrossRefGoogle Scholar
  2. 2.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report Technical Report 07-49, University of Massachusetts (2007)Google Scholar
  3. 3.
    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: ICCV (2009)Google Scholar
  4. 4.
    Nowak, E., Jurie, F.: Learning visual similarity measures for comparing never seen objects. In: CVPR (2007)Google Scholar
  5. 5.
    Huang, G.B., Jain, V., Learned-Miller, E.: Unsupervised joint alignment of complex images. In: ICCV (2007)Google Scholar
  6. 6.
    Yin, Q., Tang, X.O., Sun, J.: An associate-predict model for face recognition. In: CVPR (2011)Google Scholar
  7. 7.
    Wagner, A., Wright, J., Ganesh, A., Zhou, Z.H., Mobahi, H., Ma, Y.: Towards a practical face recognition system: robust alignment and illumination by sparse representation. IEEE PAMI 34, 372–386 (2012)CrossRefGoogle Scholar
  8. 8.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE PAMI 31, 210–227 (2009)CrossRefGoogle Scholar
  9. 9.
    Elhamifar, E., Vidal, R.: Robust classification using structured sparse representation. In: CVPR (2011)Google Scholar
  10. 10.
    Yang, M., Zhang, L., Yang, J., Zhang, D.: Robust sparse coding for face recognition. In: CVPR (2011)Google Scholar
  11. 11.
    Zhou, Z., Wagner, A., Mobahi, H., Wright, J., Ma, Y.: Face recognition with contiguous occlusion using markov random fields. In: ICCV (2009)Google Scholar
  12. 12.
    Huang, J.Z., Huang, X.L., Metaxas, D.: Simultaneous image transformation and sparse representation recovery. In: CVPR (2008)Google Scholar
  13. 13.
    Zhang, L., Yang, M., Feng, X.C.: Sparse representation or collaborative representation which helps face recognition? In: ICCV (2011)Google Scholar
  14. 14.
    Yan, S.C., Wang, H., Liu, J.Z., Tang, X.O., Huang, T.S.: Misalignment robust face recognition. IEEE IP 19, 1087–1096 (2010)MathSciNetGoogle Scholar
  15. 15.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int’l J. Computer Vision 57, 137–154 (2004)CrossRefGoogle Scholar
  16. 16.
    Shan, S., Chang, Y., Gao, W., Cao, B., Yang, P.: Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution. In: AFGR (2004)Google Scholar
  17. 17.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE PAMI 23, 681–685 (2001)CrossRefGoogle Scholar
  18. 18.
    Cootes, T., Taylor, C.: Active shape models - ’smart snakes’. In: BMVC (1992)Google Scholar
  19. 19.
    Peng, Y.G., Ganesh, A., Wright, J., Xu, W.L., Ma, Y.: RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. In: CVPR (2010)Google Scholar
  20. 20.
    Yang, A.Y., Ganesh, A., Zhou, Z.H., Sastry, S.S., Ma, Y.: Fast l 1-minimizaton algorithms and application in robust face recognition. Technical Report UCB/EECS-2010-13, UC Berkeley (2010)Google Scholar
  21. 21.
    Kim, S.J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: A interior-point method for large-scale l 1-regularized least squares. IEEE Journal on Selected Topics in Signal Processing 1, 606–617 (2007)CrossRefGoogle Scholar
  22. 22.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. Image and Vision Computing 28, 807–813 (2010)CrossRefGoogle Scholar
  23. 23.
    Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE PAMI 23, 643–660 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Meng Yang
    • 1
  • Lei Zhang
    • 1
  • David Zhang
    • 1
  1. 1.Dept. of ComputingThe Hong Kong Polytechnic UniversityHong Kong

Personalised recommendations