Why Is Facial Occlusion a Challenging Problem?

  • Hazım Kemal Ekenel
  • Rainer Stiefelhagen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5558)


This paper investigates the main reason for the obtained low performance when the face recognition algorithms are tested on partially occluded face images. It has been observed that in the case of upper face occlusion, missing discriminative information due to occlusion only accounts for a very small part of the performance drop. The main factor is found to be the registration errors due to erroneous facial feature localization. It has been shown that by solving the misalignment problem, very high correct recognition rates can be achieved with a generic local appearance-based face recognition algorithm. In the case of a lower face occlusion, only a slight decrease in the performance is observed, when a local appearance-based face representation approach is used. This indicates the importance of local processing when dealing with partial face occlusion. Moreover, improved alignment increases the correct recognition rate also in the experiments against the lower face occlusion, which shows that face registration plays a key role on face recognition performance.


  1. 1.
    Martinez, A.M.: Recognizing imprecisely localized, partially occluded and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(6), 748–763 (2002)Google Scholar
  2. 2.
    Tan, X., Chen, S., Zhou, Z.H., Zhang, F.: Recognizing partially occluded, expression variant faces from single training image per person with SOM and soft k-NN ensemble. IEEE Transactions on Neural Networks 16(4), 875–886 (2005)Google Scholar
  3. 3.
    Park, B.G., Lee, K.M., Lee, S.U.: Face recognition using face-ARG matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(12), 1982–1988 (2005)Google Scholar
  4. 4.
    Fidler, S., Skočaj, D., Leonardis, A.: Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(3), 337–350 (2006)Google Scholar
  5. 5.
    Wright, J., Ganesh, A., Yang, A., Ma, Y.: Robust face recognition via sparse representation. Technical report, University of Illinois, USA (2007)Google Scholar
  6. 6.
    Jia, H., Martinez, A.M.: Face recognition with occlusions in the training and testing sets. In: Proceedings of IEEE Int’l. Conf. on Automatic Face and Gesture Recognition (2008)Google Scholar
  7. 7.
    Ekenel, H.K., Stiefelhagen, R.: Local appearance-based face recognition using discrete cosine transform. In: Proc. of 13th European Signal Processing Conf. (EUSIPCO 2005) (2005)Google Scholar
  8. 8.
    Ekenel, H.K., Stiefelhagen, R.: Analysis of local appearance-based face recognition: Effects of feature selection and feature normalization. In: Conf. on Computer Vision and Pattern Recognition Workshop 2006, p. 34 (2006)Google Scholar
  9. 9.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)Google Scholar
  10. 10.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: Recognition using class-specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)Google Scholar
  11. 11.
    Ekenel, H.K., Sankur, B.: Feature selection in the independent component subspace for face recognition. Pattern Recognition Letters 25(12), 1377–1388 (2004)Google Scholar
  12. 12.
    Gross, R., Shi, J., Cohn, J.F.: Que vadis face recognition? In: Proceedings of Workshop on Empirical Evaluation Methods in Computer Vision (2001)Google Scholar
  13. 13.
    Martinez, A.M., Benavente, R.: The AR face database. Technical Report 24, CVC (1998)Google Scholar
  14. 14.
    Penev, P.S., Atick, J.: Local feature analysis: a general statistical theory for object represntation. Network: Computation in Neural Systems 7, 477–500 (1996)Google Scholar
  15. 15.
    Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the Face Recognition Grand Challenge. In: Proc. 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Los Alamitos, CA, USA, vol. 1, pp. 947–954 (2005)Google Scholar
  16. 16.
    Lemieux, A., Parizeau, M.: Experiments on eigenfaces robustness. In: Proc. Int’l. Conf. on Pattern Recognition, vol. 1, pp. 421–424 (2002)Google Scholar
  17. 17.
    Rentzeperis, E., Stergiou, A., Pnevmatikakis, A., Polymenakos, L.: Impact of face registration errors on recognition. In: Maglogiannis, I., Karpouzis, K., Bramer, M. (eds.) Artificial Intelligence Applications and Innovations (AIAI 2006), pp. 187–194. Springer, Heidelberg (2006)Google Scholar
  18. 18.
    Wiskott, L., Fellous, J.M., Kruger, N., Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 775–779 (1997)Google Scholar
  19. 19.
    He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3), 328–340 (2005)Google Scholar
  20. 20.
    Xie, C., Kumar, B.V.K.V., Palanivel, S., Yegnanarayana, B.: A still-to-video face verification system using advanced correlation filters. In: Zhang, D., Jain, A.K. (eds.) ICBA 2004. LNCS, vol. 3072, pp. 102–108. Springer, Heidelberg (2004)Google Scholar
  21. 21.
    Samaria, F.S., Harter, A.C.: Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, pp. 138–142 (1994)Google Scholar
  22. 22.
    Ekenel, H.K.: A Robust Face Recognition Algorithm for Real-World Applications. PhD thesis, Universität Karlsruhe (TH), Germany (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Hazım Kemal Ekenel
    • 1
  • Rainer Stiefelhagen
    • 1
  1. 1.Computer Science DepatmentUniversität Karlsruhe (TH)KarlsruheGermany

Personalised recommendations