Skip to main content
SpringerLink
Log in

Spatial pyramid face feature representation and weighted dissimilarity matching for improved face recognition

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we present a novel face recognition (FR) algorithm based on multiresolution spatial pyramid. In our method, a face is subdivided into increasingly finer subregions (local regions) and represented at multiple levels of histogram representations. To address image misalignment problem, overlapped patch-based local descriptor extraction has been also developed in an effective way. To preserve multiple levels of detail in facial local characteristics and to encode holistic spatial configuration, face features obtained for concatenated histograms (coming from all levels of spatial pyramid) are integrated into a combined feature set, termed spatial pyramid face feature representation (SPFR). In addition, to perform recognition by matching between the pair of probe and gallery SPFR sets, we propose the use of a weighted sum of the dissimilarity scores computed at all spatial pyramid levels. For this purpose, we develop a novel weight determination solution based on class-wise discriminant power estimation for face feature at a specific pyramid level. We incorporate our proposed algorithm into general FR pipeline and achieve encouraging identification results on the CMU-PIE, FERET, and LFW datasets, compared to previously developed methods. In addition, the feasibility of our method has been successfully demonstrated by making comparisons with other state-of-the-art FR methods (including deep CNN based method) under the FERET and FRGC 2.0 evaluation protocols. Based on results, our method is advantageous in terms of high recognition accuracy and low complexity, as well as straightforward implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Article  Google Scholar 

  2. An, L., Kafai, M., Bhanu, B.: Face recognition in multi-camera surveillance videos using dynamic Bayesian network. In: International Conference on Distributed Smart Cameras (ICDSC), pp. 1–6 (2012)

  3. Brown, M., Hua, G., Winder, S.: Discriminative learning of local image descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 43–57 (2011)

    Article  Google Scholar 

  4. Chai, Z., Sun, Z., Mendez-Vazquez, H., He, R., Tan, T.: Gabor ordinal measures for face recognition. IEEE Trans. Inf. Forensics Secur. 9(1), 14–26 (2014)

    Article  Google Scholar 

  5. Chan, C., Tahir, M., Kittler, J., Pietikainen, M.: Multiscale local phase quantisation for robust component-based face recognition using kernel fusion of multiple descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1164–1177 (2013)

    Article  Google Scholar 

  6. Cheng, Y., Jiao, L., Cao, X., Li, Z.: Illumination-insensitive features for face recognition. Vis. Comput. (2017). (Online published)

  7. Choi, J.Y., Ro, Y.M., Plataniotis, K.N.: Color local texture features for color face recognition. IEEE Trans. Image Process. 21(3), 1366–1380 (2012)

    Article  MathSciNet  Google Scholar 

  8. Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV Workshop on Statistical Learning in Computer Vision, vol. 1(1–22), pp. 1–2 (2004)

  9. Ding, C., Tao, D.: A comprehensive survey on pose-invariant face recognition. ACM Trans. Intell. Syst. Technol. (TIST) 7(3), 1–40 (2016)

    Article  Google Scholar 

  10. Ding, C., Choi, J., Tao, D., Davis, L.S.: Multi-directional multi-level dual-cross patterns for robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 518–531 (2016)

    Article  Google Scholar 

  11. Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proceedings of the IEEE International Conference on CVPR , Vol. 2, pp. 524–531 (2005)

  12. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic, New York (1992)

    MATH  Google Scholar 

  13. Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV, pp. 221–228 (2009)

  14. Geng, C., Jiang, X.: Face recognition using SIFT features. In: Proceedings of the IEEE International Conference in Image Processing (ICIP), pp. 3313–3316 (2009)

  15. Hou, X.-N., Ding, S.-H., Ma, L.-Z., Wang, C.-J., Li, J.-L., Huang, F.-Y.: Similarity metric learning for face verification using sigmoid decision function. Vis. Comput. 32(4), 479–490 (2016)

    Article  Google Scholar 

  16. Hu, J.: Discriminative transfer learning with sparsity regularization for single-sample face recognition. Image Vis. Comput. 60, 48–57 (2017)

    Article  Google Scholar 

  17. Hua, S., Chen, G., Wei, H., Jiang, Q.: Similarity measure for image resizing using SIFT feature. EURASIP J. Image Video Process. 6, 1–11 (2012)

    Article  Google Scholar 

  18. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, vol. 1, pp. 7–49. University of Massachusetts, Amherst (2007)

    Google Scholar 

  19. Hwang, W., Wang, H., Kim, H., Kee, S.-C., Kim, J.: Face recognition system using multiple face model of hybrid fourier feature under uncontrolled illumination variation. IEEE Trans. Image Process. 20(4), 1152–1165 (2011)

    Article  MathSciNet  Google Scholar 

  20. Jiang, X., Mandal, B., Kot, A.: Eigenfeature regularization and extraction in face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 383–394 (2008)

    Article  Google Scholar 

  21. Ju, J., Plataniotis, K.N., Venetsanopoulos, A.N.: Regularized discriminant analysis for the small sample size problem in face recognition. Pattern Recognit. Lett. 24(16), 3079–3087 (2003)

    Article  Google Scholar 

  22. Kawahara, T., Yamaguchi, O.: Feature vector compression based on least error quantization. In: Proceedings of the IEEE International Conference on Computer Vision Pattern Recognition Workshops, pp. 16–24 (2016)

  23. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the IEEE International Conference on CVPR, vol. 2, pp. 2169–2178 (2006)

  24. Lei, Z., Li, S.Z., Chu, R., Zhu, X.: Face recognition with local gabor textons. In: Proceedings of the International Conference on Advances in Biometrics (ICB), Vol. 4642, pp. 49–57 (2007)

  25. Leung, T., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vis. 43(1), 29–44 (2001)

    Article  Google Scholar 

  26. Li, Z.S., Imai, J., Kaneko, M.: Robust face recognition using block-based bag of words. In: International Conference on Pattern Recognition (ICPR), pp. 1285–1288 (2010)

  27. Li, Z.S., Imai, J., Kaneko, M.: Block-based bag of words for robust face recognition under variant conditions of facial expression, illumination, and partial occlusion. IEICE Trans. Fundam. E94–A(2), 533–541 (2011)

    Article  Google Scholar 

  28. Liao, S., Jain, A.K.: Partial face recognition: an alignment free approach. In: Proceedings of the IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8 (2011)

  29. Liu, C., Wechsler, H.: Gabor feature based classification using the enhanced fisher liner discrimiant model for face recognition. IEEE Trans. Image Process. 11(4), 467–476 (2002)

    Article  Google Scholar 

  30. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  31. Lu, J., Plataniotis, K.N., Venetsanopoulos, A.N.: Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans. Neural Netw. 14(1), 117–126 (2003)

    Article  Google Scholar 

  32. Mantecón, T., Del-Blanco, C.R., Jaurequizar, F., García, N.: Visual face recognition using bag of dense derivative depth patterns. IEEE Trans. Signal Process. Lett. 23(6), 771–775 (2016)

    Article  Google Scholar 

  33. Maturana, D., Mery, D., Soto, A.: Face recognition with decision tree-based local binary patterns. In: Proceedings of the 10th Asian Conference on Computer Vision (ACCV), vol. 6495, pp. 618–629 (2010)

    Chapter  Google Scholar 

  34. Maturana, D., Mery, D., Soto, A.: Learning discriminative local binary patterns for face recognition. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG), pp. 470–475 (2011)

  35. Meng, X., Shan, S., Chen, X., Gao, W.: Local visual primitives (LVP) for face modelling and recognition. In: Proceedings of the 18th International Conference Pattern Recognition (ICPR), vol. 2, pp. 536–539 (2006)

  36. Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 1, 63–86 (2004)

    Article  Google Scholar 

  37. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1615–1630 (2005)

    Article  Google Scholar 

  38. Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, E.: Deep learning applications and challenges in big data analytics. J. Big Data 2(1), 1 (2015)

    Article  Google Scholar 

  39. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. Br. Mach. Vis. 1(3), 6 (2015)

    Google Scholar 

  40. Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenges. In: Proceedings of the IEEE International Conference on Computer Vision Pattern Recognition, vol. 1, pp. 947–954 (2005)

  41. Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)

    Article  Google Scholar 

  42. Shafiee, M.J., Siva, P., Fieguth, P., Wong, A.: Embedded motion detection via neural response mixture background modeling. In: Proceedings of the IEEE International Conference on Computer Vision Pattern Recognition Workshops, pp. 19–26 (2016)

  43. Sim, T., Baker, S., Bsat, M.: The CMU pose illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)

    Article  Google Scholar 

  44. Sivic, J., Zisserman, A.: Efficient visual search cast as text retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 31, 591–606 (2009)

    Article  Google Scholar 

  45. Su, Y., Shan, S., Chen, X., Gao, W.: Hierarchical ensemble of global and local classifiers for face recognition. IEEE Trans. Image Process. 18, 1885–1886 (2009)

    Article  MathSciNet  Google Scholar 

  46. Tan, X., Triggs, B.: Fusing gabor and lbp feature sets for kernel- based face recognition. In: Proceedings of the International Conference on Analysis and Modeling of Faces and Gestures, pp. 235–249 (2007)

  47. Turk, M.A., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  48. Vedaldi, A., Fulkerson, B.: VLFeat—an open and portable library of computer vision algorithms. In: Proceedings of the ACM-MM, pp. 1469–1472 (2010)

  49. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE International Conference on CVPR. vol. 1, pp. I-511–I518 (2001)

  50. Vu, N.-S., Caplier, A.: Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans. Image Process. 21(3), 1352–1365 (2012)

    Article  MathSciNet  Google Scholar 

  51. Wang, Z., Miao, Z., Jonathan Wu, Q.M., Wan, Y., Tang, Z.: Low-resolution face recognition: a review. Vis. Comput. 30(4), 359–386 (2014)

    Article  Google Scholar 

  52. Xie, S., Shan, S., Chen, X., Meng, X., Gao, W.: Learned local gabor patterns for face representation and recognition. Signal Process. 89(12), 2333–2344 (2009)

    Article  Google Scholar 

  53. Xie, S., Shan, S., Chen, X., Chen, J.: Fusing local patterns of gabor magnitude and phase for face recognition. IEEE Trans. Image Process. 19, 1349–1361 (2010)

    Article  MathSciNet  Google Scholar 

  54. Yang, M., Zhang, L., Shiu, Simon C.K., Zhang, D.: Robust Kernel representation with statistical local features for face recognition. IEEE Trans. Neural Netw. Learn. Syst. 24(6), 900–912 (2013)

    Article  Google Scholar 

  55. Zhang, W., Shan, S., Gao, W., Zhang, H.: Local gabor binary pattern histogram sequence (lgbphs): a novel non-statistical model for face representation and recognition. In: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV), vol. 1, pp. 786–791 (2005)

  56. Zhang, B., Shan, S., Chen, X., Gao, W.: Histogram of gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Trans. Image Process. 16(1), 57–68 (2007)

    Article  MathSciNet  Google Scholar 

  57. Zhao, C., Li, X., Cang, Y.: Bisecting \(k\text{-means }\) clustering based face recognition using block-based bag of words models. Int. J. Light Electron. Opt. 126(19), 1761–1766 (2015)

    Article  Google Scholar 

  58. Zhao, L., Zhang, Y., Yin, B., Sun, Y., Hu, Y., Piao, X., Wu, Q.: Fisher discrimination-based \(l\)2,1-norm sparse representation for face recognition. Vis. Comput. 32(9), 1165–1178 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Hankuk University of Foreign Studies Research Fund. This research was also supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No.2015R1D1A1A01057420).

Author information

Authors and Affiliations

  1. Pattern Recognition and Machine Intelligence Lab., Division of Computer and Electronic Systems Engineering, Hankuk University of Foreign Studies, 81, Oedae-ro, Mohyeon-myeon, Cheoin-gu, Yongin-si, Gyeonggi-do, 17305, Republic of Korea

    Jae Young Choi

Authors
  1. Jae Young Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jae Young Choi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, J.Y. Spatial pyramid face feature representation and weighted dissimilarity matching for improved face recognition. Vis Comput 34, 1535–1549 (2018). https://doi.org/10.1007/s00371-017-1429-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1429-y

Keywords

Search

Navigation