Early Features Fusion over 3D Face for Face Recognition

  • Claudio Tortorici
  • Naoufel WerghiEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 684)


In this paper, a novel approach for fusing shape and texture Local Binary Patterns (LBP) for 3D Face Recognition is presented. Using the recently proposed mesh-LBP [23], it is now possible to compute LBP directly on a mesh manifold, allowing Early Feature Fusion to enhance face description power. Compared to its depth image counterparts, the proposed method (a) inherits the intrinsic advantages of mesh surfaces, (such as preservation of full geometry), (b) does not require face registration, (c) can accommodate partial or rotation matching, and (d) natively allows early-level fusion of texture and shape descriptors. The advantages of early-fusion is presented together with an experimentation of two merging schemes tested on the Bosphorus database.


Local Binary Pattern Early feature-fusion LBP 3D face recognition 


  1. 1.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. In: European Conference on Computer Vision, Prague, pp. 469–481, May 2004Google Scholar
  2. 2.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 469–481. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-24670-1_36 CrossRefGoogle Scholar
  3. 3.
    Alyüz, N., Gökberk, B., Akarun, L.: 3D face recognition system for expression and occlusion invariance. In: IEEE International Conference on Biometrics: Theory, Applications, and Systems, Washington, DC, pp. 1–7, September 2008Google Scholar
  4. 4.
    Berretti, S., Werghi, N., Del Bimbo, A., Pala, P.: Matching 3D face scans using interest points and local histogram descriptors. Comput. Graph. 37(5), 509–525 (2013)CrossRefGoogle Scholar
  5. 5.
    Bowyer, K.W., Chang, K.I., Flynn, P.J.: A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006)CrossRefGoogle Scholar
  6. 6.
    Chang, K., Bowyer, K., Flynn, P.: An evaluation of multimodal 2-D and 3-D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 619–624 (2005)CrossRefGoogle Scholar
  7. 7.
    Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local binary patterns and its application to facial image analysis: a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6), 765–781 (2011)CrossRefGoogle Scholar
  8. 8.
    Huang, Y., Wang, Y., Tan, T.: Combining statistics of geometrical and correlative features for 3D face recognition. In: British Machine Vision Conference, Edinburgh, pp. 879–888, September 2006Google Scholar
  9. 9.
    Li, H., Chen, L., Huang, D., Wang, Y., Morvan, J.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113(2), 128–142 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Li, H., Huang, D., Lemaire, P., Morvan, J.M., Chen, L.: Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities. In: IEEE International Conference on Image Processing, pp. 3053–3056, September 2011Google Scholar
  11. 11.
    Li, S., Zhao, C., Ao, M., Lei, Z.: Learning to fuse 3D+2D based face recognition at both feature and decision levels. In: International Workshop on Analysis and Modeling of Faces and Gestures, Beijing, pp. 44–54, October 2005Google Scholar
  12. 12.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  13. 13.
    Lu, X., Jain, A.K.: Deformation modeling for robust 3D face matching. In: IEEE International Conference on Computer Vision and Pattern Recognition, New York, pp. 1377–1383, June 2006Google Scholar
  14. 14.
    Maes, C., Fabry, T., Keustermans, J., Smeets, D., Suetens, P., Vandermeulen, D.: Feature detection on 3D face surfaces for pose normalisation and recognition. In: 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–6. IEEE (2010)Google Scholar
  15. 15.
    Mian, A.S., Bennamoun, M., Owens, R.: An efficient multimodal 2D-3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)CrossRefGoogle Scholar
  16. 16.
    Mian, A.S., Bennamoun, M., Owens, R.: Keypoint detection and local feature matching for textured 3D face recognition. Int. J. Comput. Vis. 79(1), 1–12 (2008)CrossRefGoogle Scholar
  17. 17.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distribution. Pattern Recognit. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  18. 18.
    Sandbach, G., Zafeiriou, S., Pantic, M.: Local normal binary patterns for 3D facial action unit detection. In: IEEE International Conference on Image Processing, Orlando, pp. 1813–1816, September 2012Google Scholar
  19. 19.
    Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: meshSIFT: local surface features for 3D face recognition under expression variations and partial data. Comput. Vis. Image Underst. 117(2), 158–169 (2013)CrossRefGoogle Scholar
  20. 20.
    Tang, H., Yin, B., Sun, Y., Hu, Y.: 3D face recognition using local binary patterns. Signal Process. 93(8), 2190–2198 (2013)CrossRefGoogle Scholar
  21. 21.
    Werghi, N., Berretti, S., Del Bimbo, A., Pala, P.: The mesh-LBP: computing local binary patterns on discrete manifolds. In: ICCV International Workshop on 3D Representation and Recognition, Sydney, pp. 562–569, December 2013Google Scholar
  22. 22.
    Werghi, N., Tortorici, C., Berretti, S., Del Bimbo, A.: Representing 3D texture on mesh manifolds for retrieval and recognition applications. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, pp. 2521–2530, June 2015Google Scholar
  23. 23.
    Werghi, N., Tortorici, C., Berretti, S., del Bimbo, A.: Local binary patterns on triangular meshes: concept and applications. Comput. Vis. Image Underst. 139, 161–177 (2015)CrossRefGoogle Scholar
  24. 24.
    Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)CrossRefGoogle Scholar
  25. 25.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. (CSUR) 35(4), 399–458 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Electrical and Computer EngineeringKhalifa UniversityAbu DhabiUAE

Personalised recommendations