Skip to main content
SpringerLink
Log in

Three Dimensional Face Recognition via Surface Harmonic Mapping and Deep Learning

  • Conference paper
  • First Online:
Biometric Recognition (CCBR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10568))

Included in the following conference series:

Abstract

In this paper, we propose a general 3D face recognition framework by combining the idea of surface harmonic mapping and deep learning. In particular, given a 3D face scan, we first run the pre-processing pipeline and detect three main facial landmarks (i.e., nose tip and two inner eye corners). Then, harmonic mapping is employed to map the 3D coordinates and differential geometry quantities (e.g., normal vectors, curvatures) of each 3D face scan to a 2D unit disc domain, generating a group of 2D harmonic shape images (HSI). The 2D rotation of the harmonic shape images are removed by using the three detected landmarks. All these pose normalized harmonic shape images are fed into a pre-trained deep convolutional neural network (DCNN) to generate their deep representations. Finally, sparse representation classifier with score-level fusion is used for face similarity measurement and the final decision. The advantage of our method is twofold: (i) it is a general framework and can be easily extended to other surface mapping and deep learning algorithms. (ii) it is registration-free and only needs three landmarks. The effectiveness of the proposed framework was demonstrated on the BU-3DFE database, and reporting a rank-one recognition rate of 89.38% on the whole database.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www3.cs.stonybrook.edu/~gu/tutorial/HarmonicMap.html.

References

  1. Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3D solution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 146–155 (2016)

    Google Scholar 

  2. Jourabloo, A., Liu, X.: Large-pose face alignment via CNN-based dense 3D model fitting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4188–4196 (2016)

    Google Scholar 

  3. Tang, Y., Chen, L.: 3D facial geometric attributes based anti-spoofing approach against mask attacks. In: IEEE Conference on Face and Gesture, pp. 1–8 (2017)

    Google Scholar 

  4. Zhang, J., Huang, D., Wang, Y., Sun, J.: Lock3DFace: a large-scale database of low-cost Kinect 3D faces. In: International Conference on Biometrics, pp. 1–8 (2016)

    Google Scholar 

  5. Shen, F., Shen, C., van den Hengel, A., Tang, Z.: Approximate least trimmed sum of squares fitting and applications in image analysis. IEEE Trans. Image Process. 22(5), 1836–1847 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  6. Shen, F., Yang, Y., Zhou, X., Shen, H.T.: Face identification with second-order pooling in single-layer networks. Neurocomputing 187, 11–18 (2015)

    Article  Google Scholar 

  7. Shen, F., Shen, C., Zhou, X., Yang, Y., Shen, H.T.: Face image classification by pooling raw features. Pattern Recogn. 54, 94–103 (2016)

    Article  Google Scholar 

  8. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference, pp. 41.1–41.12 (2015)

    Google Scholar 

  9. Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. Computing Research Repository abs/1411.7923 (2014)

    Google Scholar 

  10. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

    Google Scholar 

  11. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898 (2014)

    Google Scholar 

  12. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)

    Google Scholar 

  13. Sinha, A., Bai, J., Ramani, K.: Deep learning 3D shape surfaces using geometry images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 223–240. Springer, Cham (2016). doi:10.1007/978-3-319-46466-4_14

    Chapter  Google Scholar 

  14. Maron, H., Galun, M., Aigerman, N., Trope, M., Dym, N., Yumer, E., Kim, V., Lipman, Y.: Convolutional neural networks on surfaces via seamless toric covers. ACM Trans. Graph. 36, 1–10 (2017)

    Article  Google Scholar 

  15. Wang, S., Wang, Y., Jin, M., Gu, X.D., Samaras, D.: Conformal geometry and its applications on 3D shape matching, recognition, and stitching. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1209–1220 (2007)

    Article  Google Scholar 

  16. Su, Z., Wang, Y., Shi, R., Zeng, W., Sun, J., Luo, F., Gu, X.: Optimal mass transport for shape matching and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 37(11), 2246–2259 (2015)

    Article  Google Scholar 

  17. Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Comput. Vis. Image Underst. 101, 1–15 (2006)

    Article  Google Scholar 

  18. Smeets, D., Claes, P., Hermans, J., Vandermeulen, D., Suetens, P.: A comparative study of 3-D face recognition under expression variations. IEEE Trans. Syst. Man Cybern. Part C 42(5), 710–727 (2012)

    Article  Google Scholar 

  19. Drira, H., Amor, B.B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2270–2283 (2013)

    Article  Google Scholar 

  20. Zhang, D., Hebert, M.: Harmonic maps and their applications in surface matching. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, p. 530 (1999)

    Google Scholar 

  21. Pan, G., Han, S., Wu, Z., Wang, Y.: 3D face recognition using mapped depth images. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p. 175 (2005)

    Google Scholar 

  22. Szeptycki, P., Ardabilian, M., Chen, L.: A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking. In: International Conference on Biometrics: Theory, Applications and Systems, pp. 3206–3211 (2009)

    Google Scholar 

  23. Szeptycki, P., Ardabilian, M., Chen, L., Zeng, W., Gu, D., Samaras, D.: Partial face biometry using shape decomposition on 2D conformal maps of faces. In: 20th International Conference on Pattern Recognition, pp. 1505–1508 (2010)

    Google Scholar 

  24. Echeagaray-Patron, B.A., Miramontes-Jaramillo, D., Kober, V.: Conformal parameterization and curvature analysis for 3D facial recognition. In: International Conference on Computational Science and Computational Intelligence, pp. 843–844 (2015)

    Google Scholar 

  25. Junhui, H., Fang, C., Ding, X., Sun, J., Gu, X.: 3D face recognition via conformal representation. In: Proceedings of SPIE - The International Society for Optical Engineering, vol. 9013 (2014)

    Google Scholar 

  26. Schoen, R., Yau, S.T.: Lectures on Harmonic Maps. International Press, Inc., Boston (1997)

    MATH  Google Scholar 

  27. Goldfeather, J., Interrante, V.: A novel cubic-order algorithm for approximating principal direction vectors. ACM Trans. Graph. 23(1), 45–63 (2004)

    Article  Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  29. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  30. Li, H., Huang, D., Morvan, J., Chen, L., Wang, Y.: Expression-robust 3D face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing 133, 179–193 (2014)

    Article  Google Scholar 

  31. Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: 7th International Conference on Automatic Face and Gesture Recognition, pp. 211–216 (2006)

    Google Scholar 

  32. Hajati, F., Raiea, A.A., Gao, Y.: 2.5 D face recognition using patch geodesic moments. Pattern Recogn. 45, 969–982 (2012)

    Article  MATH  Google Scholar 

  33. Emambakhsh, M., Evans, A.: Nasal patches and curves for expression-robust 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 995–1007 (2017)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the NSFC under grant 11401464, Chinese Postdoctoral Science Foundation under grant 2014M560785, and International Exchange Foundation of China NSFC and United Kingdom RS under grant 61711530242.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, China

    Xiaofan Wei & Huibin Li

  2. Department of Computer Science and Department of Mathematics, State University of New York at Stony Brook, New York, USA

    Xianfeng David Gu

Authors
  1. Xiaofan Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huibin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianfeng David Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huibin Li .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Jie Zhou

  2. Beihang University, Beijing, China

    Yunhong Wang

  3. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Zhenan Sun

  4. Computing and Technology, Chinese Academy of Sciences, Beijing, China

    Yong Xu

  5. Shenzhen University, Shenzhen, China

    Linlin Shen

  6. Tsinghua University, Beijing, China

    Jianjiang Feng

  7. Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

  8. Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Yu Qiao

  9. Graduate School at Shenzhen, Tsinghua University, Shenzhen, China

    Zhenhua Guo

  10. Shenzhen University, Shenzhen, China

    Shiqi Yu

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Wei, X., Li, H., Gu, X.D. (2017). Three Dimensional Face Recognition via Surface Harmonic Mapping and Deep Learning. In: Zhou, J., et al. Biometric Recognition. CCBR 2017. Lecture Notes in Computer Science(), vol 10568. Springer, Cham. https://doi.org/10.1007/978-3-319-69923-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-69923-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-69922-6

  • Online ISBN: 978-3-319-69923-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation