A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

  • Michael GruppEmail author
  • Philipp Kopp
  • Patrik Huber
  • Matthias Rätsch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9776)


Face analysis techniques have become a crucial component of human-machine interaction in the fields of assistive and humanoid robotics. However, the variations in head-pose that arise naturally in these environments are still a great challenge.

In this paper, we present a real-time capable 3D face modelling framework for 2D in-the-wild images that is applicable for robotics. The fitting of the 3D Morphable Model is based exclusively on automatically detected landmarks. After fitting, the face can be corrected in pose and transformed back to a frontal 2D representation that is more suitable for face recognition. We conduct face recognition experiments with non-frontal images from the MUCT database and uncontrolled, in the wild images from the PaSC database, the most challenging face recognition database to date, showing an improved performance.

Finally, we present our SCITOS G5 robot system, which incorporates our framework as a means of image pre-processing for face analysis.



The authors would like to thank Huan Fui Lee and the RT-LIONS robocup team of Reutlingen University. We would also like to thank CyberExtruder, Inc. for supporting our research.


  1. 1.
    Aldrian, O., Smith, W.A.: Inverse rendering of faces with a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1080–1093 (2013)CrossRefGoogle Scholar
  2. 2.
    Asthana, A., Jones, M.J., Marks, T.K., Tieu, K.H., Goecke, R.: Pose normalization via learned 2D warping for fully automatic face recognition. In: BMVC, pp. 1–11 (2011)Google Scholar
  3. 3.
    Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: IEEE International Conference on Computer Vision (ICCV), pp. 937–944 (2011)Google Scholar
  4. 4.
    Bas, A., Smith, W.A., Bolkart, T., Wuhrer, S.: Fitting a 3D morphable model to edges: a comparison between hard and soft correspondences. arXiv preprint (2016). arXiv:1602.01125
  5. 5.
    Beveridge, J.R., Phillips, P.J., Bolme, D.S., Draper, B.A., Givens, G.H., Lui, Y.M., Teli, M.N., Zhang, H., Scruggs, W.T., Bowyer, K.W., et al.: The challenge of face recognition from digital point-and-shoot cameras. In: IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8 (2013)Google Scholar
  6. 6.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)Google Scholar
  7. 7.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)CrossRefGoogle Scholar
  8. 8.
    Cootes, T.F., Taylor, C.J.: Statistical models of appearance for computer vision. Technical report, University of Manchester (2004)Google Scholar
  9. 9.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  10. 10.
    Dementhon, D.F., Davis, L.S.: Model-based object pose in 25 lines of code. Int. J. Comput. Vis. 15(1–2), 123–141 (1995)CrossRefGoogle Scholar
  11. 11.
    Egger, B., Schönborn, S., Forster, A., Vetter, T.: Pose normalization for eye gaze estimation and facial attribute description from still images. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 317–327. Springer, Cham (2014). Scholar
  12. 12.
    Feng, Z.H., Huber, P., Kittler, J., Christmas, W., Wu, X.J.: Random cascaded-regression copse for robust facial landmark detection. Signal Process. Lett. 22(1), 76–80 (2015)CrossRefGoogle Scholar
  13. 13.
    Feng, Z.-H., Kittler, J., Christmas, W., Wu, X.-J.: Feature level multiple model fusion using multilinear subspace analysis with incomplete training set and its application to face image analysis. In: Zhou, Z.-H., Roli, F., Kittler, J. (eds.) MCS 2013. LNCS, vol. 7872, pp. 73–84. Springer, Heidelberg (2013). Scholar
  14. 14.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2004). ISBN 0521540518CrossRefGoogle Scholar
  15. 15.
    Hu, G., Chan, C.H., Kittler, J., Christmas, B.: Resolution-aware 3D morphable model. In: BMVC, pp. 1–10 (2012)Google Scholar
  16. 16.
    Huber, P., Feng, Z.H., Christmas, W., Kittler, J., Rätsch, M.: Fitting 3D morphable models using local features. In: ICIP (2015)Google Scholar
  17. 17.
    Khoshelham, K.: Accuracy analysis of kinect depth data. In: ISPRS Workshop Laser Scanning, vol. 38, pp. 133–138 (2011)CrossRefGoogle Scholar
  18. 18.
    Kopp, P., Grupp, M., Poschmann, P., Böhme, H.J., Rätsch, M.: Tracking system with pose-invariant face analysis for human-robot interaction. In: Informatics Inside (2015)Google Scholar
  19. 19.
    Martin, A., Doddington, G., Kamm, T., Ordowski, M., Przybocki, M.: The DET curve in assessment of detection task performance. Technical report (1997)Google Scholar
  20. 20.
    Milborrow, S., Morkel, J., Nicolls, F.: The MUCT landmarked face database. Pattern Recogn. Assoc. S. Afr. (2010).
  21. 21.
    Poschmann, P., Huber, P., Rätsch, M., Kittler, J., Böhme, H.J.: Fusion of tracking techniques to enhance adaptive real-time tracking of arbitrary objects. In: Conference on Intelligent Human Computer Interaction (IHCI) (2014)CrossRefGoogle Scholar
  22. 22.
    Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1952–1961 (2011)CrossRefGoogle Scholar
  23. 23.
    Rodríguez, J.R.T.: 3D face modelling for 2D+3D face recognition. Ph.D. thesis, University of Surrey (2007)Google Scholar
  24. 24.
    Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 986–993 (2005)Google Scholar
  25. 25.
    van Rootseler, R., Spreeuwers, L., Veldhuis, R.: Using 3D morphable models for face recognition in video. In: Proceedings of the 33rd WIC Symposium on Information Theory in the Benelux (2012)Google Scholar
  26. 26.
    Tena, J.R., Smith, R.S., Hamouz, M., Kittler, J., Hilton, A., Illingworth, J.: 2D face pose normalisation using a 3D morphable model. In: IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 51–56 (2007)Google Scholar
  27. 27.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  28. 28.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-511–I-518 (2001)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Michael Grupp
    • 1
    Email author
  • Philipp Kopp
    • 2
  • Patrik Huber
    • 3
  • Matthias Rätsch
    • 2
  1. 1.Technische Universität MünchenMunichGermany
  2. 2.Reutlingen UniversityReutlingenGermany
  3. 3.University of SurreyGuildfordUK

Personalised recommendations