Intrinsic Face Image Decomposition from RGB Images with Depth Cues

  • Shirui LiuEmail author
  • Hamid A. Jalab
  • Zhen Dai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11870)


As a pre-step of reconstructing face attributes technology, the quality of face intrinsic image decomposition result has a direct impact on the sub-operations of reconstructing face attributes detail. There are two challenging problems with the intrinsic face image decomposition methods which are the quality of face-base intrinsic image, and the details of the shading image. In this study a new image model for intrinsic face image decomposition from RGB images with depth cues is proposed to produce high quality results even with simple constraints. The proposed model consists of three main steps: face cropping operation, processing the RGB color normalization, and the super-pixel segmentation. The face image is first cropped to get face area, then a color normalization process for the cropped face image is used to normalize RGB pixels, and finally the super-pixel segmentation based on mean shift algorithm is applied which has a good performance on reduce artifact and shading image’s detail retention. To evaluate the proposed model, both qualitative and quantitative assessments be used. The qualitative assessment is based on human subjective visual standards to compare the intrinsic images results, and the quantitative assessment is based on the data analyze of the image information entropy. Qualitative and quantitative results both demonstrate that the performance of the proposed model is better than other techniques in the field of intrinsic face image decomposition.


Intrinsic image decomposition Super-pixel segmentation Human face RGB and depth images 


  1. 1.
    Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 697–704. IEEE, June 2011Google Scholar
  2. 2.
    Barrow, H.G., Tannenbaum, J.M.: Recovering intrinsic scene characteristics from images. In: Hanson, A., Riseman, E. (eds.) Computer Vision Systems, 1st edn, 418 p. Academic Press (1978). ISBN 9780323151207Google Scholar
  3. 3.
    Jeon, J., Cho, S., Tong, X., Lee, S.: Intrinsic image decomposition using structure-texture separation and surface normals. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 218–233. Springer, Cham (2014). Scholar
  4. 4.
    Shu, Z., Yumer, E., Hadap, S., Sunkavalli, K., Shechtman, E., Samaras, D.: Neural face editing with intrinsic image disentangling. arXiv preprint arXiv:1704.04131 (2017)
  5. 5.
    Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (TOG) 33(4), 159 (2014)CrossRefGoogle Scholar
  6. 6.
    Liu, X., et al.: Intrinsic colorization. In: ACM Transactions on Graphics (TOG), vol. 27, no. 5, p. 152. ACM, December 2008CrossRefGoogle Scholar
  7. 7.
    Wang, Y., Li, K., Yang, J., Ye, X.: Intrinsic decomposition from a single RGB-D image with sparse and non-local priors. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 1201–1206. IEEE, July 2017Google Scholar
  8. 8.
    Yu, J.: Rank-constrained PCA for intrinsic images decomposition. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3578–3582. IEEE, September 2016Google Scholar
  9. 9.
    Nie, X., Feng, W., Wan, L., Dai, H., Pun, C.M.: Intrinsic image decomposition by hierarchical L 0 sparsity. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE, July 2014Google Scholar
  10. 10.
    Shi, J., Dong, Y., Tong, X., Chen, Y.: Efficient intrinsic image decomposition for RGBD images. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, pp. 17–25. ACM, November 2015Google Scholar
  11. 11.
    Bi, S., Han, X., Yu, Y.: An L 1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition. ACM Trans. Graph. (TOG) 34(4), 78 (2015)CrossRefGoogle Scholar
  12. 12.
    Hachama, M., Ghanem, B., Wonka, P.: Intrinsic scene decomposition from RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 810–818 (2015)Google Scholar
  13. 13.
    Barron, J.T., Malik, J.: High-frequency shape and albedo from shading using natural image statistics, pp. 2521–2528 (2011)Google Scholar
  14. 14.
    Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 334–341. IEEE, June 2012Google Scholar
  15. 15.
    Barron, J.T., Malik, J.: Intrinsic scene properties from a single RGB-D image. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17–24. IEEE, June 2013Google Scholar
  16. 16.
    Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 241–248. IEEE, December 2013Google Scholar
  17. 17.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: 2001 Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, p. 1. IEEE (2001)Google Scholar
  18. 18.
    Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. In: 2001 Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 2, pp. 383–390. IEEE (2001)Google Scholar
  19. 19.
    Tao, W., Jin, H., Zhang, Y.: Color image segmentation based on mean shift and normalized cuts. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 37(5), 1382–1389 (2007)CrossRefGoogle Scholar
  20. 20.
    Munaro, M., Ghidoni, S., Dizmen, D.T., Menegatti, E.: A feature-based approach to people re-identification using skeleton keypoints. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5644–5651. IEEE, May 2014Google Scholar
  21. 21.
    Zhao, Q., Tan, P., Dai, Q., Shen, L., Wu, E., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1437–1444 (2012)CrossRefGoogle Scholar
  22. 22.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  23. 23.
    Saini, S., Sakurikar, P., Narayanan, P.J.: Intrinsic image decomposition using focal stacks. In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing, p. 88. ACM, December 2016Google Scholar
  24. 24.
    Han, G., Xie, X., Lai, J., Zheng, W.S.: Learning an intrinsic image decomposer using synthesized RGB-D dataset. IEEE Signal Process. Lett. 25(6), 753–757 (2018)CrossRefGoogle Scholar
  25. 25.
    Nestmeyer, T., Gehler, P.V.: Reflectance adaptive filtering improves intrinsic image estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6789–6798 (2017)Google Scholar
  26. 26.
    Jiang, X., Pan, Q., Zheng, Y., Feng, X.: Intrinsic image extraction based on deconvolutional neural networks. In: 2017 International Conference on the Frontiers and Advances in Data Science (FADS), pp. 141–146. IEEE, October 2017Google Scholar
  27. 27.
    Jin, X., Gu, Y.: Superpixel-based intrinsic image decomposition of hyperspectral images. IEEE Trans. Geosci. Remote Sens. 55(8), 4285–4295 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of MalayaKuala LumpurMalaysia

Personalised recommendations