A Hierarchical Level Set Approach to for RGBD Image Matting

  • Wenliang ZengEmail author
  • Ji Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


This paper presents a novel method for the image matting of RGBD data, using a Hierarchical Level Set. The approach has four main steps. First of all, the color and depth channel is preprocessed. Noise is eliminated by using a Directional Joint Bilateral Filter and holes are removed from the depth map. Secondly, color cues and depth cues are integrated to segment the image using a Hierarchical Level Set Framework. After this, the segmentation of the color and depth cues is used to generate a trimap. Finally, an extended alpha matting approach is used to obtain the final matting result, with the color image, depth image and trimap serving as input. Experiments using complex natural images demonstrate that the proposed RGBD matting approach is able to generate good matting results.


Image matting Depth image Hierarchical level set 



This work is supported by the National Natural Science Foundation of China (No. 61502060), National Natural Science Foundation of China (No. 61701051).


  1. 1.
    Smith, A.R., Blinn, J.F.: Blue screen matting. In: International Conference on Computer Graphics and Interactive Techniques, pp. 259–268 (1996)Google Scholar
  2. 2.
    Naqvi, S.S., Browne, W.N., Hollitt, C.: Salient object detection via spectral matting. Pattern Recogn. 51, 209–224 (2016)CrossRefGoogle Scholar
  3. 3.
    Chuang, Y., Curless, B., Salesin, D., Szeliski, R.: A Bayesian approach to digital matting, vol. 2, pp. 264–271 (2001)Google Scholar
  4. 4.
    Sun, J., Jia, J., Tang, C., Shum, H.: Poisson matting. In: International Conference on Computer Graphics and Interactive Techniques (2004)CrossRefGoogle Scholar
  5. 5.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)CrossRefGoogle Scholar
  6. 6.
    Cho, J., Ziegler, R., Gross, M.H., Lee, K.H.: Improving alpha matte with depth information. IEICE Electron. Express 6(22), 1602–1607 (2009)CrossRefGoogle Scholar
  7. 7.
    Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Computer Graphics Forum, vol. 29, no. 2, pp. 575–584 (2010)CrossRefGoogle Scholar
  8. 8.
    Pollefeys, M., Aksoy, Y., Aydin, T.O.: Designing effective inter-pixel information flow for natural image matting (2017)Google Scholar
  9. 9.
    Crabb, R., Tracey, C., Puranik, A., Davis, J.: Real-time foreground segmentation via range and color imaging, pp. 1–5 (2008)Google Scholar
  10. 10.
    Wang, O., Finger, J., Yang, Q., Davis, J., Yang, R.: Automatic natural video matting with depth. In: Pacific Conference on Computer Graphics and Applications, pp. 469–472 (2007)Google Scholar
  11. 11.
    Pitie, F., Kokaram, A.: Matting with a depth map. In: IEEE International Conference on Image Processing, pp. 21–24 (2010)Google Scholar
  12. 12.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Xu, L., Sun, W., Au, O.C., et al.: Adaptive depth map assisted matting in 3D video. In: IEEE International Conference on Multimedia and Expo, pp. 1–6 (2011)Google Scholar
  14. 14.
    Lee, S.W., Seo, Y.H., Yang, H.S.: Efficient foreground extraction using RGB-D imaging. Kluwer Academic Publishers (2016)Google Scholar
  15. 15.
    Wang, L., Gong, M., Zhang, C., Yang, R., Zhang, C., Yang, Y.H.: Automatic real-time video matting using time-of-flight camera and multichannel poisson equations. Int. J. Comput. Vision 97(1), 104–121 (2012)CrossRefGoogle Scholar
  16. 16.
    Ge, L., Ju, R., Ren, T., Wu, G.: Interactive RGB-D image segmentation using hierarchical graph cut and geodesic distance. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9314, pp. 114–124. Springer, Cham (2015). Scholar
  17. 17.
    Lu, T., Li, S.: Image matting with color and depth information, pp. 3787–3790 (2012)Google Scholar
  18. 18.
    Memar, S., Jin, K., Boufama, B.: Object detection using active contour model with depth clue. In: Kamel, M., Campilho, A. (eds.) ICIAR 2013. LNCS, vol. 7950, pp. 640–647. Springer, Heidelberg (2013). Scholar
  19. 19.
    Hao, W., Zheng, S., Guo, C., Xie, Y.: Level set contour extraction based on data-adaptive Gaussian smoother, pp. 11–15 (2012)Google Scholar
  20. 20.
    Hu, P., Shuai, B., Liu, J., Wang, G.: Deep level sets for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  21. 21.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefGoogle Scholar
  22. 22.
    Zanuttigh, P., Marin, G., Dal Mutto, C., Dominio, F., Minto, L., Cortelazzo, G.M.: Time-of-Flight and Structured Light Depth Cameras. Springer, Cham (2016). Scholar
  23. 23.
    Chen, L., Lin, H., Li, S.: Depth image enhancement for kinect using region growing and bilateral filter. In: International Conference on Pattern Recognition, pp. 3070-3073 (2013)Google Scholar
  24. 24.
    Le, A.V., Jung, S., Won, C.S.: Directional joint bilateral filter for depth images. Sensors 14(7), 11362–11378 (2014)CrossRefGoogle Scholar
  25. 25.
    Jung, S.: Enhancement of image and depth map using adaptive joint trilateral filter. IEEE Trans. Circuits Syst. Video Technol. 23(2), 269–280 (2013)CrossRefGoogle Scholar
  26. 26.
    Li, C., Xu, C., Gui, C., Fox, M.D.: Level set evolution without re-initialization: a new variational formulation. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 430–436 (2005)Google Scholar
  27. 27.
    Ju, R., Liu, Y., Ren, T., Ge, L., Wu, G.: Depth-aware salient object detection using anisotropic center-surround difference. Signal Process. Image Commun. 38(C), 115–126 (2015)CrossRefGoogle Scholar
  28. 28.
    Leens, J., Piérard, S., Barnich, O., Van Droogenbroeck, M., Wagner, J.-M.: Combining color, depth, and motion for video segmentation. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 104–113. Springer, Heidelberg (2009). Scholar
  29. 29.
    Varnousfaderani, E.S., Rajan, D.: Weighted color and texture sample selection for image matting. IEEE Trans. Image Process. 22(11), 4260–4270 (2013)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Li, C., Wang, P., Zhu, X., Pi, H.: Three-layer graph framework with the sumD feature for alpha matting. Comput. Vis. Image Underst. 162, 34–45 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.College of Computer Science, Chongqing UniversityChongqingChina

Personalised recommendations