Consistent Matting for Light Field Images

  • Donghyeon Cho
  • Sunyeong Kim
  • Yu-Wing Tai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)


We present a new image matting algorithm to extract consistent alpha mattes across sub-images of a light field image. Instead of matting each sub-image individually, our approach utilizes the epipolar plane image (EPI) to construct comprehensive foreground and background sample sets across the sub-images without missing a true sample. The sample sets represent all color variation of foreground and background in a light field image, and the optimal alpha matte is obtained by choosing the best combination of foreground and background samples that minimizes the linear composite error subject to the EPI correspondence constraint. To further preserve consistency of the estimated alpha mattes across different sub-images, we impose a smoothness constraint along the EPI of alpha mattes. In experimental evaluations, we have created a dataset where the ground truth alpha mattes of light field images were obtained by using the blue screen technique. A variety of experiments show that our proposed algorithm produces both visually and quantitatively high-quality matting results for light field images.


Image Matting Light field image EPI 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bishop, T.E., Zanetti, S., Favaro, P.: Plenoptic depth estimation from multiple aliased views. In: IEEE ICCV Workshops (2009)Google Scholar
  2. 2.
    Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: An approach to determining structure from motion. IJCV 1(1), 7–55 (1987)CrossRefGoogle Scholar
  3. 3.
    Chen, Q., Li, D., Tang, C.K.: Knn matting. In: IEEE CVPR (2012)Google Scholar
  4. 4.
    Chen, X., Zou, D., Zhou, S.Z., Zhao, Q., Tan, P.: Image matting with local and nonlocal smooth priors. In: IEEE CVPR (2013)Google Scholar
  5. 5.
    Cho, D., Lee, M., Kim, S., Tai, Y.W.: Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction. In: IEEE ICCV (2013)Google Scholar
  6. 6.
    Choi, I., Lee, M., Tai, Y.-W.: Video matting using multi-frame nonlocal matting laplacian. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 540–553. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  7. 7.
    Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A bayesian approach to digital matting. In: IEEE CVPR (2001)Google Scholar
  8. 8.
    Criminisi, A., Kang, S.B., Swaminathan, R., Szeliski, R., Anandan, P.: Extracting layers and analyzing their specular properties using epipolar-plane-image analysis. Computer Vision and Image Understanding (CVIU) 97(1), 51–85 (2005)CrossRefGoogle Scholar
  9. 9.
    Dansereau, D.G., Bruton, L.T.: Gradient-based depth estimation from 4d light fields. In: IEEE International Symposium on Circuits and Systems (ISCAS) (2004)Google Scholar
  10. 10.
    Dansereau, D.G., Pizarro, O., Williams, S.B.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: IEEE CVPR (2013)Google Scholar
  11. 11.
    Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Eurographics (2010)Google Scholar
  12. 12.
    Goldluecke, B., Wanner, S.: The variational structure of disparity and regularization of 4d light fields. In: IEEE CVPR (2013)Google Scholar
  13. 13.
    He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: IEEE CVPR (2011)Google Scholar
  14. 14.
    He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: IEEE CVPR (2010)Google Scholar
  15. 15.
    Joshi, N., Matusik, W., Avidan, S.: Natural video matting using camera arrays. In: ACM SIGGRAPH (2006)Google Scholar
  16. 16.
    Lee, P., Wu, Y.: Nonlocal matting. In: IEEE CVPR (2011)Google Scholar
  17. 17.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. on PAMI 30(2), 162–8828 (2008)CrossRefGoogle Scholar
  18. 18.
    Lin, H., Tai, Y.W., Brown, M.S.: Motion regularization for matting motion blurred objects. IEEE Trans. on PAMI 33(11), 2329–2336 (2011)CrossRefGoogle Scholar
  19. 19.
    Lytro: The lytro camera,
  20. 20.
    Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Tech. rep. (2005)Google Scholar
  21. 21.
    Rhemann, C., Rother, C., Gelautz, M.: Improving color modeling for alpha matting. In: British Machine Vision Conference (BMVC) (2008)Google Scholar
  22. 22.
    Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: IEEE CVPR (2009)Google Scholar
  23. 23.
    Shahrian, E., Rajan, D., Price, B., Cohen, S.: Improving image matting using comprehensive sampling sets. In: IEEE CVPR (2013)Google Scholar
  24. 24.
    Shahrian, E., Rajan, D.: Weighted color and texture sample selection for image matting. In: IEEE CVPR (2012)Google Scholar
  25. 25.
    Smith, A.R., Blinn, J.F.: Blue screen matting. In: ACM SIGGRAPH (1996)Google Scholar
  26. 26.
    Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. ACM TOG 23(3), 315–321 (2004)CrossRefGoogle Scholar
  27. 27.
    Tao, M.W., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: IEEE ICCV (2013)Google Scholar
  28. 28.
    Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: IEEE CVPR (2007)Google Scholar
  29. 29.
    Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4D lightfields. In: IEEE CVPR (2012)Google Scholar
  30. 30.
    Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  31. 31.
    Wanner, S., Straehle, C., Goldluecke, B.: Globally consistent multi-label assignment on the ray space of 4d light fields. In: IEEE CVPR (2013)Google Scholar
  32. 32.
    Zheng, Y., Kambhamettu, C.: Learning based digital matting. In: IEEE ICCV (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Donghyeon Cho
    • 1
  • Sunyeong Kim
    • 1
  • Yu-Wing Tai
    • 1
  1. 1.Korea Advanced Institute of Science and Technology (KAIST)Korea

Personalised recommendations