Abstract
We present a new image matting algorithm to extract consistent alpha mattes across sub-images of a light field image. Instead of matting each sub-image individually, our approach utilizes the epipolar plane image (EPI) to construct comprehensive foreground and background sample sets across the sub-images without missing a true sample. The sample sets represent all color variation of foreground and background in a light field image, and the optimal alpha matte is obtained by choosing the best combination of foreground and background samples that minimizes the linear composite error subject to the EPI correspondence constraint. To further preserve consistency of the estimated alpha mattes across different sub-images, we impose a smoothness constraint along the EPI of alpha mattes. In experimental evaluations, we have created a dataset where the ground truth alpha mattes of light field images were obtained by using the blue screen technique. A variety of experiments show that our proposed algorithm produces both visually and quantitatively high-quality matting results for light field images.
Chapter PDF
Similar content being viewed by others
References
Bishop, T.E., Zanetti, S., Favaro, P.: Plenoptic depth estimation from multiple aliased views. In: IEEE ICCV Workshops (2009)
Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: An approach to determining structure from motion. IJCV 1(1), 7–55 (1987)
Chen, Q., Li, D., Tang, C.K.: Knn matting. In: IEEE CVPR (2012)
Chen, X., Zou, D., Zhou, S.Z., Zhao, Q., Tan, P.: Image matting with local and nonlocal smooth priors. In: IEEE CVPR (2013)
Cho, D., Lee, M., Kim, S., Tai, Y.W.: Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction. In: IEEE ICCV (2013)
Choi, I., Lee, M., Tai, Y.-W.: Video matting using multi-frame nonlocal matting laplacian. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 540–553. Springer, Heidelberg (2012)
Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A bayesian approach to digital matting. In: IEEE CVPR (2001)
Criminisi, A., Kang, S.B., Swaminathan, R., Szeliski, R., Anandan, P.: Extracting layers and analyzing their specular properties using epipolar-plane-image analysis. Computer Vision and Image Understanding (CVIU) 97(1), 51–85 (2005)
Dansereau, D.G., Bruton, L.T.: Gradient-based depth estimation from 4d light fields. In: IEEE International Symposium on Circuits and Systems (ISCAS) (2004)
Dansereau, D.G., Pizarro, O., Williams, S.B.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: IEEE CVPR (2013)
Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Eurographics (2010)
Goldluecke, B., Wanner, S.: The variational structure of disparity and regularization of 4d light fields. In: IEEE CVPR (2013)
He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: IEEE CVPR (2011)
He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: IEEE CVPR (2010)
Joshi, N., Matusik, W., Avidan, S.: Natural video matting using camera arrays. In: ACM SIGGRAPH (2006)
Lee, P., Wu, Y.: Nonlocal matting. In: IEEE CVPR (2011)
Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. on PAMI 30(2), 162–8828 (2008)
Lin, H., Tai, Y.W., Brown, M.S.: Motion regularization for matting motion blurred objects. IEEE Trans. on PAMI 33(11), 2329–2336 (2011)
Lytro: The lytro camera, https://www.lytro.com
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Tech. rep. (2005)
Rhemann, C., Rother, C., Gelautz, M.: Improving color modeling for alpha matting. In: British Machine Vision Conference (BMVC) (2008)
Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: IEEE CVPR (2009)
Shahrian, E., Rajan, D., Price, B., Cohen, S.: Improving image matting using comprehensive sampling sets. In: IEEE CVPR (2013)
Shahrian, E., Rajan, D.: Weighted color and texture sample selection for image matting. In: IEEE CVPR (2012)
Smith, A.R., Blinn, J.F.: Blue screen matting. In: ACM SIGGRAPH (1996)
Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. ACM TOG 23(3), 315–321 (2004)
Tao, M.W., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: IEEE ICCV (2013)
Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: IEEE CVPR (2007)
Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4D lightfields. In: IEEE CVPR (2012)
Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013)
Wanner, S., Straehle, C., Goldluecke, B.: Globally consistent multi-label assignment on the ray space of 4d light fields. In: IEEE CVPR (2013)
Zheng, Y., Kambhamettu, C.: Learning based digital matting. In: IEEE ICCV (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Cho, D., Kim, S., Tai, YW. (2014). Consistent Matting for Light Field Images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8692. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-10593-2_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10592-5
Online ISBN: 978-3-319-10593-2
eBook Packages: Computer ScienceComputer Science (R0)