Abstract
The problem of accurate video layer decomposition is of vital importance in computer vision. Previous methods mainly focus on the foreground extraction. In this paper, we present a user-assisted framework to decompose videos and extract all layers, which is built on the depth information and over-segmented patches. The task is split into two stages: i) the clustering of over-segmented patches; ii) the propagation of layers along the video. Correspondingly, this paper has two contributions: i) a video decomposition method based on greedy over-segmented patches merging; ii) a layer propagation method via iteratively updating color Gaussian Mixture Models(GMM). We test this algorithm on real videos and verify that it outperforms state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Darrell, T., Pentland, A.: Cooperative robust estimation using layers of support. IEEE Trans. on Pattern Analysis and Machine Intelligence 17, 474–487 (1991)
Wang, J., Adelson, E.: Representing moving images with layers. IEEE Trans. on Image Processing Special Issue: Image Sequence Compression 3, 625–638 (1994)
Ke, Q., Kanade, T.: Robust subspace clustering by combined use of knnd metric and svd algorithm. In: CVPR (2004)
Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary region segmentation of objects in n-d images. In: ICCV (2001)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 1222–1239 (2001)
Rother, C., Kolmogorov, V., Blake, A.: “grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans. on Graphics 23, 309–314 (2004)
Galun, M., Sharon, E., Basri, R., Br, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV (2003)
Riklin-raviv, T., Kiryati, N., Sochen, N.: Segmentation by level sets and symmetry. In: CVPR (2006)
Bagon, S., Boiman, O., Irani, M.: What is a good image segment? A unified approach to segment extraction. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 30–44. Springer, Heidelberg (2008)
Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: CVPR (2009)
Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: Robust video object cutout using localized classifiers. In: SIGGRAPH (2009)
Li, Y., Sun, J., Shum, H.: Video object cut and paste. ACM Trans. on Graphics 24, 595–600 (2005)
Zhu, J., Liao, M., Yang, R., Pan, Z.: Joint depth and alpha matte optimization via fusion of stereo and time-of-flight sensor. In: CVPR (2009)
Xiao, J., Shah, M.: Accurate motion layer segmentation and matting. In: CVPR (2005)
Zhang, G., Dong, Z., Jia, J., Wan, L., Wong, T., Bao, H.: Refilming with depth-inferred videos. IEEE Trans. on Visualization and Computer Graphics 15, 828–840 (2009)
Sun, J., Jia, J., Tang, C., Shum, H.: Poisson matting. ACM Trans. on Graphics 23, 315–321 (2004)
Chuang, Y., Agarwala, A., Curless, B., Salesin, D., Szeliski, R.: Video matting of complex scenes. ACM Trans. on Graphics 21, 243–248 (2002)
Lhuillier, M., Quan, L.: Match propagation for image-based modeling and rendering. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 1140–1146 (2002)
Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions via graph cuts. In: ICCV, pp. 508–515 (2001)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. of Computer Vision 70, 109–131 (2004)
Li, Y., Sun, J., Tang, C., Shum, H.: Lazy snapping. ACM Trans. on Graphics 23, 303–308 (2004)
Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A bayesian approach to digital matting. In: CVPR (2001)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. of Computer Vision 47, 7–42 (2002)
Khan, S., Shah, M.: Object based segmentation of video using color, motion and spatial information. In: CVPR (2001)
Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cut. In: CVPR (2004)
Dupont, R., Paragios, N., Keriven, R., Fuchs, P.: Extraction of layers of similar motion through combinatorial techniques. In: Rangarajan, A., Vemuri, B.C., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 220–234. Springer, Heidelberg (2005)
Schoenemann, T., Cremers, D.: High resolution motion layer decomposition using dual-space graph cuts. In: CVPR (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, Y., Zhou, Z., Wu, W. (2011). Interactive Video Layer Decomposition and Matting. In: Koch, R., Huang, F. (eds) Computer Vision – ACCV 2010 Workshops. ACCV 2010. Lecture Notes in Computer Science, vol 6469. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22819-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-22819-3_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22818-6
Online ISBN: 978-3-642-22819-3
eBook Packages: Computer ScienceComputer Science (R0)