Abstract
In this paper we present an approach for separating two transparent layers in images and video sequences. Given two initial unknown physical mixtures, I 1 and I 2, of real scene layers, L 1 and L 2, we seek a layer separation which minimizes the structural correlations across the two layers, at every image point. Such a separation is achieved by transferring local grayscale structure from one image to the other wherever it is highly correlated with the underlying local grayscale structure in the other image, and vice versa. This bi-directional transfer operation, which we call the “layer information exchange”, is performed on diminishing window sizes, from global image windows (i.e., the entire image), down to local image windows, thus detecting similar grayscale structures at varying scales across pixels. We show the applicability of this approach to various real-world scenarios, including image and video transparency separation. In particular, we show that this approach can be used for separating transparent layers in images obtained under different polarizations, as well as for separating complex non-rigid transparent motions in video sequences. These can be done without prior knowledge of the layer mixing model (simple additive, alpha-mated composition with an unknown alpha-map, or other), and under unknown complex temporal changes (e.g., unknown varying lighting conditions).
This research was supported in part by the Moross Laboratory at the Weizmann Institute of science.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical Model-Based Motion Estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)
Bronstein, A., Bronstein, M., Zibulevsky, M., Zeevi, Y.Y.: Separation of semireflective layers using Sparse ICA. In: Proc. ICASSP 2003, vol. 3, pp. 733–736 (2003)
Cover, T., Thomas, J.: Elements of Information Theory. Wiley and Sons, Chichester (1991)
Farid, H., Adelson, E.H.: Separating Reflections from Images by Use of Independent Components Analysis. JOSA 16(9), 2136–2145 (1999)
Irani, M., Anandan, P.: Robust Multi-Sensor Image Alignment. In: ICCV 1998, pp. 959–966 (1998)
Levin, A.: Zomet, and Y. Weiss: Learning to perceive transparency from the statistics of natural scenes. In: NIPS 2002, pp.1247–1254 (2002)
Schechner, Y.Y., Shamir, J., Kiryati, N.: Polarization and statistical analysis of scenes containing a semi-reflector. JOSA A 17, 276–284 (2000)
Szeliski, R., Avidan, S., Anandan, P.: Layer Extraction from Multiple Images containing Reflections and Transparency. In: CVPR 2000, pp. 246–253 (2000)
Tsin, Y., Kang, S.B., Szeliski, R.: Stereo Matching with Reflections and Translucency. In: CVPR 2003, 702–709 (2003)
Weiss, Y.: Deriving Intrinsic Images from Image Sequences. In: ICCV 2001, pp.68–75 (2001)
Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Bayesian Estimation of Layers from Multiple Images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 487–501. Springer, Heidelberg (2002)
FastICA package, downloaded from: http://www.cis.hut.fi/projects/ica/fastica/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sarel, B., Irani, M. (2004). Separating Transparent Layers through Layer Information Exchange. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3024. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24673-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-24673-2_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21981-1
Online ISBN: 978-3-540-24673-2
eBook Packages: Springer Book Archive