Abstract
This paper presents a novel generalization of the optical flow equation to the case of refraction, and it describes a method for recovering the refractive structure of an object from a video sequence acquired as the background behind the refracting object moves. By structure here we mean a representation of how the object warps and attenuates (or amplifies) the light passing through it. We distinguish between the cases when the background motion is known and unknown. We show that when the motion is unknown, the refractive structure can only be estimated up to a six-parameter family of solutions without additional sources of information. Methods for solving for the refractive structure are described in both cases. The performance of the algorithm is demonstrated on real data, and results of applying the estimated refractive structure to the task of environment matting and compositing are presented.
Chapter PDF
References
Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motion. International Journal of Computer Vision 12, 5–16 (1994)
Ju, S.X., Black, M.J., Jepson, A.D.: Skin and bones: Multilayer, locally affine, optical flow and regularization with transparency. In: CVPR 1996, pp. 307–314 (1996)
Levin, A., Zomet, A., Weiss, Y.: Learning to perceive transparency from the statistics of natural scenes. In: Neural Information Processing Systems (2002)
Schechner, Y., Kiryati, N., Shamir, J.: Blind recovery of transparent and semireflected scenes. In: CVPR 2000, vol. 1, pp. 38–43. IEEE Computer Society, Los Alamitos (2000)
Chuang, Y.Y., Zongker, D.E., Hindorff, J., Curless, B., Salesin, D.H., Szeliski, R.: Environment matting extensions: Towards higher accuracy and real-time capture. In: Proc. of ACM SIGGRAPH 2000, pp. 121–130. ACM Press, New York (2000)
Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Image-based environment matting. In: Proc. of the 13th Eurographics workshop on Rendering (2002)
Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.H.: Environment matting and compositing. In: Proc. of ACM SIGGRAPH 1999, pp. 205–214. ACM Press, New York (1999)
Murase, H.: Surface shape reconstruction of a nonrigid transparent object using refraction and motion. IEEE Trans. on Pattern Analysis and Machine Intelligence 14, 1045–1052 (1992)
Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17 (1981)
Horn, B.: Robot Vision. MIT Press, Cambridge (1986)
Perona, P., Malik, J., Shiota, T.: Anisotropic Diffusion. In: Geometry-Driven Diffusion in Computer Vision, pp. 73–92. Kluwer, Amsterdam (1995)
Savarese, S., Perona, P.: Local analysis for 3D reconstruction of specular surfaces. In: CVPR 2001, pp. 738–745 (2001)
Savarese, S., Perona, P.: Local analysis for 3D reconstruction of specular surfaces: Part II. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 759–774. Springer, Heidelberg (2002)
Oren, M., Nayar, S.: A theory of specular surface geometry. IJCV 24, 105–124 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Agarwal, S., Mallick, S.P., Kriegman, D., Belongie, S. (2004). On Refractive Optical Flow. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3022. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24671-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-540-24671-8_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21983-5
Online ISBN: 978-3-540-24671-8
eBook Packages: Springer Book Archive