Abstract
Real scenes are full of specularities (highlights and reflections), and yet most vision algorithms ignore them. In order to capture the appearance of realistic scenes, we need to model specularities as separate layers. In this paper, we study the behavior of specularities in static scenes as the camera moves, and describe their dependence on varying surface geometry, orientation, and scene point and camera locations. For a rectilinear camera motion with constant velocity, we study how the specular motion deviates from a straight trajectory (disparity deviation) and how much it violates the epipolar constraint (epipolar deviation). Surprisingly, for surfaces that are convex or not highly undulating, these deviations are usually quite small. We also study the appearance of specularities, i.e., how they interact with the body reflection, and with the usual occlusion ordering constraints applicable to diffuse opaque layers. We present a taxonomy of specularities based on their photometric properties as a guide for designing separation techniques. Finally, we propose a technique to extract specularities as a separate layer, and demonstrate it using an image sequence of a complex scene.
Chapter PDF
Similar content being viewed by others
References
A. Blake and G. Brelstaff. Geometry from specularity. Proc. International Conference on Computer Vision, pages 394–403, 1988.
H. H. Baker and R. C. Bolles. Generalizing epipolar-plane image analysis on the spatiotemporal surface. International Journal of Computer Vision, 3(1):33–49, 1989.
J. W. Bruce and P. J. Giblin. Curves and Singularities. Cambridge University Press, Cambridge, 1984.
R.K. Bajcsy, S.W. Lee, and A. Leonardis. Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. International Journal of Computer Vision, 17(3):241–272, March 1996.
D. N. Bhat and S. K. Nayar. Stereo and specular reflection. International Journal of Computer Vision, 26(2):91–106, February 1998.
D. G. Burkhard and D. L. Shealy. Flux Density for Ray Propagation in Geometrical Optics. Journal of the Optical Society of America, 63(3):299–304, March 1973.
A. Criminisi, S.B. Kang, R. Swaminathan, and R. Szeliski. Separating Diffuse and Specular Layers using EPI Analysis, MSR-TR(in preparation). Technical report, Microsoft Research, 2002.
H. Farid and E.H. Adelson. Separating reflections and lighting using independent components analysis. In Proc. Computer Vision and Pattern Recognition, pages I:262–267, 1999.
W. R. Hamilton. Theory of Systems of Rays. Transactions of the Royal Irish Academy, 15:69–174, 1828.
G. Healey and T. O. Binford. Local shape from specularity. CVGIP, pages 62–86, 1988.
K. Ikeuchi. Determining surface orientation of specular surfaces by using the photometric stereo method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3(6):661–669, 1981.
M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5–16, January 1994.
M. Kemp. Visualizations: The Nature Book of Art and Science. University of California Press, 2001.
G. J. Klinker, S. A. Shafer, and T. Kanade. The measurement of highlights in color images. International Journal of Computer Vision, 2(1):7–32, June 1988.
S.W. Lee and R. Bajcsy. Detection of specularity using colour and multiple views. In Proc. European Conference on Computer Vision, pages 99–114, 1992.
J. Lengyel and J. Snyder. Rendering with coherent layers. In Computer Graphics Proceedings, Annual Conference Series, pages 233–242, Proc. SIGGRAPH’97 (Los Angeles), August 1997. ACM SIGGRAPH.
S.K. Nayar, X. S. Fang, and T. Boult. Separation of reflectance components using color and polarization. International Journal of Computer Vision, 21(3):163–186, February 1997.
M. Oren and S. Nayar. A theory of specular surface geometry. International Journal of Computer Vision, 24(2):105–124, September 1997.
R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In Proc. Conference on Computer Vision and Pattern Recognition, volume 1, pages 246–253, June 2000.
H. Schultz. Retrieving shape information from multiple images of a specular surface. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):195–201, February 1994.
R. Swaminathan, M. D. Grossberg, and S. K. Nayar. Caustics of Catadioptric Cameras. In Proc. International Conference on Computer Vision, pages II:2–9, July 2001.
S. A. Shafer, G. Healey, and L. Wolff. Physics-Based Vision: Principles and Practice. Jones & Bartlett, Cambridge, MA, 1992.
Y. Schechner, J. Shamir, and N. Kiryati. Polarization-based decorrelation of transparent layers: The inclination angle of an invisible surface. In Proc. International Conference on Computer Vision, pages 814–819, 1998.
Y. Weiss. Deriving intrinsic images from image sequences. In Proc. International Conference on Computer Vision, pages II: 68–75, 2001.
L.B. Wolff. Polarization-based material classification from specular reflection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(11):1059–1071, November 1990.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Swaminathan, R., Kang, S.B., Szeliski, R., Criminisi, A., Nayar, S.K. (2002). On the Motion and Appearance of Specularities in Image Sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2350. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47969-4_34
Download citation
DOI: https://doi.org/10.1007/3-540-47969-4_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43745-1
Online ISBN: 978-3-540-47969-7
eBook Packages: Springer Book Archive