On the Motion and Appearance of Specularities in Image Sequences

  • Rahul Swaminathan
  • Sing Bing Kang
  • Richard Szeliski
  • Antonio Criminisi
  • Shree K. Nayar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2350)


Real scenes are full of specularities (highlights and reflections), and yet most vision algorithms ignore them. In order to capture the appearance of realistic scenes, we need to model specularities as separate layers. In this paper, we study the behavior of specularities in static scenes as the camera moves, and describe their dependence on varying surface geometry, orientation, and scene point and camera locations. For a rectilinear camera motion with constant velocity, we study how the specular motion deviates from a straight trajectory (disparity deviation) and how much it violates the epipolar constraint (epipolar deviation). Surprisingly, for surfaces that are convex or not highly undulating, these deviations are usually quite small. We also study the appearance of specularities, i.e., how they interact with the body reflection, and with the usual occlusion ordering constraints applicable to diffuse opaque layers. We present a taxonomy of specularities based on their photometric properties as a guide for designing separation techniques. Finally, we propose a technique to extract specularities as a separate layer, and demonstrate it using an image sequence of a complex scene.


  1. [BB88]
    A. Blake and G. Brelstaff. Geometry from specularity. Proc. International Conference on Computer Vision, pages 394–403, 1988.Google Scholar
  2. [BB89]
    H. H. Baker and R. C. Bolles. Generalizing epipolar-plane image analysis on the spatiotemporal surface. International Journal of Computer Vision, 3(1):33–49, 1989.CrossRefGoogle Scholar
  3. [BG84]
    J. W. Bruce and P. J. Giblin. Curves and Singularities. Cambridge University Press, Cambridge, 1984.zbMATHGoogle Scholar
  4. [BLL96]
    R.K. Bajcsy, S.W. Lee, and A. Leonardis. Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. International Journal of Computer Vision, 17(3):241–272, March 1996.Google Scholar
  5. [BN98]
    D. N. Bhat and S. K. Nayar. Stereo and specular reflection. International Journal of Computer Vision, 26(2):91–106, February 1998.Google Scholar
  6. [BS73]
    D. G. Burkhard and D. L. Shealy. Flux Density for Ray Propagation in Geometrical Optics. Journal of the Optical Society of America, 63(3):299–304, March 1973.Google Scholar
  7. [CKSS02]
    A. Criminisi, S.B. Kang, R. Swaminathan, and R. Szeliski. Separating Diffuse and Specular Layers using EPI Analysis, MSR-TR(in preparation). Technical report, Microsoft Research, 2002.Google Scholar
  8. [FA99]
    H. Farid and E.H. Adelson. Separating reflections and lighting using independent components analysis. In Proc. Computer Vision and Pattern Recognition, pages I:262–267, 1999.Google Scholar
  9. [Ham28]
    W. R. Hamilton. Theory of Systems of Rays. Transactions of the Royal Irish Academy, 15:69–174, 1828.Google Scholar
  10. [HB88]
    G. Healey and T. O. Binford. Local shape from specularity. CVGIP, pages 62–86, 1988.Google Scholar
  11. [Ike81]
    K. Ikeuchi. Determining surface orientation of specular surfaces by using the photometric stereo method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3(6):661–669, 1981.CrossRefGoogle Scholar
  12. [IRP94]
    M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5–16, January 1994.Google Scholar
  13. [Kem01]
    M. Kemp. Visualizations: The Nature Book of Art and Science. University of California Press, 2001.Google Scholar
  14. [KSK88]
    G. J. Klinker, S. A. Shafer, and T. Kanade. The measurement of highlights in color images. International Journal of Computer Vision, 2(1):7–32, June 1988.Google Scholar
  15. [LB92]
    S.W. Lee and R. Bajcsy. Detection of specularity using colour and multiple views. In Proc. European Conference on Computer Vision, pages 99–114, 1992.Google Scholar
  16. [LS97]
    J. Lengyel and J. Snyder. Rendering with coherent layers. In Computer Graphics Proceedings, Annual Conference Series, pages 233–242, Proc. SIGGRAPH’97 (Los Angeles), August 1997. ACM SIGGRAPH.Google Scholar
  17. [NFB97]
    S.K. Nayar, X. S. Fang, and T. Boult. Separation of reflectance components using color and polarization. International Journal of Computer Vision, 21(3):163–186, February 1997.Google Scholar
  18. [ON97]
    M. Oren and S. Nayar. A theory of specular surface geometry. International Journal of Computer Vision, 24(2):105–124, September 1997.Google Scholar
  19. [SAA00]
    R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In Proc. Conference on Computer Vision and Pattern Recognition, volume 1, pages 246–253, June 2000.Google Scholar
  20. [Sch94]
    H. Schultz. Retrieving shape information from multiple images of a specular surface. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):195–201, February 1994.Google Scholar
  21. [SGN01]
    R. Swaminathan, M. D. Grossberg, and S. K. Nayar. Caustics of Catadioptric Cameras. In Proc. International Conference on Computer Vision, pages II:2–9, July 2001.Google Scholar
  22. [SHW92]
    S. A. Shafer, G. Healey, and L. Wolff. Physics-Based Vision: Principles and Practice. Jones & Bartlett, Cambridge, MA, 1992.Google Scholar
  23. [SSK98]
    Y. Schechner, J. Shamir, and N. Kiryati. Polarization-based decorrelation of transparent layers: The inclination angle of an invisible surface. In Proc. International Conference on Computer Vision, pages 814–819, 1998.Google Scholar
  24. [Wei01]
    Y. Weiss. Deriving intrinsic images from image sequences. In Proc. International Conference on Computer Vision, pages II: 68–75, 2001.Google Scholar
  25. [Wol90]
    L.B. Wolff. Polarization-based material classification from specular reflection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(11):1059–1071, November 1990.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Rahul Swaminathan
    • 1
  • Sing Bing Kang
    • 2
  • Richard Szeliski
    • 2
  • Antonio Criminisi
    • 2
  • Shree K. Nayar
    • 1
  1. 1.Columbia UniversityUSA
  2. 2.Microsoft ResearchRedmondUSA

Personalised recommendations