Advertisement

Detecting Ground Shadows in Outdoor Consumer Photographs

  • Jean-François Lalonde
  • Alexei A. Efros
  • Srinivasa G. Narasimhan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)

Abstract

Detecting shadows from images can significantly improve the performance of several vision tasks such as object detection and tracking. Recent approaches have mainly used illumination invariants which can fail severely when the qualities of the images are not very good, as is the case for most consumer-grade photographs, like those on Google or Flickr. We present a practical algorithm to automatically detect shadows cast by objects onto the ground, from a single consumer photograph. Our key hypothesis is that the types of materials constituting the ground in outdoor scenes is relatively limited, most commonly including asphalt, brick, stone, mud, grass, concrete, etc. As a result, the appearances of shadows on the ground are not as widely varying as general shadows and thus, can be learned from a labelled set of images. Our detector consists of a three-tier process including (a) training a decision tree classifier on a set of shadow sensitive features computed around each image edge, (b) a CRF-based optimization to group detected shadow edges to generate coherent shadow contours, and (c) incorporating any existing classifier that is specifically trained to detect grounds in images. Our results demonstrate good detection accuracy (85%) on several challenging images. Since most objects of interest to vision applications (like pedestrians, vehicles, signs) are attached to the ground, we believe that our detector can find wide applicability.

Keywords

Outdoor Scene Shadow Detection Ground Color Shadow Boundary Invariant Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lalonde, J.F., Efros, A.A., Narasimhan, S.G.: Illumination estimation from a single outdoor image. In: IEEE International Conference on Computer Vision (2009)Google Scholar
  2. 2.
    Sato, I., Sato, Y., Ikeuchi, K.: Illumination from shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (2003)Google Scholar
  3. 3.
    Matsushita, Y., Nishino, K., Ikeuchi, K., Sakauchi, M.: Illumination normalization with time-dependent intrinsic images for video surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (2004)Google Scholar
  4. 4.
    Finlayson, G.D., Fredembach, C., Drew, M.S.: Detecting illumination in images. In: IEEE International Conference on Computer Vision (2007)Google Scholar
  5. 5.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: IEEE International Conference on Computer Vision (2001)Google Scholar
  6. 6.
    Huerta, I., Holte, M., Moeslund, T., Gonzàlez, J.: Detection and removal of chromatic moving shadows in surveillance scenarios. In: IEEE International Conference on Computer Vision (2009)Google Scholar
  7. 7.
    Wu, T.P., Tang, C.K.: A bayesian approach for shadow extraction from a single image. In: IEEE International Conference on Computer Vision (2005)Google Scholar
  8. 8.
    Bousseau, A., Paris, S., Durand, F.: User-assisted intrinsic images. ACM Transactions on Graphics (SIGGRAPH Asia 2009) 28 (2009)Google Scholar
  9. 9.
    Shor, Y., Lischinski, D.: The shadow meets the mask: pyramid-based shadow removal. Computer Graphics Forum Journal (Eurographics 2008) 27 (2008)Google Scholar
  10. 10.
    Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 146–160. Springer, Heidelberg (2002)Google Scholar
  11. 11.
    Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 582–595. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. International Journal of Computer Vision 85 (2009)Google Scholar
  13. 13.
    Maxwell, B.A., Friedhoff, R.M., Smith, C.A.: A bi-illuminant dichromatic reflection model for understanding images. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  14. 14.
    Tian, J., Sun, J., Tang, Y.: Tricolor attenuation model for shadow detection. IEEE Transactions on Image Processing 18 (2009)Google Scholar
  15. 15.
    Narasimhan, S.G., Ramesh, V., Nayar, S.K.: A class of photometric invariants: Separating material from shape and illumination. In: IEEE International Conference on Computer Vision (2005)Google Scholar
  16. 16.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision 77 (2008)Google Scholar
  17. 17.
    Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. International Journal of Computer Vision 40 (2000)Google Scholar
  18. 18.
    Hoiem, D., Efros, A.A., Hebert, M.: Recovering surface layout from an image. International Journal of Computer Vision 75 (2007)Google Scholar
  19. 19.
    Zhu, J., Samuel, K.G.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  20. 20.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the 6th International Conference on Computer Vision (1998)Google Scholar
  21. 21.
    Barnard, K., Finlayson, G.D.: Shadow identification using colour ratios. In: Proc. IS&T/SID 8th Color Imaging Conf. Color Science, Systems and Applications (2000)Google Scholar
  22. 22.
    Khan, E.A., Reinhard, E.: Evaluation of color spaces for edge classification in outdoor scenes. In: IEEE International Conference on Image Processing (2005)Google Scholar
  23. 23.
    Chong, H.Y., Gortler, S.J., Zickler, T.: A perception-based color space for illumination-invariant image processing. ACM Transactions on Graphics, SIGGRAPH 2008 (2008)Google Scholar
  24. 24.
    Collins, M., Shapire, R., Singer, Y.: Logistic regression, adaboost and Bregman distances. Machine Learning 48 (2002)Google Scholar
  25. 25.
    Hoiem, D., Stein, A., Efros, A.A., Hebert, M.: Recovering occlusion boundaries from a single image. In: IEEE International Conference on Computer Vision (2007)Google Scholar
  26. 26.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: IEEE International Conference on Computer Vision (2001)Google Scholar
  27. 27.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (2001)Google Scholar
  28. 28.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (2004)Google Scholar
  29. 29.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transaction on Pattern Analysis and Machine Intelligence 26 (2004)Google Scholar
  30. 30.
    Sinha, P., Adelson, E.H.: Recovering reflectance and illumination in a world of painted polyhedra. In: IEEE International Conference on Computer Vision (1993)Google Scholar
  31. 31.
    Chakrabarti, A., Scharstein, D., Zickler, T.: An empirical camera model for internet color vision. In: British Machine Vision Conference (2009)Google Scholar
  32. 32.
    Fredembach, C., Finlayson, G.D.: Simple shadow removal. In: International Conference on Pattern Recognition (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jean-François Lalonde
    • 1
  • Alexei A. Efros
    • 1
  • Srinivasa G. Narasimhan
    • 1
  1. 1.School of Computer ScienceCarnegie Mellon University 

Personalised recommendations