Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2014: Computer Vision -- ACCV 2014 pp 659–674Cite as

Clouds in the Cloud

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9006))

Abstract

Light-field imaging can be scaled up to a very large area, to map the Earth’s atmosphere in 3D. Multiview spaceborne instruments suffer low spatio-temporal-angular resolution, and are very expensive and unscalable. We develop sky light-field imaging, by a wide, scalable network of wide-angle cameras looking upwards, which upload their data to the cloud. This new type of imaging-system poses new computational vision and photography problems, some of which generalize prior monocular tasks. These include radiometric self-calibration across a network, overcoming flare by a network, and background estimation. On the other hand, network redundancy offers solutions to these problems, which we derive. Based on such solutions, the light-field network enables unprecedented ways to measure nature. We demonstrate this experimentally by 3D recovery of clouds, in high spatio-temporal resolution. It is achieved by space carving of the volumetric distribution of semi-transparent clouds. Such sensing can complement satellite imagery, be useful to meteorology, make aerosol tomography realizable, and give new, powerful tools to atmospheric and avian wildlife scientists.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    There are also ground viewing webcams that happen to see sky parts [13, 26] and weather cameras that are too sparse to be integrated for recovery.

  2. 2.

    Manual tracking of a special flight and long exposures at night were used in [44].

  3. 3.

    Higher than 20\(^{\circ }\) above the horizon [11], errors caused by atmospheric refraction are smaller than \(0.05^{\circ }\), much less than the angular size of each of our pixels, 0.18\(^{\circ }\).

  4. 4.

    Sun blocker was not used here, since saturation and blooming do not impair cloud shape reconstruction.

References

  1. Adelson, E., Wang, J.: Single lens stereo with a plenoptic camera. IEEE Trans. PAMI 14, 99–106 (1992)

    Article  Google Scholar 

  2. Aides, A., Schechner, Y.Y., Holodovsky, V., Garay, M.J., Davis, A.B.: Multi sky-view 3D aerosol distribution recovery. Opt. Express 21, 25820–25833 (2013)

    Article  Google Scholar 

  3. Allmen, M.C., Kegelmeyer Jr., P.: The computation of cloud base height from paired whole-sky imaging cameras. Mach. Vis. Appl. 9, 160–165 (1997)

    Article  Google Scholar 

  4. Alterman, M., Schechner, Y.Y., Swirski, Y.: Triangulation in random refractive distortions. In: Proceedings of the IEEE ICCP, pp. 1–10 (2013)

    Google Scholar 

  5. Alterman, M., Schechner, Y.Y., Vo, M., Narasimhan, S.G.: Passive tomography of turbulence strength. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 47–60. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  6. Alterman, M., Swirski, Y., Schechner, Y.Y.: STELLA MARIS: stellar marine refractive imaging sensor. In: Proceedings of the IEEE ICCP, pp. 1–10 (2014)

    Google Scholar 

  7. Atkinson, B.W.: Meso-Scale Atmospheric Circulations. Academic Press, New York (1989)

    Google Scholar 

  8. Basha, T., Avidan, S., Hornung, A., Matusik, W.: Structure and motion from scene registration. In: Proceedings of the IEEE CVPR, pp. 1426–1433 (2012)

    Google Scholar 

  9. Baumgarten, G., Fiedler, J., Fricke, K.H., Gerding, M., Hervig, M., Hoffmann, P., Müller, N., Pautet, P.D., Rapp, M., Robert, C., Rusch, D., von Savigny, C., Singer, W.: The noctilucent cloud (NLC) display during the ECOMA/MASS sounding. Ann. Geophys. 27, 953–965 (2009)

    Article  Google Scholar 

  10. Baxter, B., Hooper, B.A., Williams, J.Z., Dugan, J.P.: Polarimetric remote sensing of ocean waves. In: Proceedings of MTS/IEEE OCEANS, pp. 1–5 (2009)

    Google Scholar 

  11. Bennett, G.G.: The calculation of astronomical refraction in marine navigation. J. Navig. 35, 255–259 (1982)

    Article  Google Scholar 

  12. Bishop, T.E., Zanetti, S., Favaro, P.: Light field superresolution. Proc. IEEE ICCP 129, 1–9 (2009)

    Google Scholar 

  13. Bradley, E.S., Toomey, M.P., Still, C.J., Roberts, D.A.: Multi-scale sensor fusion with an online application: integrating GOES, MODIS, and webcam imagery for environmental monitoring. IEEE Sel. Top. Appl. Earth Obs. Remote Sen. 3, 497–506 (2010)

    Article  Google Scholar 

  14. Brdon, E.M., Bréon, F.M.: An analytical model for the cloud-free atmosphere/ocean system reflectance. Remote Sens. Environ. 43, 179–192 (1993)

    Article  Google Scholar 

  15. Breslav, M., Fuller, N., Sclaroff, S., Betke, M.: 3D Pose estimation of bats in the wild. In: Proceedings of the IEEE WACV (2014)

    Google Scholar 

  16. Cazorla, A., Olmo, F.J., Aladosarboledas, L., Alados-Arboledas, L.: Using a sky imager for aerosol characterization. Atmos. Environ. 42, 2739–2745 (2008)

    Article  Google Scholar 

  17. Clouds in the Cloud webpage and data link. http://webee.technion.ac.il/people/yoav/research/clouds_inthe_cloud.html

  18. Diner, D.J., Beckert, J.C., Reilly, T.H., Bruegge, C.J., Conel, J.E., Kahn, R.A., Martonchik, J.V., Ackerman, T.P., Davies, R., Gerstl, S.A.: Multi-angle imaging spectro-radiometer (MISR) instrument description and experiment overview. IEEE Trans. Geosci. Remote Sens. 36, 1072–1087 (1998)

    Article  Google Scholar 

  19. Diner, D.J., Davis, A., Hancock, B., Gutt, G., Chipman, R.A., Cairns, B.: Dual-photoelastic-modulator-based polarimetric imaging remote sensing. Appl. Opt. 46, 8428–8445 (2007)

    Article  Google Scholar 

  20. Diner, D.J., Davis, A., Hancock, B., Geier, S., Rheingans, B., Jovanovic, V., Bull, M., Rider, D.M., Chipman, R.A., Mahler, A.B., McClain, S.C.: First results from a dual photoelastic-modulator-based polarimetric camera. Appl. Opt. 49, 2929 (2010)

    Article  Google Scholar 

  21. Diner, D.J., Martonchik, J.V.: Atmospheric transmittance from spacecraft using multiple view angle imagery. Appl. Opt. 24, 3503–3511 (1985)

    Article  Google Scholar 

  22. Evans, K.F.: The spherical harmonics discrete ordinate method for three-dimensional atmospheric radiative transfer. J. Atmos. Sci. 55, 429–446 (1998)

    Article  Google Scholar 

  23. Horstmeyer, R., Euliss, G., Athale, R., Levoy, M.: Flexible multimodal camera using a light field architecture. In: Proceedings of the IEEE ICCP, pp. 1–8 (2009)

    Google Scholar 

  24. Hosek, L., Wilkie, A.: An analytic model for full spectral sky-dome radiance. ACM TOG 31, 95:1–95:9 (2012)

    Article  Google Scholar 

  25. Ihrke, I., Magnor, M.: Image-based tomographic reconstruction of flames. In: Proceedings of the ACM SIGGRAPH, pp. 365–373 (2004)

    Google Scholar 

  26. Jacobs, N., King, J., Bowers, D., Souvenir, R.: Estimating cloud maps from outdoor image sequences. In: Proceedings of the IEEE WACV (2014)

    Google Scholar 

  27. Kang, S.B., Weiss, R.: Can we calibrate a camera using an image of a flat, textureless lambertian surface? In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 640–653. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  28. Kassianov, E., Long, C., Christy, J.: Cloud-base-height estimation from paired ground-based hemispherical observations. J. Appl. Meteorol. 44, 1221–1233 (2005)

    Article  Google Scholar 

  29. Kim, J., Lanman, D., Mukaigawa, Y., Raskar, R.: Descattering transmission via angular filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 86–99. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  30. Koreban, F., Schechner, Y.Y.: Geometry by deflaring. In: Proceedings of the IEEE ICCP, pp. 1–8 (2009)

    Google Scholar 

  31. Kutulakos, K., Seitz, S.: A theory of shape by space carving. IJCV 38, 199–218 (2000)

    Article  MATH  Google Scholar 

  32. Lalonde, J.F., Narasimhan, S., Efros, A.: What do the sun and the sky tell us about the camera? IJCV 88, 24–51 (2010)

    Article  Google Scholar 

  33. Lensky, I., Rosenfeld, D.: The time-space exchangeability of satellite retrieved relations between cloud top temperature and particle effective radius. Atmos. Chem. Phys. 6, 2887–2894 (2006)

    Article  Google Scholar 

  34. Levoy, M.: Efficient ray tracing of volume data. ACM TOG 9, 245–261 (1990)

    Article  MATH  Google Scholar 

  35. Levoy, M., Ng, R., Adams, A., Footer, M., Horowitz, M.: Light field microscopy. ACM TOG 25, 924–934 (2006)

    Article  Google Scholar 

  36. Litvinov, A., Schechner, Y.Y.: Addressing radiometric nonidealities: a unified framework. In: Proceedings of the IEEE CVPR, vol. 2, pp. 52–59 (2005)

    Google Scholar 

  37. Long, C.N., Sabburg, J.M., Calbo, J., Pages, D., Calbó, J., Pagès, D.: Retrieving cloud characteristics from ground-based daytime color all-sky images. J. Atmos. Oceanic Technol. 23, 633–652 (2006)

    Article  Google Scholar 

  38. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60, 91–110 (2004)

    Article  Google Scholar 

  39. Mol, B.V., Ruddick, K., van Mol, B., Ruddick, K.: The compact high resolution imaging spectrometer (CHRIS): the future of hyperspectral satellite sensors. In: Proceedings of Airborne Imaging Spectroscopy Workshop on Imagery of Oostende Coastal and Inland Waters (2004)

    Google Scholar 

  40. Peng, Z., Yoo, S., Yu, D., Huang, D., Kalb, P., Heiser, J.: 3D cloud detection and tracking for solar forecast using multiple sky imagers. In: Proceedings of the ACM Symposium Applied Computing, pp. 512–517 (2014)

    Google Scholar 

  41. Pust, N.J., Shaw, J.A.: Digital all-sky polarization imaging of partly cloudy skies. Appl. Opt. 47, H190–H198 (2008)

    Article  Google Scholar 

  42. Raskar, R., Agrawal, A., Wilson, C.A., Veeraraghavan, A.: Glare aware photography. ACM TOG 27, 56:1–56:10 (2008)

    Article  Google Scholar 

  43. Rouf, M., Mantiuk, R., Heidrich, W., Trentacoste, M., Lau, C.: Glare encoding of high dynamic range images. In: Proceedings of the IEEE CVPR, pp. 289–296 (2011)

    Google Scholar 

  44. Seiz, G., Baltsavias, E., Gruen, A.A.: Cloud mapping from the ground: use of photogrammetric methods. Photogram. Eng. Remote Sens. 68, 941–951 (2002)

    Google Scholar 

  45. Seiz, G., Davies, R.: Reconstruction of cloud geometry from multi-view satellite images. Remote Sens. Environ. 100, 143–149 (2006)

    Article  Google Scholar 

  46. Seiz, G., Shields, J., Feister, U., Baltsavias, E.P., Gruen, A.: Cloud mapping with ground-based photogrammetric cameras. Int. J. Remote Sens. 28, 2001–2032 (2007)

    Article  Google Scholar 

  47. Schechner, Y.Y., Diner, D., Martonchik, J.: Spaceborne underwater imaging. In: Proceedings of the IEEE ICCP, pp. 1–8 (2011)

    Google Scholar 

  48. Talvala, E.V., Adams, A., Horowitz, M., Levoy, M.: Veiling glare in high dynamic range imaging. ACM TOG 26, 37:1–37:9 (2007)

    Article  Google Scholar 

  49. Veikherman, D., Aides, A., Schechner, Y.Y., Levis, A.: Clouds in the cloud: supplementary material for Proceedings of ACCV (2014)

    Google Scholar 

  50. Yamashita, M.: Cloud cover estimation using multitemporal hemisphere imageries. In: Proceedings of the XXth Congress of the Society for Photogrammetry and Remote Sensing, pp. 818–821 (2004)

    Google Scholar 

Download references

Acknowledgments

We are grateful to Pinhas Alpert, Daniel Rosenfeld, Orit Altaratz-Stollar, Nir Stav, Raanan Fattal, Arnon Karnieli, David Diner and Anthony Davis for useful discussions. We thank Mark Shenin and Technion building superintendents for experiment assistance. We thank Johanan Erez, Ina Talmon, Tamar Galateanu and Dani Yagodin for technical support. Yoav Schechner is a Lanadu Fellow - supported by the Taub Foundation. His research is supported in part by the Israel Science Foundation (ISF Grant 1467/12) and the Asher Space Research Institute. This work was conducted in the Ollendorff Minerva Center. Minerva is funded through the BMBF.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Technion - Israel Institute of Technology, Haifa, Israel

    Dmitry Veikherman, Amit Aides, Yoav Y. Schechner & Aviad Levis

Authors
  1. Dmitry Veikherman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit Aides
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yoav Y. Schechner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aviad Levis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dmitry Veikherman .

Editor information

Editors and Affiliations

  1. Technische Universität München, Garching, Bayern, Germany

    Daniel Cremers

  2. University of Adelaide, Adelaide, South Australia, Australia

    Ian Reid

  3. Keio University, Yokohama, Kanagawa, Japan

    Hideo Saito

  4. University of California at Merced, Merced, California, USA

    Ming-Hsuan Yang

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material (pdf 706 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Veikherman, D., Aides, A., Schechner, Y.Y., Levis, A. (2015). Clouds in the Cloud. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9006. Springer, Cham. https://doi.org/10.1007/978-3-319-16817-3_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16817-3_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16816-6

  • Online ISBN: 978-3-319-16817-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation