Abstract
Light-field imaging can be scaled up to a very large area, to map the Earth’s atmosphere in 3D. Multiview spaceborne instruments suffer low spatio-temporal-angular resolution, and are very expensive and unscalable. We develop sky light-field imaging, by a wide, scalable network of wide-angle cameras looking upwards, which upload their data to the cloud. This new type of imaging-system poses new computational vision and photography problems, some of which generalize prior monocular tasks. These include radiometric self-calibration across a network, overcoming flare by a network, and background estimation. On the other hand, network redundancy offers solutions to these problems, which we derive. Based on such solutions, the light-field network enables unprecedented ways to measure nature. We demonstrate this experimentally by 3D recovery of clouds, in high spatio-temporal resolution. It is achieved by space carving of the volumetric distribution of semi-transparent clouds. Such sensing can complement satellite imagery, be useful to meteorology, make aerosol tomography realizable, and give new, powerful tools to atmospheric and avian wildlife scientists.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Manual tracking of a special flight and long exposures at night were used in [44].
- 3.
Higher than 20\(^{\circ }\) above the horizon [11], errors caused by atmospheric refraction are smaller than \(0.05^{\circ }\), much less than the angular size of each of our pixels, 0.18\(^{\circ }\).
- 4.
Sun blocker was not used here, since saturation and blooming do not impair cloud shape reconstruction.
References
Adelson, E., Wang, J.: Single lens stereo with a plenoptic camera. IEEE Trans. PAMI 14, 99–106 (1992)
Aides, A., Schechner, Y.Y., Holodovsky, V., Garay, M.J., Davis, A.B.: Multi sky-view 3D aerosol distribution recovery. Opt. Express 21, 25820–25833 (2013)
Allmen, M.C., Kegelmeyer Jr., P.: The computation of cloud base height from paired whole-sky imaging cameras. Mach. Vis. Appl. 9, 160–165 (1997)
Alterman, M., Schechner, Y.Y., Swirski, Y.: Triangulation in random refractive distortions. In: Proceedings of the IEEE ICCP, pp. 1–10 (2013)
Alterman, M., Schechner, Y.Y., Vo, M., Narasimhan, S.G.: Passive tomography of turbulence strength. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 47–60. Springer, Heidelberg (2014)
Alterman, M., Swirski, Y., Schechner, Y.Y.: STELLA MARIS: stellar marine refractive imaging sensor. In: Proceedings of the IEEE ICCP, pp. 1–10 (2014)
Atkinson, B.W.: Meso-Scale Atmospheric Circulations. Academic Press, New York (1989)
Basha, T., Avidan, S., Hornung, A., Matusik, W.: Structure and motion from scene registration. In: Proceedings of the IEEE CVPR, pp. 1426–1433 (2012)
Baumgarten, G., Fiedler, J., Fricke, K.H., Gerding, M., Hervig, M., Hoffmann, P., Müller, N., Pautet, P.D., Rapp, M., Robert, C., Rusch, D., von Savigny, C., Singer, W.: The noctilucent cloud (NLC) display during the ECOMA/MASS sounding. Ann. Geophys. 27, 953–965 (2009)
Baxter, B., Hooper, B.A., Williams, J.Z., Dugan, J.P.: Polarimetric remote sensing of ocean waves. In: Proceedings of MTS/IEEE OCEANS, pp. 1–5 (2009)
Bennett, G.G.: The calculation of astronomical refraction in marine navigation. J. Navig. 35, 255–259 (1982)
Bishop, T.E., Zanetti, S., Favaro, P.: Light field superresolution. Proc. IEEE ICCP 129, 1–9 (2009)
Bradley, E.S., Toomey, M.P., Still, C.J., Roberts, D.A.: Multi-scale sensor fusion with an online application: integrating GOES, MODIS, and webcam imagery for environmental monitoring. IEEE Sel. Top. Appl. Earth Obs. Remote Sen. 3, 497–506 (2010)
Brdon, E.M., Bréon, F.M.: An analytical model for the cloud-free atmosphere/ocean system reflectance. Remote Sens. Environ. 43, 179–192 (1993)
Breslav, M., Fuller, N., Sclaroff, S., Betke, M.: 3D Pose estimation of bats in the wild. In: Proceedings of the IEEE WACV (2014)
Cazorla, A., Olmo, F.J., Aladosarboledas, L., Alados-Arboledas, L.: Using a sky imager for aerosol characterization. Atmos. Environ. 42, 2739–2745 (2008)
Clouds in the Cloud webpage and data link. http://webee.technion.ac.il/people/yoav/research/clouds_inthe_cloud.html
Diner, D.J., Beckert, J.C., Reilly, T.H., Bruegge, C.J., Conel, J.E., Kahn, R.A., Martonchik, J.V., Ackerman, T.P., Davies, R., Gerstl, S.A.: Multi-angle imaging spectro-radiometer (MISR) instrument description and experiment overview. IEEE Trans. Geosci. Remote Sens. 36, 1072–1087 (1998)
Diner, D.J., Davis, A., Hancock, B., Gutt, G., Chipman, R.A., Cairns, B.: Dual-photoelastic-modulator-based polarimetric imaging remote sensing. Appl. Opt. 46, 8428–8445 (2007)
Diner, D.J., Davis, A., Hancock, B., Geier, S., Rheingans, B., Jovanovic, V., Bull, M., Rider, D.M., Chipman, R.A., Mahler, A.B., McClain, S.C.: First results from a dual photoelastic-modulator-based polarimetric camera. Appl. Opt. 49, 2929 (2010)
Diner, D.J., Martonchik, J.V.: Atmospheric transmittance from spacecraft using multiple view angle imagery. Appl. Opt. 24, 3503–3511 (1985)
Evans, K.F.: The spherical harmonics discrete ordinate method for three-dimensional atmospheric radiative transfer. J. Atmos. Sci. 55, 429–446 (1998)
Horstmeyer, R., Euliss, G., Athale, R., Levoy, M.: Flexible multimodal camera using a light field architecture. In: Proceedings of the IEEE ICCP, pp. 1–8 (2009)
Hosek, L., Wilkie, A.: An analytic model for full spectral sky-dome radiance. ACM TOG 31, 95:1–95:9 (2012)
Ihrke, I., Magnor, M.: Image-based tomographic reconstruction of flames. In: Proceedings of the ACM SIGGRAPH, pp. 365–373 (2004)
Jacobs, N., King, J., Bowers, D., Souvenir, R.: Estimating cloud maps from outdoor image sequences. In: Proceedings of the IEEE WACV (2014)
Kang, S.B., Weiss, R.: Can we calibrate a camera using an image of a flat, textureless lambertian surface? In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 640–653. Springer, Heidelberg (2000)
Kassianov, E., Long, C., Christy, J.: Cloud-base-height estimation from paired ground-based hemispherical observations. J. Appl. Meteorol. 44, 1221–1233 (2005)
Kim, J., Lanman, D., Mukaigawa, Y., Raskar, R.: Descattering transmission via angular filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 86–99. Springer, Heidelberg (2010)
Koreban, F., Schechner, Y.Y.: Geometry by deflaring. In: Proceedings of the IEEE ICCP, pp. 1–8 (2009)
Kutulakos, K., Seitz, S.: A theory of shape by space carving. IJCV 38, 199–218 (2000)
Lalonde, J.F., Narasimhan, S., Efros, A.: What do the sun and the sky tell us about the camera? IJCV 88, 24–51 (2010)
Lensky, I., Rosenfeld, D.: The time-space exchangeability of satellite retrieved relations between cloud top temperature and particle effective radius. Atmos. Chem. Phys. 6, 2887–2894 (2006)
Levoy, M.: Efficient ray tracing of volume data. ACM TOG 9, 245–261 (1990)
Levoy, M., Ng, R., Adams, A., Footer, M., Horowitz, M.: Light field microscopy. ACM TOG 25, 924–934 (2006)
Litvinov, A., Schechner, Y.Y.: Addressing radiometric nonidealities: a unified framework. In: Proceedings of the IEEE CVPR, vol. 2, pp. 52–59 (2005)
Long, C.N., Sabburg, J.M., Calbo, J., Pages, D., Calbó, J., Pagès, D.: Retrieving cloud characteristics from ground-based daytime color all-sky images. J. Atmos. Oceanic Technol. 23, 633–652 (2006)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60, 91–110 (2004)
Mol, B.V., Ruddick, K., van Mol, B., Ruddick, K.: The compact high resolution imaging spectrometer (CHRIS): the future of hyperspectral satellite sensors. In: Proceedings of Airborne Imaging Spectroscopy Workshop on Imagery of Oostende Coastal and Inland Waters (2004)
Peng, Z., Yoo, S., Yu, D., Huang, D., Kalb, P., Heiser, J.: 3D cloud detection and tracking for solar forecast using multiple sky imagers. In: Proceedings of the ACM Symposium Applied Computing, pp. 512–517 (2014)
Pust, N.J., Shaw, J.A.: Digital all-sky polarization imaging of partly cloudy skies. Appl. Opt. 47, H190–H198 (2008)
Raskar, R., Agrawal, A., Wilson, C.A., Veeraraghavan, A.: Glare aware photography. ACM TOG 27, 56:1–56:10 (2008)
Rouf, M., Mantiuk, R., Heidrich, W., Trentacoste, M., Lau, C.: Glare encoding of high dynamic range images. In: Proceedings of the IEEE CVPR, pp. 289–296 (2011)
Seiz, G., Baltsavias, E., Gruen, A.A.: Cloud mapping from the ground: use of photogrammetric methods. Photogram. Eng. Remote Sens. 68, 941–951 (2002)
Seiz, G., Davies, R.: Reconstruction of cloud geometry from multi-view satellite images. Remote Sens. Environ. 100, 143–149 (2006)
Seiz, G., Shields, J., Feister, U., Baltsavias, E.P., Gruen, A.: Cloud mapping with ground-based photogrammetric cameras. Int. J. Remote Sens. 28, 2001–2032 (2007)
Schechner, Y.Y., Diner, D., Martonchik, J.: Spaceborne underwater imaging. In: Proceedings of the IEEE ICCP, pp. 1–8 (2011)
Talvala, E.V., Adams, A., Horowitz, M., Levoy, M.: Veiling glare in high dynamic range imaging. ACM TOG 26, 37:1–37:9 (2007)
Veikherman, D., Aides, A., Schechner, Y.Y., Levis, A.: Clouds in the cloud: supplementary material for Proceedings of ACCV (2014)
Yamashita, M.: Cloud cover estimation using multitemporal hemisphere imageries. In: Proceedings of the XXth Congress of the Society for Photogrammetry and Remote Sensing, pp. 818–821 (2004)
Acknowledgments
We are grateful to Pinhas Alpert, Daniel Rosenfeld, Orit Altaratz-Stollar, Nir Stav, Raanan Fattal, Arnon Karnieli, David Diner and Anthony Davis for useful discussions. We thank Mark Shenin and Technion building superintendents for experiment assistance. We thank Johanan Erez, Ina Talmon, Tamar Galateanu and Dani Yagodin for technical support. Yoav Schechner is a Lanadu Fellow - supported by the Taub Foundation. His research is supported in part by the Israel Science Foundation (ISF Grant 1467/12) and the Asher Space Research Institute. This work was conducted in the Ollendorff Minerva Center. Minerva is funded through the BMBF.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Veikherman, D., Aides, A., Schechner, Y.Y., Levis, A. (2015). Clouds in the Cloud. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9006. Springer, Cham. https://doi.org/10.1007/978-3-319-16817-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-16817-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16816-6
Online ISBN: 978-3-319-16817-3
eBook Packages: Computer ScienceComputer Science (R0)