research-article

Programmable aperture photography: multiplexed light field acquisition

Authors:

Chia-Kai Liang,

Homer H. ChenAuthors Info & Claims

SIGGRAPH '08: ACM SIGGRAPH 2008 papers

Article No.: 55, Pages 1 - 10

https://doi.org/10.1145/1399504.1360654

Published: 01 August 2008 Publication History

Abstract

In this paper, we present a system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition. The shape of the programmable aperture can be adjusted and used to capture light field at full sensor resolution through multiple exposures without any additional optics and without moving the camera. High acquisition efficiency is achieved by employing an optimal multiplexing scheme, and quality data is obtained by using the two post-processing algorithms designed for self calibration of photometric distortion and for multi-view depth estimation. View-dependent depth maps thus generated help boost the angular resolution of light field. Various post-exposure photographic effects are given to demonstrate the effectiveness of the system and the quality of the captured light field.

Supplementary Material

MOV File (a55-liang.mov)

Download
21.30 MB

References

[1]

Adelson, E. H., and Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 99--106.

Digital Library

[2]

Aggarwal, M., Hua, H., and Ahuja, N. 2001. On cosinefourth and vignetting effects in real lenses. In Proc. ICCV '01: Proc. the Eighth IEEE International Conference on Computer Vision, vol. 1, 472--479.

[3]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11, 1222--1239.

Digital Library

[4]

Chai, J.-X., Chan, S.-C., and Tong, H.-Y. S. X. 2000. Plenoptic sampling. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 307--318.

Digital Library

[5]

Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 11--20.

Digital Library

[6]

Donoho, D. 2006. Compressed Sensing. IEEE Trans. Information Theory 52, 4, 1289--1306.

Digital Library

[7]

Durand, F., Holzschuch, N., Soler, C., Chan, E., and Sillion, F. X. 2005. A frequency analysis of light transport. In SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM Press, New York, NY, USA, 1115--1126.

Digital Library

[8]

Farid, H., and Simoncelli, E. P. 1998. Range estimation by optical differentiation. Journal of the Optical Society of America A 15, 7, 1777--1786.

[9]

Georgiev, T., Zheng, K. C., Curless, B., Salesin, D., Nayar, S., and Intwala, C. 2006. Spatio-angular resolution tradeoff in integral photography. In EGRW '06: Proc. the 17th Eurographics workshop on Rendering.

[10]

Georgiev, T., Intwala, C., and Babacan, D. 2007. Lightfield capture by multiplexing in the frequency domain. Adobe technical report, Adobe Systems Incorporated.

[11]

Goldman, D. B., and Chen, J.-H. 2005. Vignette and exposure calibration and compensation. In Proc. ICCV '05: Proc. the 10th IEEE International Conference on Computer Vision, 899--906.

Digital Library

[12]

Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 43--54.

Digital Library

[13]

Green, P., Sun, W., Matusik, W., and Durand, F. 2007. Multi-aperture photography. ACM Trans. Graph. 26, 3, 68.

Digital Library

[14]

Harwit, M., and Sloane, N. J. 1979. Hadamard Transform Optics. Academic Press, New York.

[15]

HP components group. 1998. Noise sources in CMOS image sensors. Technical report, Hewlett-Packard Company.

[16]

Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 297--306.

Digital Library

[17]

Ive, H. E. 1930. Parallax panoramagrams made with a large diameter lens. Journal of the Optical Society of America 20, 6 (June), 332--342.

[18]

Joshi, N., Matusik, W., and Avidan, S. 2006. Natural video matting using camera arrays. ACM Trans. Graph. 25, 3, 779--786.

Digital Library

[19]

Kang, S. B., and Szeliski, R. 2004. Extracting view-dependent depth maps from a collection of images. International Journal of Computer Vision 58, 2, 139--163.

Digital Library

[20]

Kolmogorov, V., and Zabih, R. 2002. Multi-camera scene reconstruction via graph cuts. Proc. European Conference on Computer Vision 3, 82--96.

Digital Library

[21]

Kolmogorov, V. 2006. Convergent tree-reweighted message passing for energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 28, 10, 1568--1583.

Digital Library

[22]

Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26, 3, 70.

Digital Library

[23]

Levoy, M., and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 31--42.

Digital Library

[24]

Liang, C.-K., Liu, G., and Chen, H. H. 2007. Light field acquisition using programmable aperture camera. In Proc. IEEE International Conference on Image Processing, vol. 5, 233--236.

[25]

Lippmann, M. G. 1908. Epreuves reversible donnant la sensation du relief. J. Phys 7, 821--825.

[26]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91--110.

Digital Library

[27]

Nayar, S. K., and Branzoi, V. 2003. Adaptive dynamic range imaging: Optical control of pixel exposures over space and time. In Proc. ICCV '03: Proc. the Ninth IEEE International Conference on Computer Vision, vol. 2, 1168--1175.

Digital Library

[28]

Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light field photography with a hand-held plenoptic camera. CSTR 2005-02, Stanford University, April.

[29]

Ng, R. 2005. Fourier slice photography. In SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM, New York, NY, USA, 735--744.

Digital Library

[30]

Okoshi, T. 1976. Three-dimensional imaging techniques. Academic Press New York.

[31]

Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. In SIGGRAPH '04: ACM SIGGRAPH 2004 Papers, ACM Press, New York, NY, USA, 679--688.

Digital Library

[32]

Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3, 795--804.

Digital Library

[33]

Ratner, N., and Schechner, Y. Y. 2007. Illumination multiplexing within fundamental limits. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1--8.

[34]

Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47, 1--3, 7--42.

Digital Library

[35]

Schechner, Y. Y., and Nayar, S. K. 2004. Uncontrolled modulation imaging. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 197--204.

Digital Library

[36]

Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2003. A theory of multiplexed illumination. In Proc. ICCV '03: Proc. the Ninth IEEE International Conference on Computer Vision, vol. 2, 808--815.

Digital Library

[37]

Senkichi, C., Toshio, M., Toshinori, H., Yuichi, M., and Hidetoshi, K., 2003. Device and method for correcting camera-shake and device for detecting camera shake. JP Patent No. 2003-138436.

[38]

Sun, J., Li, Y., Kang, S. B., and Shum, H.-Y. 2005. Symmetric stereo matching for occlusion handling. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 399--406.

Digital Library

[39]

Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., and Rother, C. 2006. A comparative study of energy minimization methods for markov random fields. In Proc. European Conference on Computer Vision, vol. 2, 16--29.

Digital Library

[40]

Tsin, Y., Ramesh, V., and Kanade, T. 2001. Statistical calibration of CCD imaging process. In Proc. ICCV '01: Proc. the Eighth IEEE International Conference on Computer Vision, 480--487.

[41]

Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26, 3, 69.

Digital Library

[42]

Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. In ACM SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM Press, New York, NY, USA, 756--764.

Digital Library

[43]

Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. In SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM, New York, NY, USA, 765--776.

Digital Library

[44]

Yang, J. C., Everett, M., Buehler, C., and McMillan, L. 2002. A real-time distributed light field camera. In EGRW '02: Proc. the 13th Eurographics workshop on Rendering, 77--86.

Digital Library

[45]

Yang, Q., Yang, R., Davis, J., and Nister, D. 2007. Spatial-depth super resolution for range images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1--8.

[46]

Zomet, A., and Nayar, S. K. 2006. Lensless imaging with a controllable aperture. Proc. IEEE Conference on Computer Vision and Pattern Recognition, 339--346.

Digital Library

[47]

Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3d displays. In EGSR'06: Proc. the 17th Eurographics Symposium on Rendering.

Cited By

ZHANG Runnan 张ZHOU Ning 周ZHOU Zihao 周DU Heheng 杜CHEN Qian 陈ZUO Chao 左(2024)光场表征及其分辨率提升技术：文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
https://doi.org/10.3788/IRLA20240347
Guindy MKara P(2024)Lessons Learned from Implementing Light Field Camera Animation: Implications, Limitations, Potentials, and Future Research EffortsMultimodal Technologies and Interaction10.3390/mti80800688:8(68)Online publication date: 1-Aug-2024
https://doi.org/10.3390/mti8080068
Zhang YYang JWu TAn W(2024)Self-calibration and Refocusing with One Interpolation for Infrared Array Cameras2024 5th International Conference on Computer Vision, Image and Deep Learning (CVIDL)10.1109/CVIDL62147.2024.10603919(523-529)Online publication date: 19-Apr-2024
https://doi.org/10.1109/CVIDL62147.2024.10603919
Show More Cited By

Index Terms

Programmable aperture photography: multiplexed light field acquisition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
      2. Modeling methodologies

Recommendations

Light field rendering
SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques

A number of techniques have been proposed for flying through scenes by redisplaying previously rendered or digitized views. Techniques have also been proposed for interpolating between views by warping input images, using depth information or ...
Compressive light field photography using overcomplete dictionaries and optimized projections

Light field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into ...
Light field microscopy
SIGGRAPH '06: ACM SIGGRAPH 2006 Papers

By inserting a microlens array into the optical train of a conventional microscope, one can capture light fields of biological specimens in a single photograph. Although diffraction places a limit on the product of spatial and angular resolution in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '08: ACM SIGGRAPH 2008 papers

August 2008

887 pages

ISBN:9781450301121

DOI:10.1145/1399504

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Excellent Research Projects of National Taiwan University
National Science Council Taiwan

Conference

SIGGRAPH '08

Sponsor:

SIGGRAPH

SIGGRAPH '08: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 11 - 15, 2008

California, Los Angeles

Acceptance Rates

SIGGRAPH '08 Paper Acceptance Rate 90 of 518 submissions, 17%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
2,662
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)4

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

ZHANG Runnan 张ZHOU Ning 周ZHOU Zihao 周DU Heheng 杜CHEN Qian 陈ZUO Chao 左(2024)光场表征及其分辨率提升技术：文献综述及最新进展(特邀)Infrared and Laser Engineering10.3788/IRLA2024034753:9(20240347)Online publication date: 2024
https://doi.org/10.3788/IRLA20240347
Guindy MKara P(2024)Lessons Learned from Implementing Light Field Camera Animation: Implications, Limitations, Potentials, and Future Research EffortsMultimodal Technologies and Interaction10.3390/mti80800688:8(68)Online publication date: 1-Aug-2024
https://doi.org/10.3390/mti8080068
Zhang YYang JWu TAn W(2024)Self-calibration and Refocusing with One Interpolation for Infrared Array Cameras2024 5th International Conference on Computer Vision, Image and Deep Learning (CVIDL)10.1109/CVIDL62147.2024.10603919(523-529)Online publication date: 19-Apr-2024
https://doi.org/10.1109/CVIDL62147.2024.10603919
Ducrocq JCaron G(2024)A Survey on Adaptive CamerasInternational Journal of Computer Vision10.1007/s11263-024-02025-7132:8(2989-3022)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s11263-024-02025-7
Lyu XHou J(2024)Probabilistic-Based Feature Embedding of 4-D Light Fields for Compressive Imaging and DenoisingInternational Journal of Computer Vision10.1007/s11263-023-01974-9132:6(2255-2275)Online publication date: 12-Jan-2024
https://doi.org/10.1007/s11263-023-01974-9
Liang J(2024)Introduction to Coded Optical ImagingCoded Optical Imaging10.1007/978-3-031-39062-3_1(3-13)Online publication date: 20-Mar-2024
https://doi.org/10.1007/978-3-031-39062-3_1
Hu YYao MHuang ZPeng JZhang ZZhong J(2022)Full-Resolution Light-Field Camera via Fourier Dual PhotographyPhotonics10.3390/photonics90805599:8(559)Online publication date: 10-Aug-2022
https://doi.org/10.3390/photonics9080559
Xiang MPan ALiu JXi TGuo XLiu FShao X(2022)Phase Diversity-Based Fourier Ptychography for Varying Aberration CorrectionFrontiers in Physics10.3389/fphy.2022.84894310Online publication date: 16-Mar-2022
https://doi.org/10.3389/fphy.2022.848943
Zhou SZhu TShi KLi YZheng WYong J(2021)Review of light field technologiesVisual Computing for Industry, Biomedicine, and Art10.1186/s42492-021-00096-84:1Online publication date: 3-Dec-2021
https://doi.org/10.1186/s42492-021-00096-8
Lyu XZhu ZGuo MJin JHou JZeng HShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Learning Spatial-angular Fusion for Compressive Light Field Imaging in a Cycle-consistent FrameworkProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475214(4613-4621)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475214
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten