research-article

Computational stereo camera system with programmable control loop

Authors:

Pierre Greisen,

Christine Chen,

Aljoscha Smolic,

Wojciech Matusik,

Markus GrossAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 30, Issue 4

Article No.: 94, Pages 1 - 10

https://doi.org/10.1145/2010324.1964989

Published: 25 July 2011 Publication History

Abstract

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, and artistic intent. In this paper, we present a computational stereo camera system that closes the control loop from capture and analysis to automatic adjustment of physical parameters. Intuitive interaction metaphors are developed that replace cumbersome handling of rig parameters using a touch screen interface with 3D visualization. Our system is designed to make stereoscopic 3D production as easy, intuitive, flexible, and reliable as possible. Captured signals are processed and analyzed in real-time on a stream processor. Stereoscopy and user settings define programmable control functionalities, which are executed in real-time on a control processor. Computational power and flexibility is enabled by a dedicated software and hardware architecture. We show that even traditionally difficult shots can be easily captured using our system.

Supplemental Material

ZIP File - Supplemental material.

Download
121.84 MB

References

[1]

3ality Digital, 2011. Beam splitter rigs and stereoscopic image processors. http://www.3alitydigital.com/.

[2]

Adams, A., Talvala, E.-V., Park, S. H., Jacobs, D. E., Ajdin, B., Gelfand, N., Dolson, J., Vaquero, D., Baek, J., Tico, M., Lensch, H. P. A., Matusik, W., Pulli, K., Horowitz, M., and Levoy, M. 2010. The Frankencamera: an experimental platform for computational photography. ACM Transactions of Graphics 29, 29:1--29:12.

Digital Library

[3]

Babenko, B., Yang, M., and Belongie, S. 2009. Visual tracking with online multiple instance learning. In Conference on Computer Vision and Pattern Recognition, IEEE, 983--990.

[4]

Bouguet, J.-Y., 2010. Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/.

[5]

Brown, D. C. 1966. Decentering distortion of lenses. Photogram-matric Engineering 32, 444--462.

[6]

Fraser, C. S., and Al-Ajlouni, S. 2006. Zoom-dependent camera calibration in digital close-range photogrammetry. Photogrammetric Engineering & Remote Sensing 72, 1017--1026.

[7]

Hoffman, D., Girshick, A., Akeley, K., and Banks, M. 2008. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3, 33.

[8]

Howard, I. P., and Rogers, B. J. 2002. Seeing in Depth. Oxford University Press.

[9]

Jones, G., Lee, D., Holliman, N., and Ezra, D. 2001. Controlling perceived depth in stereoscopic images. SPIE, vol. 4297 of Stereoscopic Displays and Virtual Reality Systems, 42--53.

[10]

Kawai, T., Shibata, T., Inoue, T., Sakaguchi, Y., Okabe, K., and Kuno, Y. 2002. Development of software for editing of stereoscopic 3D movies. SPIE, vol. 4660 of Stereoscopic Displays and Virtual Reality Systems, 58--65.

[11]

Koppal, S., Zitnick, C. L., Cohen, M., Kang, S. B., Ressler, B., and Colburn, A. 2011. A viewer-centric editor for 3D movies. IEEE Computer Graphics and Applications 31, 1, 20--35.

Digital Library

[12]

Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. H. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Transactions on Graphics 29, 4.

Digital Library

[13]

Maier, F., 2010. Stereotec stereoscopic calculator. http://stereotec.com/.

[14]

Mallon, J., and Whelan, P. F. 2005. Projective rectification from the fundamental matrix. Image Vision Computing 23, 643--650.

Digital Library

[15]

Malvar, H., He, L.-W., and Cutler, R. 2004. High-quality linear interpolation for demosaicing of bayer-patterned color images. In International Conference of Acoustic, Speech and Signal Processing, IEEE.

[16]

Masaoka, K., Hanazato, A., Emoto, M., Yamanoue, H., Nojiri, Y., and Okano, F. 2006. Spatial distortion prediction system for stereoscopic images. Electronic Imaging 15, 1, 013002:1--12.

[17]

Mendiburu, B. 2008. 3D Movie Making - Stereoscopic Digital Cinema from Script to Screen. Elsevier.

[18]

Mueller, R., Ward, C., and Husak, M. 2008. A systematized WYSIWYG pipeline for digital stereoscopic 3D filmmaking. SPIE, vol. 6803 of Stereoscopic Displays and Applications.

[19]

Pitié, F., Kokaram, A., and Dahyot, R. 2007. Automated colour grading using colour distribution transfer. Computer Vision and Image Understanding 107, 1-2, 123--137.

Digital Library

[20]

Ramanath, R., Snyder, W., Yoo, Y., and Drew, M. 2005. Color image processing pipeline. Signal Processing Magazine, IEEE 22, 1, 34--43.

[21]

Sony, 2011. MPE200 multi image processor and software. http://pro.sony.com/.

[22]

Sun, G., and Holliman, N. 2009. Evaluating methods for controlling depth perception in stereoscopic cinematography. SPIE, vol. 7237 of Stereoscopic Displays and Virtual Reality Systems.

[23]

The Foundry, 2010. Ocula stereoscopic post-production plug-ins for Nuke. http://www.thefoundry.co.uk/.

[24]

Wang, C., and Sawchuk, A. A. 2008. Disparity manipulation for stereo images and video. SPIE, vol. 6803 of Stereoscopic displays and applications.

[25]

Zach, C., Karner, K., and Bischof, H. 2004. Hierarchical disparity estimation with programmable 3D hardware. In Short communications of WSCG, 275--282.

[26]

Zilly, F., Eisert, P., and Kauff, P. 2009. Real-time analysis and correction of stereoscopic HDTV sequences. In Proceedings of Visual Media Production.

[27]

Zilly, F., Mueller, M., Eisert, P., and Kauff, P. 2010. Joint estimation of epipolar geometry and rectification parameters using point correspondences for stereoscopic TV sequences. In Proceedings of 3DPVT.

[28]

Zilly, F., Mueller, M., Eisert, P., and Kauff, P. 2010. The stereoscopic analyzer - an image-based assistance tool for stereo shooting and 3D production. In IEEE International Conference on Image Processing.

Cited By

Verma PMeyer DXu HKuester F(2021)Splatty- A Unified Image Demosaicing and Rectification Method2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00083(786-795)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00083
Yan TMao YWang JLiu WQian XLau R(2020)Generating Stereoscopic Images With Convergence Control Ability From a Light Field Image PairIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2019.290355630:5(1435-1450)Online publication date: May-2020
https://doi.org/10.1109/TCSVT.2019.2903556
Yang YWu SWang XLi Z(2020)Exposure Interpolation for Two Large-Exposure-Ratio ImagesIEEE Access10.1109/ACCESS.2020.30462688(227141-227151)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3046268
Show More Cited By

Index Terms

Computational stereo camera system with programmable control loop
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition

Recommendations

Computational stereo camera system with programmable control loop
SIGGRAPH '11: ACM SIGGRAPH 2011 papers

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, ...
Single Lens Stereo with a Plenoptic Camera
Special issue on interpretation of 3-D scenes—part II

Ordinary cameras gather light across the area of their lens aperture, and the light striking a given subregion of the aperture is structured somewhat differently than the light striking an adjacent subregion. By analyzing this optical structure, one can ...
Single-Camera multi-baseline stereo using fish-eye lens and mirrors
ACCV'09: Proceedings of the 9th Asian conference on Computer Vision - Volume Part II

This report proposes a monocular range measurement system with a fish-eye lens and mirrors placed around the lens. The fish-eye lens has a wide view-angle; the captured image includes a centered region of direct observation and surrounding regions of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 30, Issue 4

July 2011

829 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2010324

Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2011

Published in TOG Volume 30, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
1,220
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Verma PMeyer DXu HKuester F(2021)Splatty- A Unified Image Demosaicing and Rectification Method2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00083(786-795)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00083
Yan TMao YWang JLiu WQian XLau R(2020)Generating Stereoscopic Images With Convergence Control Ability From a Light Field Image PairIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2019.290355630:5(1435-1450)Online publication date: May-2020
https://doi.org/10.1109/TCSVT.2019.2903556
Yang YWu SWang XLi Z(2020)Exposure Interpolation for Two Large-Exposure-Ratio ImagesIEEE Access10.1109/ACCESS.2020.30462688(227141-227151)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3046268
Haynes BMazumdar AAlaghi ABalazinska MCeze LCheung A(2018)LightDBProceedings of the VLDB Endowment10.14778/3231751.323176811:10(1192-1205)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.14778/3231751.3231768
Huo JZhang GCui JYang M(2018)Corrected calibration algorithm with a fixed constraint relationship and an error compensation technique for a binocular vision measurement systemApplied Optics10.1364/AO.57.00549257:19(5492)Online publication date: 29-Jun-2018
https://doi.org/10.1364/AO.57.005492
Scandolo LBauszat PEisemann E(2018)Gradient‐Guided Local Disparity EditingComputer Graphics Forum10.1111/cgf.1353738:1(394-404)Online publication date: 6-Sep-2018
https://doi.org/10.1111/cgf.13537
YOON YNO JCHOI S(2017)Saliency-Guided Stereo Camera Control for Comfortable VR ExplorationsIEICE Transactions on Information and Systems10.1587/transinf.2016EDL8246E100.D:9(2245-2248)Online publication date: 2017
https://doi.org/10.1587/transinf.2016EDL8246
Delis SMademlis INikolaidis NPitas I(2017)Automatic Detection of 3D Quality Defects in Stereoscopic Videos Using Binocular DisparityIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2015.251151827:5(977-991)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TCSVT.2015.2511518
Li D(2016)A Method for Stereoscopic Images Generation in Virtual Reality Systems2016 International Conference on Virtual Reality and Visualization (ICVRV)10.1109/ICVRV.2016.71(385-392)Online publication date: Sep-2016
https://doi.org/10.1109/ICVRV.2016.71
Huang XFan LZhang JWu QYuan C(2016)Real Time Complete Dense Depth Reconstruction for a Monocular Camera2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2016.89(674-679)Online publication date: Jun-2016
https://doi.org/10.1109/CVPRW.2016.89
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents