research-article

VirtualStudio2Go: digital video composition for real environments

Authors:

Anselm Grundhöfer,

Oliver BimberAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 27, Issue 5

Article No.: 151, Pages 1 - 8

https://doi.org/10.1145/1409060.1409104

Published: 01 December 2008 Publication History

Abstract

We synchronize film cameras and LED lighting with off-the-shelf video projectors. Radiometric compensation allows displaying keying patterns and other spatial codes on arbitrary real world surfaces. A fast temporal multiplexing of coded projection and flash illumination enables professional keying, environment matting, displaying moderator information, scene reconstruction, and camera tracking for non-studio film sets without being limited to the constraints of a virtual studio. This makes digital video composition more flexible, since static studio equipment, such as blue screens, teleprompters, or tracking devices, is not required. Authentic film locations can be supported with our portable system without causing a lot of installation effort.

Supplementary Material

JPG File (a151-grundhofer-mp4_hi.jpg)

Download
13.20 KB

MOV File (a151-grundhofer-mp4_hi.mov)

Download
366.89 MB

References

[1]

Bimber, O., Emmerling, A., and Klemmer, T. 2005. Embedded Entertainment with Smart Projectors. IEEE Computer 38, 1, 56--63.

Digital Library

[2]

Bimber, O., Iwai, D., Wetzstein, G., and Grundhoefer, A. 2007. The Visual Computing of Projector-Camera Systems. In Proc. Eurographics (State-of-the-Art Report), 23--46.

[3]

Choudhury, B., Singla, D., and Chandran, S. 2008. Fast color-space decomposition based environment matting. In SI3D '08: Proceedings of the 2008 symposium on Interactive 3D graphics and games, ACM, New York, NY, USA, 1--1.

[4]

Chuang, Y.-Y., Zongker, D. E., Hindordd, J., Curless, B., Salesin, D. H., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, ACM Press / ACM SIGGRAPH / Addison Wesley Logman, Computer Graphics Proceedings, Annual Conference Series, 121--130. ISBN 1-58113-208-5.

[5]

Cotting, D., Näf, M., Gross, M. H., and Fuchs, H. 2004. Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'04), 100--109.

[6]

Cotting, D., Ziegler, R., Gross, M. H., and Fuchs, H. 2005. Adaptive instant displays: Continuously calibrated projections using per-pixel light control. In Proceedings Eurographics 2005, 705--714. Eurographics 2005, Dublin, Ireland, August 29 -- September 2, 2005.

[7]

Fukaya, T., Fujikake, H., Yamanouchi, Y., Mitsumine, H., Yagi, N., Inoue, S., and Kikuchi, H. 2003. An effective interaction tool for performance in the virtual studio -- invisible light projection system. NHK Science and Technical Research Laboratories.

[8]

Graham, T., and Richard, R. 2005. Flash-based keying. European Patent Application EP1499117.

[9]

Grau, O., Pullen, T., and Thomas, G. A. 2004. A combined studio production system for 3-d capturing of live action and immersive actor feedback. IEEE Transactions on Circuits and Systems for Video Technology 14, 3, 370--380.

Digital Library

[10]

Grundhöfer, A., Seeger, M., Häntsch, F., and Bimber, O. 2007. Dynamic Adaptation of Projected Imperceptible Codes. Proc. of IEEE International Symposium on Mixed and Augmented Reality.

[11]

Nayar, S. K., Peri, H., Grossberg, M. D., and Belhumeur, P. N. 2003. A Projection System with Radiometric Compensation for Screen Imperfections. In Proc. of IEEE International Workshop on Projector-Camera Systems (ProCams).

[12]

Park, H., Lee, M.-H., Jin, B.-K. S. Y., and Park, J.-I. 2007. Content adaptive embedding of complementary patterns for non-intrusive direct-projected augmented reality. In HCI International 2007, vol. 14.

Digital Library

[13]

Peers, P., and Dutré, P. 2003. Wavelet environment matting. In EGRW '03: Proceedings of the 14th Eurographics workshop on Rendering, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 157--166.

Digital Library

[14]

Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. 1998. The office of the future: A unified approach to image-based modeling and spatially immersive displays. Computer Graphics 32, Annual Conference Series, 179--188.

[15]

Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual Photography. In Proc. of ACM SIGGRAPH, 745--755.

[16]

Shirai, A., Takahashi, M., Kobayashi, K., Mitsumine, H., and Richir, S. 2005. Lumina studio: Supportive information display for virtual studio environments. In Proc. of IEEE VR Workshop on Emerging Display Technologies, 17--20.

[17]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In ICCV, 839--846.

[18]

Vieira, M. B., Velho, L., Sa, A., and Carvalho, P. C. 2005. A camera-projector system for real-time 3d video. In CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) -- Workshops, IEEE Computer Society, Washington, DC, USA, 96.

[19]

Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., and Gross, M. H. 2005. Scalable 3d video of dynamic scenes. The Visual Computer 21, 8-10, 629--638.

[20]

Wetzstein, G., and Bimber, O. 2007. Radiometric Compensation through Inverse Light Transport. Proc. of Pacific Graphics.

[21]

Wojdala, A., Gruszsewski, M., and Olech, R. 2002. Real-time shadow casting in virtual studio. Machine Graphics and Vision (MGV) 9, 1/2, 315--329.

[22]

Yoshida, T., Horii, C., and Sato, K. 2003. A Virtual Color Reconstruction System for Real Heritage with Light Projection. In Proc. of International Conference on Virtual Systems and Multimedia (VSMM), 161--168.

[23]

Zhu, J., and Yang, Y.-H. 2004. Frequency-based environment matting. In PG '04: Proceedings of the Computer Graphics and Applications, 12th Pacific Conference, IEEE Computer Society, Washington, DC, USA, 402--410.

Digital Library

[24]

Zollmann, S., and Bimber, O. 2007. Imperceptible calibration for radiometric compensation. In Proceedings Eurographics 2007, Short Paper.

[25]

Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment matting and compositing. In SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 205--214.

Cited By

Da Costa JLeyva ROrdozgoith Da F S PSouto E(2024)Enhancing Virtual Reality Experiences through Embedded 3D Models in Video Content2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444434(1-5)Online publication date: 6-Jan-2024
https://doi.org/10.1109/ICCE59016.2024.10444434
Li HZhu WJin HMa Y(2021)Automatic, Illumination-Invariant and Real-Time Green-Screen Keying Using Deeply Guided Linear ModelsSymmetry10.3390/sym1308145413:8(1454)Online publication date: 9-Aug-2021
https://doi.org/10.3390/sym13081454
Grundhöfer AIwai D(2018)Recent Advances in Projection Mapping Algorithms, Hardware and ApplicationsComputer Graphics Forum10.1111/cgf.1338737:2(653-675)Online publication date: 22-May-2018
https://doi.org/10.1111/cgf.13387
Show More Cited By

Index Terms

VirtualStudio2Go: digital video composition for real environments
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

VirtualStudio2Go: digital video composition for real environments
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers

We synchronize film cameras and LED lighting with off-the-shelf video projectors. Radiometric compensation allows displaying keying patterns and other spatial codes on arbitrary real world surfaces. A fast temporal multiplexing of coded projection and ...
Dynamic bluescreens
SIGGRAPH '08: ACM SIGGRAPH 2008 posters

We synchronize cameras and analog lighting with high speed projectors. Radiometric compensation allows displaying flexible blue screens on arbitrary real world surfaces. A fast temporal multiplexing of coded projection and flash illumination enables ...
Immersive front-projection analysis using a radiosity-based simulation method

Video projectors are designed to project onto flat white diffuse screens. Over the last few years, projector-based systems have been used, in virtual reality applications, to light non-specific environments such as the walls of a room. However, in these ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 27, Issue 5

December 2008

552 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1409060

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008

Published in TOG Volume 27, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
701
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Da Costa JLeyva ROrdozgoith Da F S PSouto E(2024)Enhancing Virtual Reality Experiences through Embedded 3D Models in Video Content2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444434(1-5)Online publication date: 6-Jan-2024
https://doi.org/10.1109/ICCE59016.2024.10444434
Li HZhu WJin HMa Y(2021)Automatic, Illumination-Invariant and Real-Time Green-Screen Keying Using Deeply Guided Linear ModelsSymmetry10.3390/sym1308145413:8(1454)Online publication date: 9-Aug-2021
https://doi.org/10.3390/sym13081454
Grundhöfer AIwai D(2018)Recent Advances in Projection Mapping Algorithms, Hardware and ApplicationsComputer Graphics Forum10.1111/cgf.1338737:2(653-675)Online publication date: 22-May-2018
https://doi.org/10.1111/cgf.13387
Aksoy YAydin TPollefeys MSmolić A(2017)Interactive High-Quality Green-Screen Keying via Color UnmixingACM Transactions on Graphics10.1145/3072959.290794036:4(1)Online publication date: 16-Jul-2017
https://dl.acm.org/doi/10.1145/3072959.2907940
Aksoy YAydin TPollefeys MSmolić A(2016)Interactive High-Quality Green-Screen Keying via Color UnmixingACM Transactions on Graphics10.1145/290794035:5(1-12)Online publication date: 8-Aug-2016
https://dl.acm.org/doi/10.1145/2907940
Koch RSchiller IBartczak BKellner FKöser K(2009)MixIn3DProceedings of the DAGM 2009 Workshop on Dynamic 3D Imaging10.1007/978-3-642-03778-8_10(126-141)Online publication date: 3-Sep-2009
https://dl.acm.org/doi/10.1007/978-3-642-03778-8_10
Bimber OYang X(2009)Projector-Camera Systems in Entertainment and ArtHandbook of Multimedia for Digital Entertainment and Arts10.1007/978-0-387-89024-1_21(471-496)Online publication date: 9-Jun-2009
https://doi.org/10.1007/978-0-387-89024-1_21
Beato NZhang YColbert MYamazawa KHughes C(2009)Interactive chroma keying for mixed realityComputer Animation and Virtual Worlds10.1002/cav.30520:2-3(405-415)Online publication date: Jun-2009
https://doi.org/10.1002/cav.305

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents