research-article

Gigapixel Panorama Video Loops

Authors:

Pedro V. Sander,

Hugues HoppeAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 37, Issue 1

Article No.: 3, Pages 1 - 15

https://doi.org/10.1145/3144455

Published: 16 November 2017 Publication History

Abstract

We present the first technique to create wide-angle, high-resolution looping panoramic videos. Starting with a 2D grid of registered videos acquired on a robotic mount, we formulate a combinatorial optimization to determine for each output pixel the source video and looping parameters that jointly maximize spatiotemporal consistency. This optimization is accelerated by reducing the set of source labels using a graph-coloring scheme. We parallelize the computation and implement it out-of-core by partitioning the domain along low-importance paths. The merged panorama is assembled using gradient-domain blending and stored as a hierarchy of video tiles. Finally, an interactive viewer adaptively preloads these tiles for responsive browsing and allows the user to interactively edit and improve local regions. We demonstrate these techniques on gigapixel-sized looping panoramas.

Supplementary Material

MP4 File (tog37-1-a3-he.mp4)

Download
231.14 MB

References

[1]

Aseem Agarwala. 2007. Efficient gradient-domain compositing using quadtrees. ACM Trans. Graph. 26, 3 (2007), 94.

Digital Library

[2]

A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen. 2004. Interactive Digital Photomontage. ACM Trans. Graph. 23, 3 (2004).

Digital Library

[3]

Aseem Agarwala, Ke Colin Zheng, Chris Pal, Maneesh Agrawala, Michael Cohen, Brian Curless, David Salesin, and Richard Szeliski. 2005. Panoramic video textures. ACM Trans. Graph. 24, 3 (July 2005), 7.

Digital Library

[4]

Jiamin Bai, Aseem Agarwala, Maneesh Agrawala, and Ravi Ramamoorthi. 2012. Selectively De-Animating Video. ACM Trans. Graph. 31, 4 (2012).

Digital Library

[5]

Jiamin Bai, Aseem Agarwala, Maneesh Agrawala, and Ravi Ramamoorthi. 2013. Automatic cinemagraph portraits. Computer Graphics Forum 32, 4 (2013), 17--25.

Digital Library

[6]

Jamie Beck and Kevin Burg. 2012. Cinemagraphs. http://cinemagraphs.com/. (2012).

[7]

Jean-Yves Bouguet. 2001. Pyramidal implementation of the affine Lucas-Kanade feature tracker: Description of the algorithm. Intel Corporation 5, 1--10 (2001), 4.

[8]

Matthew Brown and David G. Lowe. 2007. Automatic panoramic image stitching using invariant features. IJCV 74, 1 (2007), 59--73.

Digital Library

[9]

V. Couture, M. Langer, and S. Roy. 2011. Panoramic stereo video textures. ICCV (2011), 1251--1258.

Digital Library

[10]

Vincent Couture, Michael S. Langer, and Sébastien Roy. 2012. Perception of blending in stereo motion panoramas. ACM Trans. Applied Perception 9, 3 (2012), 15.

Digital Library

[11]

Dan B. Goldman. 2010. Vignette and exposure calibration and compensation. IEEE transactions on pattern analysis and machine intelligence 32, 12 (2010), 2276--2288.

Digital Library

[12]

João F. Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 583--596.

Digital Library

[13]

Chris Hermans, Cedric Vanaken, Tom Mertens, Frank Van Reeth, and Philippe Bekaert. 2008. Augmented panoramic video. In Computer Graphics Forum, Vol. 27.

[14]

Cheuk Yiu Ip and Amitabh Varshney. 2011. Saliency-assisted navigation of very large landscape images. IEEE TVCG 17, 12 (2011).

Digital Library

[15]

Neel Joshi, Sisil Mehta, Steven Drucker, Eric Stollnitz, Hugues Hoppe, Matt Uyttendaele, and Michael Cohen. 2012. Cliplets: Juxtaposing still and dynamic imagery. Proc. of UIST (2012).

Digital Library

[16]

George Karypis. 2003. Multi-constraint mesh partitioning for contact/impact computations. In Proc. of ACM/IEEE conference on Supercomputing. ACM, 56.

Digital Library

[17]

M. Kazhdan and H. Hoppe. 2008. Streaming Multigrid for Gradient-Domain Operations on Large Images. ACM Trans. Graph. 27, 3 (2008).

Digital Library

[18]

Michael Kazhdan, Dinoj Surendran, and Hugues Hoppe. 2010. Distributed gradient-domain processing of planar and spherical images. ACM Trans. Graph. 29, 2 (2010).

Digital Library

[19]

V. Kolmogorov and R. Zabih. 2004. What Energy Functions can be Minimized via Graph Cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26, 2 (2004).

Digital Library

[20]

Johannes Kopf, Matt Uyttendaele, Oliver Deussen, and Michael F. Cohen. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. 26, 3 (2007), 93.

Digital Library

[21]

Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, and Aaron Bobick. 2003. Graphcut textures: image and video synthesis using graph cuts. In ACM Trans. Graph., Vol. 22. 277--286.

Digital Library

[22]

Jing Liao, Mark Finch, and Hugues Hoppe. 2015. Fast computation of seamless video loops. ACM Trans. Graph. 34, 6 (2015), 197.

Digital Library

[23]

Jing Liao, Neel Joshi, and Hugues Hoppe. 2013. Automated video looping with progressive dynamism. ACM Trans. Graph. 32, 4 (2013).

Digital Library

[24]

Jiangyu Liu and Jian Sun. 2010. Parallel graph-cuts by adaptive bottom-up merging. In Proc. CVPR. IEEE, 2181--2188.

[25]

Qing Luan, Steven M. Drucker, Johannes Kopf, Ying-Qing Xu, and Michael F. Cohen. 2008. Annotating gigapixel images. In Proc. ACM UIST. 33--36.

Digital Library

[26]

Microsoft Research. 2016. Image Composite Editor. http://research.microsoft.com/en-us/um/redmond/groups/ivm/ICE/. (2016).

[27]

Federico Perazzi, Alexander Sorkine-Hornung, Henning Zimmer, Peter Kaufmann, Oliver Wang, S. Watson, and Markus H. Gross. 2015. Panoramic video from unstructured camera arrays. Computer Graphics Forum 34, 2 (2015), 57--68.

Digital Library

[28]

Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. ACM Trans. Graph. 22, 3 (2003).

Digital Library

[29]

Sujin Philip, Brian Summa, Valerio Pascucci, and P.-T. Bremer. 2011. Hybrid CPU-GPU solver for gradient domain processing of massive images. In Intl. Conf. on Parallel and Distributed Systems.

Digital Library

[30]

Sujin Philip, Brian Summa, Julien Tierny, Peer-Timo Bremer, and Valerio Pascucci. 2015. Distributed Seams for Gigapixel Panoramas. IEEE TVCG 21, 3 (2015), 350--362.

[31]

Sören Pirk, Michael F. Cohen, Oliver Deussen, Matt Uyttendaele, and Johannes Kopf. 2012. Video enhanced gigapixel panoramas. SIGGRAPH Asia 2012 Technical Briefs (2012).

Digital Library

[32]

Alex Rav-Acha, Yael Pritch, Dani Lischinski, and Shmuel Peleg. 2007. Dynamosaicing: Mosaicing of Dynamic Scenes. IEEE Trans. on Pattern Anal. Mach. Intell. 29, 10 (2007).

Digital Library

[33]

Arno Schödl, Richard Szeliski, David H. Salesin, and Irfan Essa. 2000. Video textures. In Proc. SIGGRAPH. 489--498.

Digital Library

[34]

Laura Sevilla-Lara, Jonas Wulff, Kalyan Sunkavalli, and Eli Shechtman. 2015. Smooth Loops from Unconstrained Video. Computer Graphics Forum 34, 4 (2015), 99--107.

Digital Library

[35]

Petter Strandmark and Fredrik Kahl. 2010. Parallel and distributed graph cuts by dual decomposition. In Proc. CVPR. IEEE, 2085--2092.

[36]

Brian Summa, Giorgio Scorzelli, Ming Jiang, Peer-Timo Bremer, and Valerio Pascucci. 2011. Interactive editing of massive imagery made simple: Turning Atlanta into Atlantis. ACM Transactions on Graphics (TOG) 30, 2 (2011), 7.

Digital Library

[37]

Brian Summa, Julien Tierny, and Valerio Pascucci. 2012. Panorama weaving: Fast and flexible seam processing. ACM Trans. Graph. 31, 4 (2012).

Digital Library

[38]

Richard Szeliski. 2006. Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Vision 2, 1 (2006).

Digital Library

[39]

Richard Szeliski and Heung-Yeung Shum. 1997. Creating full view panoramic image mosaics and environment maps. In Proc. ACM SIGGRAPH.

Digital Library

[40]

James Tompkin, Fabrizio Pece, Rajvi Shah, Shahram Izadi, Jan Kautz, and Christian Theobalt. 2013. Video collections in panoramic contexts. In Proc. ACM UIST.

Digital Library

[41]

James Tompkin, Fabrizio Pece, Kartic Subr, and Jan Kautz. 2011. Towards Moment Images: Automatic Cinemagraphs. In Proc. of the 8th European Conference on Visual Media Production (CVMP 2011).

Digital Library

Cited By

Mahapatra ASiarohin ALee HTulyakov SZhu J(2023)Text-Guided Synthesis of Eulerian CinemagraphsACM Transactions on Graphics10.1145/361832642:6(1-13)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618326
Jia SWang SLi TChern A(2023)Physical Cyclic AnimationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069386:3(1-18)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.1145/3606938
Ma LLi XLiao JSander P(2023)3D Video Loops from Asynchronous Input2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00038(310-320)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00038
Show More Cited By

Index Terms

Gigapixel Panorama Video Loops
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Computational photography

Recommendations

Fast computation of seamless video loops

Short looping videos concisely capture the dynamism of natural scenes. Creating seamless loops usually involves maximizing spatiotemporal consistency and applying Poisson blending. We take an end-to-end view of the problem and present new techniques ...
Automated video looping with progressive dynamism

Given a short video we create a representation that captures a spectrum of looping videos with varying levels of dynamism, ranging from a static image to a highly animated loop. In such a progressively dynamic video, scene liveliness can be adjusted ...
Towards Moment Imagery: Automatic Cinemagraphs
CVMP '11: Proceedings of the 2011 Conference for Visual Media Production

The imagination of the online photographic community has recently been sparked by so-called cinema graphs: short, seamlessly looping animated GIF images created from video in which only parts of the image move. These cinema graphs capture the dynamics ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 37, Issue 1

February 2018

167 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3151031

Editor:
Kavita Bala
Cornell University

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 November 2017

Accepted: 01 September 2017

Revised: 01 July 2017

Received: 01 February 2017

Published in TOG Volume 37, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Hong Kong GRF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
661
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mahapatra ASiarohin ALee HTulyakov SZhu J(2023)Text-Guided Synthesis of Eulerian CinemagraphsACM Transactions on Graphics10.1145/361832642:6(1-13)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618326
Jia SWang SLi TChern A(2023)Physical Cyclic AnimationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069386:3(1-18)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.1145/3606938
Ma LLi XLiao JSander P(2023)3D Video Loops from Asynchronous Input2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00038(310-320)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00038
Chen X(2022)Sports Video Panorama Synthesis Technology Based on Edge Computing and Video Shot Boundary DetectionWireless Communications & Mobile Computing10.1155/2022/40608522022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/4060852
Thonat TAksoy YAittala MParis SDurand FDrettakis G(2021)Video‐Based Rendering of Dynamic Stationary Environments from Unsynchronized InputsComputer Graphics Forum10.1111/cgf.1434240:4(73-86)Online publication date: 15-Jul-2021
https://doi.org/10.1111/cgf.14342
Wehrwein SBala KSnavely N(2021)Scene Summarization via Motion NormalizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.299319527:4(2495-2501)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TVCG.2020.2993195
Ferreira CSoares FPedrini HBruce NFerreira Wda Cruz G(2020)A Study of Dimensionality Reduction Impact on an Approach to People Detection in Gigapixel ImagesCanadian Journal of Electrical and Computer Engineering10.1109/CJECE.2019.292578043:3(122-128)Online publication date: Oct-2021
https://doi.org/10.1109/CJECE.2019.2925780
Li XZhang BLiao JSander P(2019)Document rectification and illumination correction using a patch-based CNNACM Transactions on Graphics10.1145/3355089.335656338:6(1-11)Online publication date: 8-Nov-2019
https://dl.acm.org/doi/10.1145/3355089.3356563
Liu SAgrawala MDiVerdi SHertzmann AGuimbretière FBernstein MReinecke K(2019)View-Dependent Video Textures for 360° VideoProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology10.1145/3332165.3347887(249-262)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3332165.3347887
Kluge MWeyrich TKolb A(2019)Progressive Refinement ImagingComputer Graphics Forum10.1111/cgf.1380839:1(360-374)Online publication date: 4-Sep-2019
https://doi.org/10.1111/cgf.13808
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents