research-article

Improved seam carving for video retargeting

Authors:

Michael Rubinstein,

Shai AvidanAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 27, Issue 3

Pages 1 - 9

https://doi.org/10.1145/1360612.1360615

Published: 01 August 2008 Publication History

Abstract

Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes. In the new formulation, a seam is given by a minimal cut in the graph and we show how to construct a graph such that the resulting cut is a valid seam. That is, the cut is monotonic and connected. In addition, we present a novel energy criterion that improves the visual quality of the retargeted images and videos. The original seam carving operator is focused on removing seams with the least amount of energy, ignoring energy that is introduced into the images and video by applying the operator. To counter this, the new criterion is looking forward in time - removing seams that introduce the least amount of energy into the retargeted result. We show how to encode the improved criterion into graph cuts (for images and video) as well as dynamic programming (for images). We apply our technique to images and videos and present results of various applications.

Supplementary Material

MOV File (a16-rubinstein.mov)

Download
28.59 MB

References

[1]

Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10.

Digital Library

[2]

Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26, 9, 1124--1137.

Digital Library

[3]

Chen, B., and Sen, P. 2008. Video carving. In Short Papers Proceedings of Eurographics.

[4]

Fan, X., Xie, X., Zhou, H.-Q., and Ma, W.-Y. 2003. Looking into video frames on small displays. In MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia, ACM, 247--250.

Digital Library

[5]

Kohli, P., and Torr, P. H. S. 2007. Dynamic graph cuts for efficient inference in markov random fields. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 29, 12, 2079--2088.

Digital Library

[6]

Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3, 277--286.

Digital Library

[7]

Liu, F., and Gleicher, M. 2006. Video retargeting: automating pan and scan. In MULTIMEDIA '06: Proceedings of the 14th annual ACM international conference on Multimedia, ACM, 241--250.

Digital Library

[8]

Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05), vol. 1, 259--265.

Digital Library

[9]

Pritch, Y., Rav-Acha, A., and Peleg, S. 2008. Nonchronological video synopsis and indexing. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), to appear.

Digital Library

[10]

Rav-Acha, A., Pritch, Y., Lischinski, D., and Peleg, S. 2007. Dynamosaicing: Mosaicing of dynamic scenes. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 29, 10, 1789--1801.

Digital Library

[11]

Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 489--498.

Digital Library

[12]

Setlur, V., Takagi, S., Raskar, R., Gleicher, M., and Gooch, B. 2005. Automatic image retargeting. In In the Mobile and Ubiquitous Multimedia (MUM), ACM Press.

Digital Library

[13]

Tao, C., Jia, J., and Sun, H. 2007. Active window oriented dynamic video retargeting. In Proceedings of the Workshop on Dynamical Vision, ICCV 2007.

[14]

Viola, P., and Jones, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57 2, 137--154.

Digital Library

[15]

Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583.

Digital Library

[16]

Wang, J., Reinders, M., Lagendijk, R., Lindenberg, J., and Kankanhalli, M. 2004. Video content presentation on tiny devices. In IEEE International Conference on Multimedia and Expo (ICME), vol. 3, 1711--1714.

[17]

Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graph. 24, 3, 585--594.

Digital Library

[18]

Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Nonhomogeneous content-driven video-retargeting. In Proceedings of the Eleventh IEEE International Conference on Computer Vision (ICCV '07), 1--6.

Cited By

Dickman EDiefenbach P(2024)Directorial Editing: A Hybrid Deep-Learning Approach to Content-Aware Image Retargeting and ResizingElectronics10.3390/electronics1322445913:22(4459)Online publication date: 14-Nov-2024
https://doi.org/10.3390/electronics13224459
Li XLi ZZhou LHuang Z(2024)FOLD: Low-Level Image Enhancement for Low-Light Object Detection Based on FPGA MPSoCElectronics10.3390/electronics1301023013:1(230)Online publication date: 4-Jan-2024
https://doi.org/10.3390/electronics13010230
Zhang LChen MTu BLi YXia Y(2024)Retargeting HR Aerial Photos Under Contaminated Labels With Application in Smart NavigationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.328887725:1(349-358)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TITS.2023.3288877
Show More Cited By

Index Terms

Improved seam carving for video retargeting
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization
  2. Computer graphics

Recommendations

Seam carving for content-aware image resizing

Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is ...
Sparse Seam-Carving for Structure Preserving Image Retargeting

The seam carving technique is one of the most popular techniques for content-aware image retargeting. There have been many attempts to improve existing methods of seam carving to provide more aesthetic retargeting results. However, most previous methods ...
Reverse Seam Carving
ICIG '11: Proceedings of the 2011 Sixth International Conference on Image and Graphics

Seam carving is an effective operator supporting content-aware resizing for both image reduction and expansion. However, repeated seam removing and inserting processes lead to excessively distortion image when imposed on seam insertion then removal ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 27, Issue 3

August 2008

844 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1360612

Issue’s Table of Contents

Copyright © 2008 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2008

Published in TOG Volume 27, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

570
Total Citations
View Citations
4,222
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)15

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dickman EDiefenbach P(2024)Directorial Editing: A Hybrid Deep-Learning Approach to Content-Aware Image Retargeting and ResizingElectronics10.3390/electronics1322445913:22(4459)Online publication date: 14-Nov-2024
https://doi.org/10.3390/electronics13224459
Li XLi ZZhou LHuang Z(2024)FOLD: Low-Level Image Enhancement for Low-Light Object Detection Based on FPGA MPSoCElectronics10.3390/electronics1301023013:1(230)Online publication date: 4-Jan-2024
https://doi.org/10.3390/electronics13010230
Zhang LChen MTu BLi YXia Y(2024)Retargeting HR Aerial Photos Under Contaminated Labels With Application in Smart NavigationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.328887725:1(349-358)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TITS.2023.3288877
Lee CMandal MBirkbeck NWang YAdsumilli BBovik A(2024)Subjective Portrait Region Cropping On Landscape Video Study2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10648155(48-54)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10648155
Alswaidan ZShullar MChikhaoui KAlfarraj M(2024)Object-Aware Adaptive Image Retargeting Via Importance Map Fusion2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10648064(1528-1533)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10648064
Fan LCai SHe CLi XZheng L(2024)Geometry-Based Feature Selection and Deep Aggregation Model for Architectural Scenery Recomposition Toward EducationIEEE Access10.1109/ACCESS.2024.349220512(163724-163738)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3492205
Ma P(2024)Soccer Training Optimization for Education: A Multi-Layer Architecture Simulating How Observers Understand Soccer SceneriesIEEE Access10.1109/ACCESS.2024.345272512(125510-125522)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3452725
Zhang YLiu LZhu Y(2024)Application of Multimodal Feature Selection-Based Scene Recognition for Medical EducationIEEE Access10.1109/ACCESS.2024.340968612(87934-87943)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3409686
Xu KMousavi VLiu D(2024)Retargeting Low-Resolution Aerial Imagery by Distribution-Preserving Perceptual Feature SelectionIEEE Access10.1109/ACCESS.2024.336439912(25612-25622)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3364399
Zhang XXiao YPeng JGao XHu B(2024)Confidence-based dynamic cross-modal memory network for image aesthetic assessmentPattern Recognition10.1016/j.patcog.2023.110227149(110227)Online publication date: May-2024
https://doi.org/10.1016/j.patcog.2023.110227
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents