research-article

Improved seam carving for video retargeting

Authors:

Michael Rubinstein,

Shai AvidanAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 27, Issue 3

Pages 1 - 9

https://doi.org/10.1145/1360612.1360615

Published: 01 August 2008 Publication History

Abstract

Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes. In the new formulation, a seam is given by a minimal cut in the graph and we show how to construct a graph such that the resulting cut is a valid seam. That is, the cut is monotonic and connected. In addition, we present a novel energy criterion that improves the visual quality of the retargeted images and videos. The original seam carving operator is focused on removing seams with the least amount of energy, ignoring energy that is introduced into the images and video by applying the operator. To counter this, the new criterion is looking forward in time - removing seams that introduce the least amount of energy into the retargeted result. We show how to encode the improved criterion into graph cuts (for images and video) as well as dynamic programming (for images). We apply our technique to images and videos and present results of various applications.

Supplementary Material

MOV File (a16-rubinstein.mov)

Download
28.59 MB

References

[1]

Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10.

Digital Library

[2]

Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26, 9, 1124--1137.

Digital Library

[3]

Chen, B., and Sen, P. 2008. Video carving. In Short Papers Proceedings of Eurographics.

[4]

Fan, X., Xie, X., Zhou, H.-Q., and Ma, W.-Y. 2003. Looking into video frames on small displays. In MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia, ACM, 247--250.

Digital Library

[5]

Kohli, P., and Torr, P. H. S. 2007. Dynamic graph cuts for efficient inference in markov random fields. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 29, 12, 2079--2088.

Digital Library

[6]

Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3, 277--286.

Digital Library

[7]

Liu, F., and Gleicher, M. 2006. Video retargeting: automating pan and scan. In MULTIMEDIA '06: Proceedings of the 14th annual ACM international conference on Multimedia, ACM, 241--250.

Digital Library

[8]

Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05), vol. 1, 259--265.

Digital Library

[9]

Pritch, Y., Rav-Acha, A., and Peleg, S. 2008. Nonchronological video synopsis and indexing. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), to appear.

Digital Library

[10]

Rav-Acha, A., Pritch, Y., Lischinski, D., and Peleg, S. 2007. Dynamosaicing: Mosaicing of dynamic scenes. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 29, 10, 1789--1801.

Digital Library

[11]

Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 489--498.

Digital Library

[12]

Setlur, V., Takagi, S., Raskar, R., Gleicher, M., and Gooch, B. 2005. Automatic image retargeting. In In the Mobile and Ubiquitous Multimedia (MUM), ACM Press.

Digital Library

[13]

Tao, C., Jia, J., and Sun, H. 2007. Active window oriented dynamic video retargeting. In Proceedings of the Workshop on Dynamical Vision, ICCV 2007.

[14]

Viola, P., and Jones, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57 2, 137--154.

Digital Library

[15]

Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583.

Digital Library

[16]

Wang, J., Reinders, M., Lagendijk, R., Lindenberg, J., and Kankanhalli, M. 2004. Video content presentation on tiny devices. In IEEE International Conference on Multimedia and Expo (ICME), vol. 3, 1711--1714.

[17]

Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graph. 24, 3, 585--594.

Digital Library

[18]

Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Nonhomogeneous content-driven video-retargeting. In Proceedings of the Eleventh IEEE International Conference on Computer Vision (ICCV '07), 1--6.

Cited By

Du ZYu Y(2025)Efficiency-Sensitive Feature Selection and Probabilistic Feature Transfer for Scenic Image RecompositionIEEE Access10.1109/ACCESS.2025.354631013(38430-38442)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3546310
Chen JLv ZJiao GXia MYang G(2025)SPNet: Seam carving detection via spatial-phase learningJournal of Information Security and Applications10.1016/j.jisa.2025.10396389(103963)Online publication date: Mar-2025
https://doi.org/10.1016/j.jisa.2025.103963
Tang ZZhang YZhang X(2025)Visual comfort and depth perception measurement for stereoscopic image retargeting quality assessmentComputers & Graphics10.1016/j.cag.2025.104179127(104179)Online publication date: Apr-2025
https://doi.org/10.1016/j.cag.2025.104179
Show More Cited By

Index Terms

Improved seam carving for video retargeting
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization
  2. Computer graphics

Recommendations

Seam carving for content-aware image resizing

Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is ...
Sparse Seam-Carving for Structure Preserving Image Retargeting

The seam carving technique is one of the most popular techniques for content-aware image retargeting. There have been many attempts to improve existing methods of seam carving to provide more aesthetic retargeting results. However, most previous methods ...
Reverse Seam Carving
ICIG '11: Proceedings of the 2011 Sixth International Conference on Image and Graphics

Seam carving is an effective operator supporting content-aware resizing for both image reduction and expansion. However, repeated seam removing and inserting processes lead to excessively distortion image when imposed on seam insertion then removal ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 27, Issue 3

August 2008

844 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1360612

Issue’s Table of Contents

Copyright © 2008 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2008

Published in TOG Volume 27, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

578
Total Citations
View Citations
4,249
Total Downloads

Downloads (Last 12 months)105
Downloads (Last 6 weeks)11

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Du ZYu Y(2025)Efficiency-Sensitive Feature Selection and Probabilistic Feature Transfer for Scenic Image RecompositionIEEE Access10.1109/ACCESS.2025.354631013(38430-38442)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3546310
Chen JLv ZJiao GXia MYang G(2025)SPNet: Seam carving detection via spatial-phase learningJournal of Information Security and Applications10.1016/j.jisa.2025.10396389(103963)Online publication date: Mar-2025
https://doi.org/10.1016/j.jisa.2025.103963
Tang ZZhang YZhang X(2025)Visual comfort and depth perception measurement for stereoscopic image retargeting quality assessmentComputers & Graphics10.1016/j.cag.2025.104179127(104179)Online publication date: Apr-2025
https://doi.org/10.1016/j.cag.2025.104179
Mujahid MKhan AHossain MLaulai AJaveed M(2025)Reinforcement Learning Based Improved Seam Carving Using Special Point and Alpha Value for Optimal Content PreservationAdvances in Computer Graphics10.1007/978-3-031-82024-3_3(30-41)Online publication date: 25-Feb-2025
https://doi.org/10.1007/978-3-031-82024-3_3
Dickman EDiefenbach P(2024)Directorial Editing: A Hybrid Deep-Learning Approach to Content-Aware Image Retargeting and ResizingElectronics10.3390/electronics1322445913:22(4459)Online publication date: 14-Nov-2024
https://doi.org/10.3390/electronics13224459
Li XLi ZZhou LHuang Z(2024)FOLD: Low-Level Image Enhancement for Low-Light Object Detection Based on FPGA MPSoCElectronics10.3390/electronics1301023013:1(230)Online publication date: 4-Jan-2024
https://doi.org/10.3390/electronics13010230
Zhu PZhang YLi JPeng JXie L(2024)Salient context aware for improved image resizingFifteenth International Conference on Information Optics and Photonics (CIOP 2024)10.1117/12.3048590(141)Online publication date: 5-Dec-2024
https://doi.org/10.1117/12.3048590
Zhang LChen MTu BLi YXia Y(2024)Retargeting HR Aerial Photos Under Contaminated Labels With Application in Smart NavigationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.328887725:1(349-358)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TITS.2023.3288877
Mei YYang LWang MGao YWu K(2024)Faster, Lighter, Stronger: Image Rectangling Using Multi-Teacher Instance-Level DistillationIEEE Transactions on Consumer Electronics10.1109/TCE.2024.341645770:3(5441-5452)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCE.2024.3416457
Lee CMandal MBirkbeck NWang YAdsumilli BBovik A(2024)Subjective Portrait Region Cropping On Landscape Video Study2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10648155(48-54)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10648155
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents