article

Interactive video cutout

Authors:

R. Alex Colburn,

Maneesh Agrawala,

Michael F. CohenAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 24, Issue 3

Pages 585 - 594

https://doi.org/10.1145/1073204.1073233

Published: 01 July 2005 Publication History

Abstract

We present an interactive system for efficiently extracting foreground objects from a video. We extend previous min-cut based image segmentation techniques to the domain of video with four new contributions. We provide a novel painting-based user interface that allows users to easily indicate the foreground object across space and time. We introduce a hierarchical mean-shift preprocess in order to minimize the number of nodes that min-cut must operate on. Within the min-cut we also define new local cost functions to augment the global costs defined in earlier work. Finally, we extend 2D alpha matting methods designed for images to work with 3D video volumes. We demonstrate that our matting approach preserves smoothness across both space and time. Our interactive video cutout system allows users to quickly extract foreground objects from video sequences for use in a variety of applications including compositing onto new backgrounds and NPR cartoon style rendering.

Supplementary Material

MP4 File (pps023.mp4)

Download
38.44 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. In Proceedings of ACM SIGGRAPH. 294--302.

Digital Library

[2]

Agarwala, A., Hertzmann, A., Salesin, D. H., and Seitz, S. M. 2004. Keyframe-based tracking for rotoscoping and animation. In Proceedings of ACM SIGGRAPH, 584--591.

Digital Library

[3]

Belongie, S., Malik. J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 4, 509--522.

Digital Library

[4]

Bennett, E. P., and McMillan, L. 2003. Proscenium: A framework for spatio-temporal video editing. In Proceedings of ACM Multimedia, 177--183.

Digital Library

[5]

Blake, A., and Isard, M. 1998. Active Contours. Springer-Verlag.

[6]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 11, 1222--1239.

Digital Library

[7]

Chuang, Y.-Y., Curless, B., Salesin, D. H., and Szeliski, R. 2001. A bayesian approach to digital matting. In Proceedings of IEEE CVPR 2001, vol. 2, 264--271.

Digital Library

[8]

Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., and Szeliski, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3, 243--248.

Digital Library

[9]

Collomosse, J. P., Rowntree, D., and Hall, P. M. 2003. Stroke surfaces: A spatio-temporal framework for temporally coherent non-photorealistic animations. University of Bath, Technical Report CSBU 2003--01 (June 2003).

[10]

Comaniciu, D., Ramesh, V., and Meer, P. 2001. The variable bandwidth mean shift and data-driven scale selection. In Proc. IEEE 8th Int. Conf. on Computer Vision.

[11]

Dementhon, D., and Megret, R. 2002. Spatio-temporal segmentation of video by hierarchical mean shift analysis. In University of Maryland Technical Report LAMP-TR-090, CAR-TR-978, CS-TR-4388, UMIACS-TR-2002-68.

[12]

Fels, S. S., and Mase, K. 1999. Interactive video cubism. In Proceedings of the Workshop on New Paradigms for Interactive Visualization and Manipulation (NPIVM), 78--82.

Digital Library

[13]

Gleicher, M. 1995. Image snapping. In Proceedings of SIGGRAPH 95, 183--190.

Digital Library

[14]

Hall, J., Greenhill, D., and Jones, G. 1997. Segmenting film sequences using active surfaces. In International Conference on Image Processing (ICIP), 751--754.

Digital Library

[15]

Incorp., A. S. 2002. Adobe photoshop user guide.

[16]

Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. International Journal of Computer Vision 1, 4, 321--331.

[17]

Klein, A. W., Sloan, P.-P. J., Finkelstein, A., and Cohen, M. F. 2002. Stylized video cubes. In Proceedings of SCA 2002.

Digital Library

[18]

Kwatra, V., Shoedl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. In Proceedings of ACM SIGGRAPH, 277--286.

Digital Library

[19]

Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazysnapping. In Proceedings of ACM SIGGRAPH, 303--308.

Digital Library

[20]

Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), 674--679.

[21]

Luo, H., and Eleftheriadis, A. 1999. Spatial temporal active contour interpolation for semi-automatic video object generation. In International Conference on Image Processing (ICIP), 944--948.

[22]

Mortensen, E., and Barrett, W. 1995. Intelligent scissors for image composition. In Proceedings of ACM SIGGRAPH, 191--198.

Digital Library

[23]

Prez, P., Blake, A., and Gangnet, M. 2001. Jetstream: Probabilistic contour extraction with particles. In Proc. Int. Conf. on Computer Vision, vol. II, 524--531.

[24]

Reese, L. J., and Barrett, W. A. 2002. Image editing with intelligent paint. Proceedings of Eurographics 21, 3, 714--724.

[25]

Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut - interactive foreground extraction using iterated graph cut. In Proceedings of ACM SIGGRAPH, 309--314.

Digital Library

[26]

Ruzon, M., and Tomasi, C. 2000. Alpha estimation in natural images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. I, 18--25.

[27]

Wang, J., Xu, Y.-Q., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. In Proceedings of ACM SIGGRAPH, 574--583.

Digital Library

Cited By

Varga VSzász M(2024)Solving Interactive Video Object Segmentation with Label-Propagating Neural Networks2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650871(1-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650871
Liu QCho JBansal MNiethammer M(2024)Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00362(3773-3782)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00362
Ingle PKim Y(2023)Video Synopsis Algorithms and Framework: A Survey and Comparative EvaluationSystems10.3390/systems1102010811:2(108)Online publication date: 17-Feb-2023
https://doi.org/10.3390/systems11020108
Show More Cited By

Index Terms

Interactive video cutout
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
      1. Paths and connectivity problems

Recommendations

Interactive video cutout
SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

We present an interactive system for efficiently extracting foreground objects from a video. We extend previous min-cut based image segmentation techniques to the domain of video with four new contributions. We provide a novel painting-based user ...
The Video Matting Based on Background Reconstruction and Prediction
ETCS '11: Proceedings of the 2011 Third International Workshop on Education Technology and Computer Science - Volume 01

In this paper, we propose a new video matting method based on background reconstruction and prediction. Different from image matting technique, video matting can benefit from temporal consistency. so we can predict or reconstruct the background from ...
Extracting the Foreground from Video Based on a New Sampling Method
Transactions on Edutainment XI - Volume 8971

In this paper, we propose a new video matting method based on sampling. By detecting the movement of foreground and background objects from video, we define the local transformation which transfer the small areas between different frames in the video. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 24, Issue 3

July 2005

826 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1073204

Issue’s Table of Contents

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2005

Published in TOG Volume 24, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

206
Total Citations
View Citations
2,722
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Varga VSzász M(2024)Solving Interactive Video Object Segmentation with Label-Propagating Neural Networks2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650871(1-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650871
Liu QCho JBansal MNiethammer M(2024)Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00362(3773-3782)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00362
Ingle PKim Y(2023)Video Synopsis Algorithms and Framework: A Survey and Comparative EvaluationSystems10.3390/systems1102010811:2(108)Online publication date: 17-Feb-2023
https://doi.org/10.3390/systems11020108
Lin GGao CHuang JKim CWang YZwicker MSaraf A(2023)OmnimatteRF: Robust Omnimatte with 3D Background Modeling2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02145(23414-23423)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.02145
Oh SLee JXu NKim S(2022)Space-Time Memory Networks for Video Object Segmentation With User GuidanceIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.300891744:1(442-455)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TPAMI.2020.3008917
Zhuang JWang ZWang B(2021)Video Semantic Segmentation With Distortion-Aware Feature CorrectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2020.303723431:8(3128-3139)Online publication date: Aug-2021
https://doi.org/10.1109/TCSVT.2020.3037234
Heo YKoh YKim C(2021)Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.00724(7318-7326)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.00724
Yao GJiang DSun J(2021)An Affinity Based Matting Method Based on Multi-Scale Space Fusion2021 33rd Chinese Control and Decision Conference (CCDC)10.1109/CCDC52312.2021.9601598(1572-1577)Online publication date: 22-May-2021
https://doi.org/10.1109/CCDC52312.2021.9601598
Jia J(2021)Matte ExtractionComputer Vision10.1007/978-3-030-63416-2_12(795-799)Online publication date: 13-Oct-2021
https://doi.org/10.1007/978-3-030-63416-2_12
Yao RLin GXia SZhao JZhou Y(2020)Video Object Segmentation and TrackingACM Transactions on Intelligent Systems and Technology10.1145/339174311:4(1-47)Online publication date: 25-May-2020
https://dl.acm.org/doi/10.1145/3391743
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents