Article

AVE: automated home video editing

Authors:

Xian-Sheng HUA,

Hong-Jiang ZHANGAuthors Info & Claims

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

Pages 490 - 497

https://doi.org/10.1145/957013.957121

Published: 02 November 2003 Publication History

Abstract

In this paper, we present a system that automates home video editing. This system automatically extracts a set of highlight segments from a set of raw home videos and aligns them with user supplied incidental music based on the content of the video and incidental music. We developed an approach for extracting temporal structure and determining the importance of a video segment in order to facilitate the selection of highlight segments. Additionally we extract temporal structure, beats and tempos from the incidental music. In order to create more professional-looking results, the selected highlight segments satisfy a set of editing rules and are matched to the content of the incidental music. This task is formulated as a non-linear 0-1 programming problem and the rules are embedded as constraints. The output video is rendered by connecting the selected highlight video segments with transition effects and the incidental music.

Supplementary Material

MPG File (957121.mpg)

Download
31.30 MB

References

[1]

Foote, J., Cooper, M., and Girgensohn, A. Creating Music Videos Using Automatic Media Analysis. ACM Multimedia 2002.

Digital Library

[2]

Gong, Y.H., and Liu, X. Video Summarization Using Singular Value Decomposition. Proc. of CVPR, June, 2000.

[3]

Hanjalic, A., Lagendijk, R. L., and J. Biemond. Automated Highlevel Movie Segmentation for Advanced Video-Retrieval Systems. IEEE Trans on Circuits and Systems For Video Technology, Vol. 9, No. 4, June 1999, 580--588.

Digital Library

[4]

Itti, L. Real-Time High-Performance Attention Focusing in Outdoors Color Video Streams. Proc. SPIE Human Vision and Electronic Imaging IV (HVEI'02), Jan 2002.

[5]

Jeho, N., and Ahmed, H.T. Dynamic Video Summarization and Visualization. Proc. of ACM Multimedia, October 1999.

Digital Library

[6]

Kender, J. R., and Yeo, B. L. Video Scene Segmentation via Continuous Video Coherence. Proc IEEE Intl Conf on Computer Vision and Pattern Recognition 1998, 367--373.

Digital Library

[7]

Li, S.Z. et al. Statistical Learning of Multi-View Face Detection. Proc. of ECCV 2002.

Digital Library

[8]

Lin, T. and Zhang, H.J. Video Scene Extraction by Force Competition. ICME, 2001.

[9]

Lu, L., Jiang, H. and Zhang, H. J. A Robust Audio Classification and Segmentation Method. ACM Multimedia 2001.

Digital Library

[10]

Ma, Y. F., Lu, L., Zhang, H.J., and Li, M.J. A User Attention Model for Video Summarization. ACM Multimedia 2002, 533--542.

Digital Library

[11]

Matsuo, Y., Amano, M., and Uehara K. Mining Video Editing Rules in Video Streams. ACM Multimedia 2002, 255--258.

Digital Library

[12]

Omoigui, N., He, L., Gupta, A., Grudin, J., and Sanoki, E. Time-compression: System Concerns, Usage, and Benefits. Proc. of ACM ICH 1999.

Digital Library

[13]

Orriols, X., and Binefa, X. An EM Algorithm for Video Summarization, Generative Model Approach. ICCV 2001.

[14]

Scheirer, E. Tempo and Beat Aalysis of Acoustic Musical Signals. Journal of the Acoustical Society of America, 103(1), 588--601, 1998.

[15]

Smith, M.A., and Kanade, T. Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques. Proc. of Computer Vision and Pattern Recognition 1997.

Digital Library

[16]

Stefanidis, A., Partsinevelos, P., Agouris, P., and Doucette, P. Summarizing Video Datasets in the Spatiotemporal Domain. Proc. of 11th Intl. Workshop on Database and Expert Systems Applications, 2000.

Digital Library

[17]

Sundaram, H., Xie, L., and Chang, S.F. A Utility Framework for the Automatic Generation of Audio-Visual Skims. ACM Multimedia 2002.

Digital Library

[18]

Whitley, D. A Genetic Algorithm Tutorial. Statistics and Computing, Vol. 4, 64--85, 1994.

[19]

Yan, W.Q., Kankanhalli, M. Detection and Removal of Lighting & Shaking Artifacts in Home Videos. ACM Multimedia 2002, 107--116.

Digital Library

[20]

Zhang, H.J., Kankanhalli, A., and Smoliar, S.W. Automatic Partitioning of Full-Motion Video. Multimedia Systems, 1, 10-2, 1993.

Digital Library

Cited By

Lin DCaba Heilbron FLee JWang OMartelaro N(2024)VideoMap: Supporting Video Exploration, Brainstorming, and Prototyping in the Latent SpaceProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656192(311-327)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3635636.3656192
Zhang LDenney BLu J(2016)Sub-event recognition and summarization for structured scenario photosMultimedia Tools and Applications10.1007/s11042-016-3346-x75:15(9295-9314)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11042-016-3346-x
Bano SCavallaro A(2016)ViCompMultimedia Tools and Applications10.1007/s11042-015-2641-275:12(7187-7210)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1007/s11042-015-2641-2
Show More Cited By

Index Terms

AVE: automated home video editing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Automatic music video generation based on temporal pattern analysis
MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

Music video (MV) is a short film meant to present a visual representation of a popular music song. In this paper, we present a system that automatically generates MV-like videos from personal home videos based on observations that generally there are ...
Automatic Mash Up Music Video Generation System by Remixing Existing Video Content
CULTURECOMPUTING '13: Proceedings of the 2013 International Conference on Culture and Computing

Music video is a short film which presents a visual representation of recent music. In these days, there is a trend that amateur users create music video in the video sharing website. Especially, the music video which is created by cutting and pasting ...
P-Karaoke: personalized karaoke system
MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

In this demonstration, a personalized Karaoke system, <i>P-Karaoke</i>, is proposed. In the P-Karaoke system, personal home videos and photographs, which are automatically selected from users' multimedia database according to their content, users' ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

November 2003

670 pages

ISBN:1581137222

DOI:10.1145/957013

General Chairs:
Lawrence Rowe
University of California, Berkeley
,
Harrick Vin
University of Texas, Austin
,
Program Chairs:
Thomas Plagemann
University of Oslo
,
Prashant Shenoy
University of Massachusetts, Amherst
,
John R. Smith
IBM T.J. Watson Research Center

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM03

Sponsor:

MM03: 2003 11th Annual ACM International Conference on Multimedia

November 2 - 8, 2003

CA, Berkeley, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
550
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin DCaba Heilbron FLee JWang OMartelaro N(2024)VideoMap: Supporting Video Exploration, Brainstorming, and Prototyping in the Latent SpaceProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656192(311-327)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3635636.3656192
Zhang LDenney BLu J(2016)Sub-event recognition and summarization for structured scenario photosMultimedia Tools and Applications10.1007/s11042-016-3346-x75:15(9295-9314)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11042-016-3346-x
Bano SCavallaro A(2016)ViCompMultimedia Tools and Applications10.1007/s11042-015-2641-275:12(7187-7210)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1007/s11042-015-2641-2
Schoeffmann KHudelist MHuber J(2015)Video Interaction ToolsACM Computing Surveys10.1145/280879648:1(1-34)Online publication date: 29-Sep-2015
https://dl.acm.org/doi/10.1145/2808796
Outtagarts ASquedin SMartinot O(2014)Cloud-Based Automatic Video Editing Using KeywordsE-Business and Telecommunications10.1007/978-3-662-44791-8_14(228-241)Online publication date: 12-Sep-2014
https://doi.org/10.1007/978-3-662-44791-8_14
Mitarai H(2012)Interaction Model for Emotive Video ProductionInternational Journal of Information and Electronics Engineering10.7763/IJIEE.2012.V2.183Online publication date: 2012
https://doi.org/10.7763/IJIEE.2012.V2.183
Mitarai HYoshitaka A(2012)EmocapInternational Journal of Multimedia Data Engineering & Management10.4018/jmdem.20120401043:2(58-75)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.4018/jmdem.2012040104
Xu QWu ZLi GQin LJiang SHuang Qdel Bimbo AChang SSmeulders A(2010)Memory matrixProceedings of the 18th ACM international conference on Multimedia10.1145/1873951.1874115(927-930)Online publication date: 25-Oct-2010
https://dl.acm.org/doi/10.1145/1873951.1874115
Xiang YKankanhalli Mdel Bimbo AChang SSmeulders A(2010)Automated aesthetic enhancement of videosProceedings of the 18th ACM international conference on Multimedia10.1145/1873951.1873991(281-290)Online publication date: 25-Oct-2010
https://dl.acm.org/doi/10.1145/1873951.1873991
O'Connor CFitzpatrick G(2010)Making video mundanePersonal and Ubiquitous Computing10.1007/s00779-009-0258-z14:3(197-208)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1007/s00779-009-0258-z
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents