article

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos

Authors:

Ali Reza Yamghani,

Farzad ZargariAuthors Info & Claims

Circuits, Systems, and Signal Processing, Volume 38, Issue 4

Pages 1695 - 1716

https://doi.org/10.1007/s00034-018-0932-3

Published: 01 April 2019 Publication History

Abstract

Video abstraction allows indexing, searching, browsing and evaluating a video only by accessing its useful contents. Several studies have been done in this field, but most of them are in pixel domain and require decoding process. It makes these methods more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.265/HEVC compressed domain, HVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes from an H.265/HEVC coded video. The frames' similarity is calculated by intersecting their I-frame prediction modes' histogram. The similarity measure detects and removes redundant key-frames to increase the quality of final video abstraction. Moreover, we employ fuzzy c-means clustering to categorize similar frames and extract key-frames as representatives of the entire video frames. The interpretation of the results shows that using the proposed method achieves on average 86% accuracy and 19% error rate in compressed domain video abstraction which is higher than the other tested methods in the pixel domain. Also, it has an acceptable robustness to coding parameters, and on average it generates video key-frames that are closer to human summaries.

References

[1]

J. Almeida, N.J. Leite, R.D.S. Torres, Online video summarization on compressed domain. J. Vis. Commun. Image Represent. 24(6), 729---738 (2013).

[2]

S.E.D. Avila, A.P.B. Lopes, A. Da Luz, A.D.A. Araújo, VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recognit. Lett. 32(1), 56---68 (2011).

Digital Library

[3]

S. Cha, Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1(4), 300---307 (2007).

[4]

T. Chheng, Video Summarization Using Clustering (Department of Computer Science University of California, Irvine, 2007)

[5]

S. De Bruyne, D. Van Deursen, J. De Cock, W. De Neve, P. Lambert, R. Van de Walle, A compressed-domain approach for shot boundary detection on H.264/AVC bit streams. Signal Process. Image Commun. 23(7), 473---489 (2008).

Digital Library

[6]

D. DeMenthon, V. Kobla, D. Doermann, Video summarization by curve simplification. In: Proceedings of the ACM International Conference on Multimedia, New York, USA (1998), pp. 211---218.

Digital Library

[7]

R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)

Digital Library

[8]

N. Ejaz, T.B. Tariq, S.W. Baik, Adaptive key frame extraction for video summarization using an aggregation mechanism. J. Vis. Commun. Image Represent. 23(7), 1031---1040 (2012).

Digital Library

[9]

M. Furini, F. Geraci, M. Montangero, M. Pellegrini, STIMO: STIll and MOving video storyboard for the web scenario. Multimed. Tools Appl. 46(1), 47---69 (2010).

Digital Library

[10]

H.265/HEVC Reference Software, https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/. Last seen on Feb 2017

[11]

A. Hanjalic, H. Zhang, An Integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. 9(8), 1280---1289 (1999).

Digital Library

[12]

J. He, F. Yang, Y. Zhou, High-speed implementation of rate-distortion optimised quantisation for H.265/HEVC. IET Image Process. 9(8), 652---661 (2015).

[13]

L. Herranz, J.M. Martínez, An efficient summarization algorithm based on clustering and bitstream extraction. In: Proceedings of International Conference on Multimedia and Expo (2009), pp. 654---657.

Digital Library

[14]

W. Hu, S. Member, N. Xie, L. Li, X. Zeng, S. Maybank, A survey on visual content-based video indexing and retrieval. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6), 797---819 (2011).

Digital Library

[15]

J. Kavitha, P.A.J. Rani, Static and multiresolution feature extraction for video summarization. Procedia Comput. Sci. 47(C), 292---300 (2015).

[16]

J. Li, T. Yao, Q. Ling, T. Mei, Detecting shot boundary with sparse coding for video summarization. Neurocomputing 266, 66---78 (2017).

Digital Library

[17]

Y. Li, T. Zhang, D. Tretter, An overview of video abstraction techniques an overview of video abstraction techniques. Imaging, 1---23 (2001). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.6173&rep=rep1&type=pdf

[18]

T. Liu, X. Zhang, J. Feng, K.T. Lo, Shot reconstruction degree: a novel criterion for key frame selection. Pattern Recognit. Lett. 25(12), 1451---1457 (2004).

[19]

A.G. Money, H. Agius, Video summarization: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121---143 (2008).

Digital Library

[20]

P. Mundur, Y. Rao, Y. Yesha, Key-frame based video summarization using Delaunay clustering. Int. J. Digit. Libr. 6(2), 219---232 (2006).

Digital Library

[21]

J.H. Oh, Q. Wen, S. Hwang, J. Lee, Video abstraction. In: Video Data Management and Information Retrieval (2005), pp. 321---346

[22]

F. Rahmani, F. Zargari, Compressed domain visual information retrieval based on I-frames in HEVC. Multimed. Tools Appl. (2016).

Digital Library

[23]

G.-H. Song, Q.G. Ji, Z.-M. Lu, Z.D. Fang, Z.H. Xie, A novel video abstraction method based on fast clustering of the regions of interest in keyframes. Int. J. Electron. Commun. (AEÜ) 68, 237---243 (2014).

[24]

X. Song, G. Fan, Joint key-frame extraction and object segmentation for content-based video analysis. IEEE Trans. Circuits Syst. Video Technol. 16(7), 904---914 (2006).

Digital Library

[25]

M. Srinivas, M.M.M. Pai, R.M. Pai, An improved algorithm for video summarization--a rank based approach. Procedia Comput. Sci. 89, 812---819 (2016).

[26]

G.J. Sullivan, J. Ohm, W. Han, T. Wiegand, Overview of the high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649---1668 (2012).

Digital Library

[27]

Z. Sun, K. Jia, H. Chen, Video keyframe extraction based on spatial-temporal color distribution. In: Proceedings--4th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP (2008), pp. 196---199.

Digital Library

[28]

B.T. Truong, S. Venkatesh, Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl. 3(1), 1---37 (2007).

Digital Library

[29]

J. Wu, S. Zhong, J. Jiang, Y. Yang, A novel clustering method for static video summarization. Multimed. Tools Appl. 76(7), 9625---9641 (2017).

Digital Library

[30]

L. Xiang-wei, Z. Li-dong, Z. Kai, Hierarchical video summarization extraction algorithm in compressed domain. Phys. Procedia 24, 2360---2366 (2012).

[31]

W. Yao, Z. Li, S. Rahardja, Dynamic threshold-based keyframe detection and its application in rate control. IET Image Process. 6(7), 986 (2012).

[32]

X.D. Zhang, T.Y. Liu, K.T. Lo, J. Feng, Dynamic selection and effective compression of keyframes for video abstraction. Pattern Recognit. Lett. 24(9---10), 1523---1532 (2003).

Digital Library

Cited By

Khan ARao YShao J(2022)ENet: event based highlight generation network for broadcast sports videosMultimedia Systems10.1007/s00530-022-00978-828:6(2453-2464)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00530-022-00978-8
Tariq JIjaz A(2020)HEVC Intra Mode Selection Using Benford’s LawCircuits, Systems, and Signal Processing10.1007/s00034-020-01482-y40:1(418-437)Online publication date: 25-Jun-2020
https://dl.acm.org/doi/10.1007/s00034-020-01482-y
Basavarajaiah MSharma P(2019)Survey of Compressed Domain Video Summarization TechniquesACM Computing Surveys10.1145/335539852:6(1-29)Online publication date: 16-Oct-2019
https://dl.acm.org/doi/10.1145/3355398

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

HEVC intra frame based compressed domain video summarization
AIIPCC '19: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud Computing

In order to overcome the shortcomings of the poor quality of the final generated video summary by reason of the incomplete selection of video feature points, a compressed domain video summarization generation algorithm is proposed based on HEVC intra-...
Compressed domain visual information retrieval based on I-frames in HEVC

Compressed domain retrieval is important in the retrieval of HEVC coded videos, because full decompression of HEVC coded videos is very time consuming. In this paper, a texture based retrieval method is proposed for either video retrieval of HEVC coded ...
Automatic Caption Localization in Compressed Video

We present a method to automatically localize captions in JPEG compressed images and the I-frames of MPEG compressed videos. Caption text regions are segmented from background images using their distinguishing texture characteristics. Unlike previously ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Circuits, Systems, and Signal Processing

Circuits, Systems, and Signal Processing Volume 38, Issue 4

April 2019

494 pages

ISSN:0278-081X

Issue’s Table of Contents

Copyright © Copyright © 2019 Springer Science+Business Media, LLC, part of Springer Nature.

Publisher

Birkhauser Boston Inc.

United States

Publication History

Published: 01 April 2019

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khan ARao YShao J(2022)ENet: event based highlight generation network for broadcast sports videosMultimedia Systems10.1007/s00530-022-00978-828:6(2453-2464)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00530-022-00978-8
Tariq JIjaz A(2020)HEVC Intra Mode Selection Using Benford’s LawCircuits, Systems, and Signal Processing10.1007/s00034-020-01482-y40:1(418-437)Online publication date: 25-Jun-2020
https://dl.acm.org/doi/10.1007/s00034-020-01482-y
Basavarajaiah MSharma P(2019)Survey of Compressed Domain Video Summarization TechniquesACM Computing Surveys10.1145/335539852:6(1-29)Online publication date: 16-Oct-2019
https://dl.acm.org/doi/10.1145/3355398

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents