research-article

Probability Model-Based Early Merge Mode Decision for Dependent Views Coding in 3D-HEVC

Authors:

Xiangling Ding,

Rongrong GongAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 14, Issue 4

Article No.: 85, Pages 1 - 15

https://doi.org/10.1145/3267128

Published: 01 October 2018 Publication History

Abstract

As a 3D extension to the High Efficiency Video Coding (HEVC) standard, 3D-HEVC was developed to improve the coding efficiency of multiview videos. It inherits the prediction modes from HEVC, yet both Motion Estimation (ME) and Disparity Estimation (DE) are required for dependent views coding. This improves coding efficiency at the cost of huge computational costs. In this article, an early Merge mode decision approach is proposed for dependent texture views and dependent depth maps coding in 3D-HEVC based on priori and posterior probability models. First, the priori probability model is established by exploiting the hierarchical and interview correlations from those previously encoded blocks. Second, the posterior probability model is built by using the Coded Block Flag (CBF) of the current coding block. Finally, the joint priori and posterior probability model is adopted to early terminate the Merge mode decision for both dependent texture views and dependent depth maps coding. Experimental results show that the proposed approach saves 45.2% and 30.6% encoding time on average for dependent texture views and dependent depth maps coding while maintaining negligible loss of coding efficiency, respectively.

References

[1]

G. Bjontegaard. 2001. Calculation of average PSNR differences between RD curves. no. ITU-T SC16/Q6, VCEG-M33, Austin, USA (April 2001).

[2]

H. Chen, C. H. Fu, Y. Zhang, Y. L. Chan, and W. C. Siu. 2017. Early merge mode decision for depth maps in 3D-HEVC. In Proceedings of the 22nd International Conference on Digital Signal Processing (DSP) (Aug. 2017), 1--5.

[3]

X. Ding, Y. Li, M. Xia, J. He, and G. Yang. 2018. Detection of motion compensated frame interpolation via motion-aligned temporal difference. Multimedia Tools and Applications (Aug. 2018).

[4]

Q. Hu, X. Zhang, Z. Shi, and Z. Gao. 2016. Neyman-pearson-based early mode decision for HEVC encoding. IEEE Transactions on Multimedia 18, 3 (March 2016), 379--391.

Digital Library

[5]

S. Jung and H. W. Park. 2016. A fast mode decision method in HEVC using adaptive ordering of modes. IEEE Transactions on Circuits and Systems for Video Technology 26, 10 (Oct. 2016), 1846--1858.

Digital Library

[6]

L. Lei, J. Duan, F. Wu, N. Ling, and C. Hou. 2018. Fast mode decision based on grayscale similarity and inter-view correlation for depth map coding in 3D-HEVC. IEEE Transactions on Circuits and Systems for Video Technology 28, 3 (March 2018), 706--718.

[7]

Y. Li, G. Yang, N. Chen, Y. Zhu, and X. Ding. 2016. Early DIRECT mode decision for MVC using MB mode homogeneity and RD cost correlation. IEEE Transactions on Broadcasting 62, 3 (May 2016), 700--708.

[8]

Y. Li, G. Yang, Y. Zhu, X. Ding, and X. Sun. 2017. Adaptive inter CU depth decision for HEVC using optimal selection model and encoding parameters. IEEE Transactions on Broadcasting 63, 3 (Sept. 2017), 535--546.

[9]

Y. Li, G. Yang, Y. Zhu, X. Ding, and X. Sun. 2017. Unimodal stopping model-based early SKIP mode decision for high efficiency video coding. IEEE Transactions on Multimedia 19, 7 (July 2017), 1431--1441.

Digital Library

[10]

Y. Li, G. Yang, Y. Zhu, C. Liu, and K. Liu. 2016. Adaptive mode decision for multiview video coding based on macroblock position constraint model. Journal of Real-Time Image Processing 12, 3 (Oct. 2016), 575--582.

Digital Library

[11]

K. Müller and A. Vetro. 2011. Common test conditions of 3DV core experiments. document JCT3V-G1100, San Jose, CA, USA (2011).

[12]

Z. Pan, J. Lei, Y. Zhang, and F. Wang. 2018. Adaptive fractional-pixel motion estimation skipped algorithm for efficient HEVC motion estimation. ACM Transactionson Multimedia Computing and Communications Applications 14, 1 (Jan. 2018), Article 12.

Digital Library

[13]

Z. Pan, S. Kwong, M.-T. Sun, and J. Lei. 2014. Early merge mode decision based on motion estimation and hierarchical depth correlation for HEVC. IEEE Transactions on Broadcasting 60, 2 (June 2014), 405--412.

[14]

Z. Pan, Y. Zhang, and S. Kwong. 2015. Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Transactions on Broadcasting 61, 2 (June 2015), 166--176.

[15]

Z. Pan, Y. Zhang, J. Lei, L. Xu, and X. Sun. 2016. Early DIRECT mode decision based on all-zero block and rate distortion cost for multiview video coding. IET Image Processing 10, 1 (Jan. 2016), 9--15.

[16]

L. Shen, P. An, Z. Zhang, Q. Hu, and Z. Chen. 2015. A 3D-HEVC fast mode decision algorithm for real-time applications. ACM Transactions on Multimedia Computing and Communications Applications 11, 3 (Jan. 2015), Article 34.

Digital Library

[17]

L. Shen, Z. Liu, R. Ma, P. An, and Z. Zhang. 2011. Low-complexity mode decision for MVC. IEEE Transactions on Circuits and Systems for Video Technology 21, 6 (June 2011), 837--843.

Digital Library

[18]

L. Shen, Z. Liu, Z. Zhang, S. Liu, and P. An. 2009. Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Transactions on Broadcasting 55, 4 (Dec. 2009), 761--766.

[19]

L. Shen, Z. Liu, T. Yan, Z. Zhang, and P. An. 2010. Early SKIP mode decision for MVC using inter-view correlation. Signal Processing: Image Communication 25, 2 (Feb. 2010), 88--93.

Digital Library

[20]

L. Shen, Z. Zhang, and Z. Liu. 2014. Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Transactions on Circuits and Systems for Video Technology 24, 10 (Oct. 2014), 1709--1722.

[21]

G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (Dec. 2012), 1649--1668.

Digital Library

[22]

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo. 2011. Free-viewpoint TV. IEEE Signal Processing Magazine 28, 1 (Jan. 2011), 67--76.

[23]

J. Tariq, S. Kwong, and H. Yuan. 2017. Spatial/temporal motion consistency based MERGE mode early decision for HEVC. Journal of Visual Communication and Image Representation 44 (April 2017), 198--213.

Digital Library

[24]

G. Tech, Y. Chen, K. Müller, J.-R. Ohm, A. Vetro, and Y. Wang. 2016. Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology 26, 1 (Jan. 2016), 35--49.

Digital Library

[25]

A. Vetro, A. M. Tourapis, K. Muller, and T. Chen. 2011. 3D-TV content storage and transmission. IEEE Transactions on Broadcasting 52, 7 (June 2011), 384--394.

[26]

A. Vetro, T. Wiegand, and G. J. Sullivan. 2011. Overview of the stereo and multi-view video coding extensions of the H. 264/MPEG-4 AVC standard. Proceedings of the IEEE 99, 4 (April 2011), 626--642.

[27]

F. Wang, H. Zeng, Q. Shen, and S. Du. 2013. Efficient early direct mode decision for multi-view video coding. Signal Processing: Image Communication 28, 7 (Aug. 2013), 736--744.

Digital Library

[28]

T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. 2003. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (Aug. 2003), 560--576.

Digital Library

[29]

Y. X. Song and K. B. Jia. 2015. Early merge mode decision for texture coding in 3D-HEVC. Journal of Visual Communication and Image Representation 33 (Nov. 2015), 60--68.

Digital Library

[30]

M. Xia, G. Yang, L. Li, R. Li, and X. Sun. 2017. Detecting video frame rate up-conversion based on frame-level analysis of average texture variation. Multimedia Tools and Applications 76, 6 (March 2017), 8399--8421.

Digital Library

[31]

J. Yang, J. Kim, K. Won, H. Lee, and B. Jeon. 2011. Early SKIP detection for HEVC. document JCTVC-G543, JCT-VC, Geneva, Switzerland (2011).

[32]

H. Zeng, X. Wang, J. Chen, C. Cai, and Y. Zhang. 2014. Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Transactions on Circuits and Systems for Video Technology 24, 9 (March 2014), 1566--1578.

[33]

D. Zhang, T. Yinand, G. Yang, M. Xia, L. Li, and X. Sun. 2017. Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. Journal of Visual Communication and Image Representation 48 (Aug. 2017), 281--291.

Digital Library

[34]

J. Zhang, B. Li, and H. Li. 2016. An efficient fast mode decision method for inter prediction in HEVC. IEEE Transactions on Circuits and Systems for Video Technology 28, 6 (Aug. 2016), 1502--1515.

[35]

N. Zhang, D. Zhao, Y.-W. Chen, J.-L. Lin, and W. Gao. 2014. Fast encoder decision for texture coding in 3D-HEVC. Signal Processing: Image Communication 29, 9 (Oct. 2014), 951--961.

Digital Library

[36]

Q. Zhang, K. Huang, X. Wang, B. Jiang, and Y. Gan. 2017. Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock. Journal of Real-Time Image Processing (May 2017), 1--18.

[37]

Q. Zhang, Q. Wu, X. Wang, and Y. Gan. 2014. Early SKIP mode decision for three-dimensional high efficiency video coding using spatial and interview correlations. Journal of Electronic Imaging 23, 5 (Oct. 2014), 053017--053024.

[38]

Q. Zhang, N. Zhang, T. Wei, X. Qian, K. Huang, and Y. Gan. 2017. Fast depth map mode decision based on depth-texture correlation and edge classification for 3D-HEVC. Journal of Visual Communication and Image Representation 45 (May 2017), 170--180.

Digital Library

[39]

Y. Zhang, S. Kwong, G. Jiang, X. Wang, and M. Yu. 2012. Statistical early termination model for fast mode decision and reference frame selection in multiview video coding. IEEE Transactions on Broadcasting 58, 1 (Dec. 2012), 10--23.

[40]

Y. Zhang, S. Kwong, L. Xu, and G. Jiang. 2013. DIRECT mode early decision optimization based on rate distortion cost property and inter-view correlation. IEEE Transactions on Broadcasting 59, 2 (April 2013), 390--398.

[41]

T. Zhao, S. Kwong, H. Wang, Z. Wang, Z. Pan, and C.-C. J. Kuo. 2013. Multiview coding mode decision with hybrid optimal stopping model. IEEE Transactions on Image Processing 22, 4 (Dec. 2013), 1598--1609.

Digital Library

[42]

W. Zhao, T. Onoye, and T. Song. 2015. Hierarchical structure-based fast mode decision for H.265/HEVC. IEEE Transactions on Circuits and Systems for Video Technology 25, 10 (Oct. 2015), 1651--1664.

Digital Library

Cited By

Li HLiao BLi JLi S(2024)A Survey on Biomimetic and Intelligent Algorithms with ApplicationsBiomimetics10.3390/biomimetics90804539:8(453)Online publication date: 24-Jul-2024
https://doi.org/10.3390/biomimetics9080453
Zeng YLiao BLi ZHua CLi S(2024)A Comprehensive Review of Recent Advances on Intelligence Algorithms and Information Engineering ApplicationsIEEE Access10.1109/ACCESS.2024.346175612(135886-135912)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3461756
Wang JChen RLv Z(2023)DNA Computing-Based Multi-Source Data Storage Model in Digital TwinsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356182319:3s(1-16)Online publication date: 24-Feb-2023
https://dl.acm.org/doi/10.1145/3561823
Show More Cited By

Index Terms

Probability Model-Based Early Merge Mode Decision for Dependent Views Coding in 3D-HEVC
1. Computing methodologies
  1. Computer graphics
    1. Image compression

Recommendations

Early merge mode decision for texture coding in 3D-HEVC

An early merge mode decision method for texture coding in 3D-HEVC is proposed.Inter-view correlation of coding modes is studied for B frame.Hierarchical depth correlation of coding modes is studied for P frame.Two combinations of conditions to early ...
A 3D-HEVC Fast Mode Decision Algorithm for Real-Time Applications

3D High Efficiency Video Coding (3D-HEVC) is an extension of the HEVC standard for coding of multiview videos and depth maps. It inherits the same quadtree coding structure as HEVC for both components, which allows recursively splitting into four equal-...
Fast intra mode decision for depth coding in 3D-HEVC

The emergent 3D High Efficiency Video Coding (3D-HEVC) is an extension of the High Efficiency Video Coding (HEVC) standard for the compression of the multi-view texture videos plus depth maps format. Since depth maps have different statistical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 14, Issue 4

Special Section on Deep Learning for Intelligent Multimedia Analytics

November 2018

221 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3282485

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2018

Accepted: 01 August 2018

Revised: 01 May 2018

Received: 01 March 2018

Published in TOMM Volume 14, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
National Key R8D Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
180
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li HLiao BLi JLi S(2024)A Survey on Biomimetic and Intelligent Algorithms with ApplicationsBiomimetics10.3390/biomimetics90804539:8(453)Online publication date: 24-Jul-2024
https://doi.org/10.3390/biomimetics9080453
Zeng YLiao BLi ZHua CLi S(2024)A Comprehensive Review of Recent Advances on Intelligence Algorithms and Information Engineering ApplicationsIEEE Access10.1109/ACCESS.2024.346175612(135886-135912)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3461756
Wang JChen RLv Z(2023)DNA Computing-Based Multi-Source Data Storage Model in Digital TwinsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356182319:3s(1-16)Online publication date: 24-Feb-2023
https://dl.acm.org/doi/10.1145/3561823
Zheng NSong XSu TLiu WYan YNie L(2023)Egocentric Early Action Prediction via Adversarial Knowledge DistillationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/354449319:2(1-21)Online publication date: 6-Feb-2023
https://dl.acm.org/doi/10.1145/3544493
Feng CHan PZhang XZhang QZong YLiu YGuo L(2023)Cost-Minimized Computation Offloading of Online Multifunction Services in Collaborative Edge-Cloud NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2022.320104820:1(292-304)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TNSM.2022.3201048
de Moura Rde Lima JCarro L(2022)Data and Computation Reuse in CNNs Using Memristor TCAMsACM Transactions on Reconfigurable Technology and Systems10.1145/354953616:1(1-24)Online publication date: 22-Dec-2022
https://dl.acm.org/doi/10.1145/3549536
Hussein HDino HMstafa RHassan M(2022)Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithmMultimedia Tools and Applications10.1007/s11042-022-12438-681:8(11563-11586)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s11042-022-12438-6
Guo J(2022)An HEVC-compliant perceptual video coding using just noticeable differenceMultimedia Tools and Applications10.1007/s11042-021-11535-281:1(1257-1286)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s11042-021-11535-2
Yuan KLi SCheng WKankanhalli MWang MChu WLiu JWorring M(2021)2.5D Pose Guided Human Image GenerationProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463580(501-505)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463580
Lin JChen MCiou YYeh CLin MKau LChang C(2021)Fast Texture Coding Based on Spatial, Temporal and Inter-View Correlations for 3D Video CodingIEEE Access10.1109/ACCESS.2021.30939509(100081-100095)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3093950
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents