research-article

QoE-oriented 3D video transcoding for mobile streaming

Authors:

Jinxia LiuAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 8, Issue 3s

Article No.: 42, Pages 1 - 20

https://doi.org/10.1145/2348816.2348821

Published: 16 October 2012 Publication History

Abstract

With advance in mobile 3D display, mobile 3D video is already enabled by the wireless multimedia networking, and it will be gradually popular since it can make people enjoy the natural 3D experience anywhere and anytime. In current stage, mobile 3D video is generally delivered over the heterogeneous network combined by wired and wireless channels. How to guarantee the optimal 3D visual quality of experience (QoE) for the mobile 3D video streaming is one of the important topics concerned by the service provider. In this article, we propose a QoE-oriented transcoding approach to enhance the quality of mobile 3D video service. By learning the pre-controlled QoE patterns of 3D contents, the proposed 3D visual QoE inferring model can be utilized to regulate the transcoding configurations in real-time according to the feedbacks of network and user-end device information. In the learning stage, we propose a piecewise linear mean opinion score (MOS) interpolation method to further reduce the cumbersome manual work of preparing QoE patterns. Experimental results show that the proposed transcoding approach can provide the adapted 3D stream to the heterogeneous network, and further provide superior QoE performance to the fixed quantization parameter (QP) transcoding and mean squared error (MSE) optimized transcoding for mobile 3D video streaming.

References

[1]

Ameigeiras, P., Ramos-Munoz, J. J., Navarro-Ortiz, J., and Mogensen, P. 2010. QoE oriented cross-layer design of a resource allocation algorithm in beyond 3G systems. Elsevier Computer Commun. 33, 5, 571--582.

Digital Library

[2]

Brust, H., Smolic, A., Müller, K., Tech, G., and Wiegand, T. 2009. Mixed resolution coding of stereoscopic video for mobile devices. In Proceedings of 3DTV-CON. 1--4.

[3]

Chen, Z., Lin, W., and Ngan, K. N. 2010a. Perceptual video coding: Challenges and approaches. In Proceedings of IEEE International Conference on Multimedia & Expo. 19--23.

[4]

Chen, W., Fournier, J., Barkowsky, M., and Le Callet, P. 2010b. New requirements of subjective video quality assessment methodologies for 3DTV. In Proceedings of VPQM.

[5]

Domański, M., Grajek, T., Klimaszewski, K., Kurc, M., Stankiewicz, O., Stankowski, J., and Wegner, K. 2009. Poznan multiview video test sequences and camera parameters. ISO/IEC JTC1/SC29/WG11, MPEG 2009/M17050.

[6]

Girod, B. 1993. What's wrong with mean-squared error. In Digital Images and Human Vision. The MIT Press, 207--220.

Digital Library

[7]

Ho, Y.-S., Lee, E.-K., and Lee, C. 2008. Multiview video test sequence and camera parameters. ISO/IEC JTC1/SC29/WG11, MPEG2008/m15419.

[8]

ITU-R. 2002. Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation BT.500-11.

[9]

ITU-T. 2007. New appendix I -Definition of quality of experience (QoE). Recommentation G.100/P.10 Amendment 1.

[10]

Jumisko-Pyykkö, S., Haustola, T., Boev, A., and Gotchev, A. 2011. Subjective evaluation of mobile 3D video content: Depth range versus compression artifacts. Proc. SPIE, vol. 7881, 78810C.

[11]

Khan, A., Sun, L., and Ifeachor, E. 2012. QoE Prediction Model and its Application in Video Quality Adaptation over UMTS Networks. IEEE Trans. Multimedia 14, 2, 431--442.

Digital Library

[12]

Liang, Y. J., Apostolopoulos, J. G., and Girod, B. 2008. Analysis of packet loss for compressed video: effect of burst losses and correlation between error frames. IEEE Trans. Circ. Syst. Video Tech. 18, 7, 861--874.

Digital Library

[13]

Liu, S. and Chen, C. W. 2010. 3D video transcoding for virtual views. In Proceedings of ACM Multimedia. 795--798.

Digital Library

[14]

Liu, Y., Huang, Q., Ma, S., Zhao, D., and Gao, W. 2009. Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process. Image Commun. 24, 8, 666--681.

Digital Library

[15]

Liu, Y., Peng, G., Hu, Y., Ci, S., and Tang, H. 2010. A multi-pass VBR rate control method for video plus depth based mobile 3D video coding. In Proceedings of 11th Pacific-Rim Conference on Multimedia (PCM'10).

Digital Library

[16]

Liu, Y., Ci, S., Tang, H., and Ye, Y. 2012. Application-Adapted Mobile 3D Video Coding and Streaming—A Survey. 3D Res. J. 3, 01(2012)5, Springer.

Digital Library

[17]

Mohamed, S. and Rubino, G. 2002. A study of real-time packet video quality using random neural networks. IEEE Trans. Circ. Syst. Video Tech. 12, 12, 1071--1083.

Digital Library

[18]

Ma, S., Gao, W., and Lu, Y. 2005. Rate-distortion analysis for H.264/AVC video coding and its application to rate control. IEEE Trans. Circ. Syst. Video Tech. 15, 12, 1533--1544.

Digital Library

[19]

Mendiburu, B. 2008. 3D Movie Making-Stereoscopic Digital Cinema from Script to Screen. Elsevier, New York, 2008978-0-240-81137-6.

[20]

Merkle, P., Wang, Y., Müller, K., Smolic, A., and Wiegand, T. 2009. Video plus depth compression for mobile 3D services. In Proceedings of IEEE 3DTV Conference. Potsdam, Germany.

[21]

Piamrat, K., Ksentini, A., Bonnin, J.-M., and Viho, C. 2009. Q-DRAM: QoE based dynamic rate adaptation mechanism for multicast in wireless networks. In Proceedings of IEEE GLOBECOM.

Digital Library

[22]

Perkins, M. G. 1992. Data compression of stereopairs. IEEE Trans. Comm. 40, 4, 684--696.

[23]

Piamrat, K., Ksentini, A., Viho, C., and Bonnin, J.-M. 2008. QoE-aware admission control for multimedia applications in IEEE 802.11 Wireless Networks. In Proceedings of IEEE VTC.

[24]

Rückert, J., Abboud, O., Zinner, T., Steinmetz, R., and Hausheer, D. 2012. Quality adaptation in P2P video streaming based on objective QoE metrics. IFIP Networking.

Digital Library

[25]

Seuntïens, P. J. H. 2006. Visual experience of 3DTV. Ph.D. Thesis, Technische Universiteit Eindhoven.

[26]

Smolic, A., Kauff, P., Knorr, S., Hornung, A., Kunter, M., Müller, M., and Lang, M. 2011. Three-dimensional video postproduction and processing. Proc. IEEE 99, 4, 607--625.

[27]

Thakolsri, S., Kellerer, W., and Steinbach, E. 2010. QoE-based rate adaptation scheme selection for resource-constrained wireless video transmission. In Proceedings of ACM Multimedia. 783--786.

Digital Library

[28]

Tanimoto Lab at Nagoya University. 2008. http://www.tanimoto.nuee.nagoya-u.ac.jp/.

[29]

Vetro, A., Xin, J., and Sun, H. 2005. Error resilience video transcoding for wireless communications. IEEE Wirel. Comm. 12, 4, 14--21.

Digital Library

[30]

Vetro, A., Tourapis, A. M., Müller K., and Chen, T. 2011. 3D-TV Content Storage and Transmission. IEEE Trans. Broadcast. 57, 2, part 2, 348--394.

[31]

Wang, Z., Bovik, A. C., and Lu, L. 2002. Why is image quality assessment so difficult&quest;. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing.

[32]

Worrall, S. T., Kondoz, A. M., Driesnack, D., Tekalp, M., Kovacs, P., Adari, T., and Gokmen, H. 2010. DIOMEDES: Content Aware Delivery of 3D Media Using P2P and DVB-T2. 2010 NEM Summit: Towards Future Media Internet, Barcelona, Spain.

[33]

Yin, P., Vetro, A., Xia, M., and Liu, B. 2003. Rate-distortion models for video transcoding. In Proceedings of the Conference on Image and Video Communications and Processing. 479--488.

[34]

Zou, W. 2009. An overview for developing end-to-end standards for 3-D TV in the home. Inf. Display, 25, 7, 14--19.

Cited By

Kougioumtzidis GPoulkov VZaharis ZLazaridis P(2022)A Survey on Multimedia Services QoE Assessment and Machine Learning-Based PredictionIEEE Access10.1109/ACCESS.2022.314959210(19507-19538)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3149592
Wang LWang X(2020)A Novel Hybrid Optimization Algorithm for Scalable Video Coding in an SDNMathematical Problems in Engineering10.1155/2020/36946422020(1-8)Online publication date: 5-Oct-2020
https://doi.org/10.1155/2020/3694642
Zhang XYang CWang HXu WKuo C(2020)Satisfied-User-Ratio Modeling for Compressed VideoIEEE Transactions on Image Processing10.1109/TIP.2020.296599429(3777-3789)Online publication date: 2020
https://doi.org/10.1109/TIP.2020.2965994
Show More Cited By

Index Terms

QoE-oriented 3D video transcoding for mobile streaming
1. Computing methodologies
  1. Computer graphics
    1. Image compression

Recommendations

Multiview video transcoding: from multiple views to single view
PCS'09: Proceedings of the 27th conference on Picture Coding Symposium

As multiview video is gaining more and more attentions, Multiview Video Coding (MVC) standard has been under development by the Joint Video Team as an extension to H.264/AVC. There will be increasingly more multiview video sources for both high end and ...
Leveraging the quantization offset for improved requantization transcoding of H.264/AVC video
PCS'09: Proceedings of the 27th conference on Picture Coding Symposium

Requantization transcoding is a method for reducing the bit rate of compressed video bitstreams. Most research on requantization is concerned with the architectural design, the selection of a suitable quantizer, or the reduction of requantization ...
Video encoding and transcoding using machine learning
MDM '08: Proceedings of the 9th International Workshop on Multimedia Data Mining: held in conjunction with the ACM SIGKDD 2008

Machine learning has been widely used in video analysis and search applications. In this paper, we describe a non-traditional use of machine learning in video processing - video encoding and transcoding. Video encoding and transcoding are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 8, Issue 3s

Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimedia

September 2012

173 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2348816

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2012

Accepted: 01 May 2012

Revised: 01 April 2012

Received: 01 January 2012

Published in TOMM Volume 8, Issue 3s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Important National Science & Technology Specific Project
National Natural Science Foundation of China
Chinese Academy of Sciences

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
441
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kougioumtzidis GPoulkov VZaharis ZLazaridis P(2022)A Survey on Multimedia Services QoE Assessment and Machine Learning-Based PredictionIEEE Access10.1109/ACCESS.2022.314959210(19507-19538)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3149592
Wang LWang X(2020)A Novel Hybrid Optimization Algorithm for Scalable Video Coding in an SDNMathematical Problems in Engineering10.1155/2020/36946422020(1-8)Online publication date: 5-Oct-2020
https://doi.org/10.1155/2020/3694642
Zhang XYang CWang HXu WKuo C(2020)Satisfied-User-Ratio Modeling for Compressed VideoIEEE Transactions on Image Processing10.1109/TIP.2020.296599429(3777-3789)Online publication date: 2020
https://doi.org/10.1109/TIP.2020.2965994
Duc TMinh CXuan TKamioka E(2020)Convolutional Neural Networks for Continuous QoE Prediction in Video Streaming ServicesIEEE Access10.1109/ACCESS.2020.30041258(116268-116278)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3004125
Liu YLiu JArgyriou ACi S(2018)3DQoE-Oriented and Energy-Efficient 2D plus Depth Based 3D Video Streaming Over Centrally Controlled NetworksIEEE Transactions on Multimedia10.1109/TMM.2018.280622120:9(2439-2453)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1109/TMM.2018.2806221
Doumanoglou AGriffin DSerrano JZioulis NPhan TJimenez DZarpalas DAlvarez FRio MDaras P(2018)Quality of Experience for 3-D Immersive Media StreamingIEEE Transactions on Broadcasting10.1109/TBC.2018.282390964:2(379-391)Online publication date: Jul-2018
https://doi.org/10.1109/TBC.2018.2823909
Zhao TLiu QChen C(2017)QoE in Video Transmission: A User Experience-Driven StrategyIEEE Communications Surveys & Tutorials10.1109/COMST.2016.261998219:1(285-302)Online publication date: Oct-2018
https://doi.org/10.1109/COMST.2016.2619982
Zarb TDebono C(2016)Broadcasting Free-Viewpoint Television Over Long-Term Evolution NetworksIEEE Systems Journal10.1109/JSYST.2015.245348310:2(773-784)Online publication date: Jul-2016
https://doi.org/10.1109/JSYST.2015.2453483
Zhu LZhang YLi NJiang GKwong S(2016)Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarityJournal of Visual Communication and Image Representation10.1016/j.jvcir.2016.04.02038:C(824-837)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.jvcir.2016.04.020
Nur Yilmaz G(2016)A bit rate adaptation model for 3D videoMultidimensional Systems and Signal Processing10.1007/s11045-014-0299-y27:1(201-215)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s11045-014-0299-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents