research-article

A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services

Authors:

Lea Skorin-Kapov,

Martín Varela,

Tobias Hoßfeld,

Kuan-Ta ChenAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 14, Issue 2s

Article No.: 29, Pages 1 - 29

https://doi.org/10.1145/3176648

Published: 22 May 2018 Publication History

Abstract

Quality of Experience (QoE) has received much attention over the past years and has become a prominent issue for delivering services and applications. A significant amount of research has been devoted to understanding, measuring, and modelling QoE for a variety of media services. The next logical step is to actively exploit that accumulated knowledge to improve and manage the quality of multimedia services, while at the same time ensuring efficient and cost-effective network operations. Moreover, with many different players involved in the end-to-end service delivery chain, identifying the root causes of QoE impairments and finding effective solutions for meeting the end users’ requirements and expectations in terms of service quality is a challenging and complex problem. In this article, we survey state-of-the-art findings and present emerging concepts and challenges related to managing QoE for networked multimedia services. Going beyond a number of previously published survey articles addressing the topic of QoE management, we address QoE management in the context of ongoing developments, such as the move to softwarized networks, the exploitation of big data analytics and machine learning, and the steady rise of new and immersive services (e.g., augmented and virtual reality). We address the implications of such paradigm shifts in terms of new approaches in QoE modeling and the need for novel QoE monitoring and management infrastructures.

Supplementary Material

a29-skorin-kapov-apndx.pdf (skorin-kapov.zip)

Supplemental movie, appendix, image and software files for, A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services

Download
135.12 KB

References

[1]

3GPP. 2017. Study on improved streaming Quality of Experience (QoE) reporting in 3GPP services and networks, 3GPP TR 26.909, Release 14, v14.0.0.

[2]

V. Aggarwal, E. Halepovic, J. Pang, S. Venkataraman, and H. Yan. 2014. Prometheus: Toward quality of experience estimation for mobile apps from passive network measurements. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications. ACM, 18.

Digital Library

[3]

A. Ahmad, L. Atzori, and M. G. Martini. 2017. Qualia: A multilayer solution for QoE passive monitoring at the user terminal. In Proceedings of the IEEE International Conference on Communications (ICC’17).

[4]

A. Ahmad, A. Floris, and L. Atzori. 2016. QoE-centric service delivery: A collaborative approach among OTTs and ISPs. Comput. Networks 110 (2016), 168--179.

Digital Library

[5]

A. Aijaz, M. Dohler, A. H. Aghvami, V. Friderikos, and M. Frodigh. 2017. Realizing the tactile internet: Haptic communications over next generation 5G cellular networks. IEEE Wireless Commun. 24, 2 (2017), 82--89.

Digital Library

[6]

M. Alreshoodi and J. Woods. 2013. Survey on QoE QoS correlation models for multimedia services. arXiv preprint arXiv:1306.0221 (2013).

[7]

S. Aroussi and A. Mellouk. 2014. Survey on machine-learning-based QoE-QoS correlation models. In Proceedinsg of the International Conference on Computing, Management and Telecommunications (ComManTel’14). IEEE, 200--204.

[8]

AT8T. 2016. ECOMP (Enhanced Control, Orchestration, Management 8 Policy) Architecture White Paper.

[9]

O. Awobuluyi, J. Nightingale, Q. Wang, and J. M. Alcaraz-Calero. 2015. Video quality in 5G networks: Context-aware QoE management in the SDN control plane. In Proceedings of the 2015 International Conference on CIT/IUCC/DASC/PICOM. IEEE, 1657--1662.

[10]

A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. 2013. Developing a predictive model of quality of experience for internet video. In ACM SIGCOMM Computer Communication Review, Vol. 43. ACM, 339--350.

Digital Library

[11]

Y. Bao, H. Wu, and X. Liu. 2017. From prediction to action: Improving user experience with data-driven resource allocation. IEEE J. Select. Areas Commun. 35, 5 (May 2017), 1062--1075.

Digital Library

[12]

A. A. Barakabitze, L. Sun, I.-H. Mkwawa, and E. Ifeachor. 2016. A novel QoE-aware SDN-enabled, NFV-based management architecture for future multimedia applications on 5G systems. In Proceedings of the 8th International Conference on Quality of Multimedia Experience (QoMEX’16). Lisbon, Portugal.

[13]

S. Baraković and L. Skorin-Kapov. 2013. Survey and challenges of QoE management issues in wireless networks. J. Comput. Networks Commun. 2013 (2013).

[14]

A. Benoit, P. Le Callet, P. Campisi, and R. Cousseau. 2009. Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 1 (2009), 659024.

[15]

A. Bentaleb, A. C. Begen, and R. Zimmermann. 2016. SDNDASH: Improving QoE of HTTP adaptive streaming using software defined networking. In Proceedings of the ACM Conference on Multimedia. ACM, 1296--1305.

Digital Library

[16]

N. Bouten, S. Latré, J. Famaey, W. Van Leekwijck, and F. De Turck. 2014. In-network quality optimization for adaptive video streaming services. IEEE Trans. Multimedia 16, 8 (Dec. 2014), 2281--2293.

[17]

A. Breznick. Feb. 2017. How to Deliver Higher Mobile Video QoE, a Heavy Reading white paper produced for Akamai Technologies Inc. (Feb. 2017).

[18]

F. E. Bustamante, D. Clark, and N. Feamster. 2017. Workshop on tracking quality of experience in the internet: Summary and outcomes. ACM SIGCOMM Comput. Commun. Rev. 47, 1 (2017), 55--60.

Digital Library

[19]

P. Casas, B. Gardlo, R. Schatz, and M. Mellia. 2016. An educated guess on QoE in operational networks through large-scale measurements. In Proceedings of the 2016 Workshop on QoE-based Analysis and Management of Data Communication Networks. ACM, 1--6.

[20]

P. Casas and R. Schatz. 2014. Quality of experience in cloud services: Survey and measurements. Computer Networks 68 (2014), 149--165.

[21]

P. Casas, M. Seufert, F. Wamser, B. Gardlo, A. Sackl, and R. Schatz. 2016. Next to you: Monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Network Service Manage. 13, 2 (2016), 181--196.

[22]

C. Cavaro-Ménard, L. Zhang-Ge, and P. Le Callet. 2013. QoE for telemedicine: Challenges and trends. In SPIE Optical Engineering+ Applications. International Society for Optics and Photonics, 88561A--88561A.

[23]

K.-T. Chen, C.-J. Chang, C.-C. Wu, Y.-C. Chang, and C.-L. Lei. 2010. Quadrant of Euphoria: A crowdsourcing platform for QoE assessment. IEEE Network 24, 2 (2010).

Digital Library

[24]

Q. A. Chen and others. 2014. QoE doctor: Diagnosing mobile app QoE with automated UI control and cross-layer analysis. In Proceedings of the 2014 Conference on Internet Measurement Conference. ACM, 151--164.

Digital Library

[25]

W. Chen, F. Jérôme, M. Barkowsky, and P. Le Callet. 2012. Exploration of quality of experience of stereoscopic images: Binocular depth. In Proceedings of the 6th International Workshop on Video Processing and Quality Metrics for Consumer Electronics. 1--6.

[26]

S. Chikkerur, V. Sundaram, M. Reisslein, and L. J. Karam. 2011. Objective video quality assessment methods: A classification, review, and performance comparison. IEEE Trans. Broadcast. 57, 2 (2011), 165--182.

[27]

B.-H. Chu, M.-S. Tsai, and C.-S. Ho. 2007. Toward a hybrid data mining model for customer retention. Knowl.-Based Syst. 20, 8 (2007), 703--718.

Digital Library

[28]

L. A. Clark and D. Watson. 1995. Constructing validity: Basic issues in objective scale development. Psychol. Assess. 7, 3 (1995), 309.

[29]

L. Cui, F. R. Yu, and Q. Yan. 2016. When big data meets software-defined networking: SDN for big data and big data for SDN. IEEE Network 30, 1 (2016), 58--65.

Digital Library

[30]

G. Dimopoulos, I. Leontiadis, P. Barlet-Ros, and K. Papagiannaki. 2016. Measuring video QoE from encrypted traffic. In Proceedings of the 2016 ACM on Internet Measurement Conference. ACM, 513--526.

Digital Library

[31]

L. Dinh-Xuan, M. Seufert, F. Wamser, and P. Tran-Gia. 2017. Study on the accuracy of QoE monitoring for HTTP adaptive video streaming using VNF. In Proceedings of the 1st IFIP/IEEE International Workshop on QoE Management. IEEE, 1--6.

[32]

F. Dobrian et al. 2011. Understanding the impact of video quality on user engagement. In ACM SIGCOMM Computer Communication Review, Vol. 41. ACM, 362--373.

Digital Library

[33]

O. Dobrijevic, M. Santl, and M. Matijasevic. 2015. Ant colony optimization for QoE-centric flow routing in software-defined networks. In Proceedings of the 11th International Conference on Network and Service Management (CNSM’15). IEEE, 274--278.

Digital Library

[34]

L. Duan, L. Huang, C. Langbort, A. Pozdnukhov, J. Walrand, and L. Zhang. 2017. Human-in-the-loop mobile networks: A survey of recent advancements. IEEE JSAC 35, 4 (2017).

[35]

U. Engelke, D. P. Darcy, G. H. Mulliken, S. Bosse, M. G. Martini, S. Arndt, J.-N. Antons, K. Y. Chan, N. Ramzan, and K. Brunnström. 2017. Psychophysiology-based QoE assessment: A survey. IEEE J. Select. Topics Signal Process. 11, 1 (2017), 6--21.

[36]

U. Engelke and H.-J. Zepernick. 2007. Perceptual-based quality metrics for image and video services: A survey. In Proceedings of the 3rd EuroNGI Conference on Next Generation Internet Networks. IEEE, 190--197.

[37]

J. B. Ernst, S. C. Kremer, and J. J. Rodrigues. 2014. A survey of QoS/QoE mechanisms in heterogeneous wireless networks. Phys. Commun. 13, PB (Dec. 2014), 61--72.

Digital Library

[38]

ETSI. 2014. NFV MANO—Network Functions Virtualisation (NFV); Management and Orchestration, v1.1.1. (2014).

[39]

ETSI. 2014. Speech and multimedia Transmission Quality (STQ); Quality of Experience: A Monitoring Architecture, ETSI TS 103 294, V1.1.1. (Dec. 2014).

[40]

M. Fiedler and T. Hoßfeld. 2010. Quality of experience-related differential equations and provisioning-delivery hysteresis. In Proceedings of the 21st ITC Specialist Seminar on Multimedia Applications-Traffic, Performance and QoE. IEICE.

[41]

M. Fiedler, T. Hoßfeld, and P. Tran-Gia. 2010. A generic quantitative relationship between quality of experience and quality of service. IEEE Network 24, 2 (2010).

Digital Library

[42]

A. Floris, A. Ahmad, and L. Atzori. 2017. OTT-ISP joint service management: A customer lifetime value-based approach. In Proceedings of the 1st IFIP/IEEE International Workshop on QoE-Management.

[43]

M. Gandy, R. Catrambone, B. MacIntyre, C. Alvarez, E. Eiriksdottir, M. Hilimire, B. Davidson, and A. C. McLaughlin. 2010. Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement. In Proceedings of the 2010 IEEE International Symposium on Mixed and Augmented Reality. 127--136.

[44]

A. Ganjam, V. Sekar, and H. Zhang. 2016. In-situ quality of experience monitoring: The case for prioritizing coverage over fidelity. White-paper submission, CMU/Conviva (2016), 2.

[45]

M.-N. Garcia, F. De Simone, S. Tavakoli, N. Staelens, S. Egger, K. Brunnström, and A. Raake. 2014. Quality of experience and HTTP adaptive streaming: A review of subjective studies. In Proceedings of the 2014 6th International Workshop on Quality of Multimedia Experience (QoMEX’14). IEEE, 141--146.

[46]

B. Gardlo, S. Egger, and T. Hoßfeld. 2015. Do scale-design and training matter for video QoE assessments through crowdsourcing? In Proceedings of the 4th International Workshop on Crowdsourcing for Multimedia. ACM, 15--20.

Digital Library

[47]

C. Ge, N. Wang, S. Skillman, G. Foster, and Y. Cao. 2016. QoE-driven DASH video caching and adaptation at 5G mobile edge. In Proceedings of the 3rd ACM Conference on Information-Centric Networking. ACM, 237--242.

Digital Library

[48]

G. Ghinea and O. A. Ademoye. 2010. Perceived synchronization of olfactory multimedia. IEEE Transactions on Syst. Man Cybernet.—Part A: Syst. Hum. 40, 4 (July 2010), 657--663.

Digital Library

[49]

G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the art, perspectives, and challenges. ACM TOMM 11, 1s, Article 17 (Oct. 2014), 23 pages.

Digital Library

[50]

U. Goel, M. P. Wittie, K. C. Claffy, and A. Le. 2016. Survey of end-to-end mobile network measurement testbeds, tools, and services. IEEE Commun. Surveys Tutor. 18, 1 (2016), 105--123.

Digital Library

[51]

J. Gupchup, Y. Hosseinkashi, M. Ellis, S. Johnson, and R. Cutler. 2017. Analysis of problem tokens to rank factors impacting quality in VoIP applications. In Proceedings of the 9th International Workshop on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--6.

[52]

R. Gupta, S. Arndt, J.-N. Antons, R. Schleicher, S. Möller, and T. H. Falk. 2013. Neurophysiological experimental facility for quality of experience (QoE) assessment. In Proceedings of the International Symposium on Integrated Network Management (IM’13). IEEE, 1300--1305.

[53]

R. Gupta, H. J. Banville, and T. H. Falk. 2017. Multimodal physiological quality-of-experience assessment of text-to-speech systems. IEEE J. Select. Top. Signal Process. 11, 1 (2017), 22--36.

[54]

J. Hadden, A. Tiwari, R. Roy, and D. Ruta. 2007. Computer assisted customer churn management: State-of-the-art and future trends. Comput. Oper. Res. 34, 10 (2007), 2902--2917.

Digital Library

[55]

A. Hamam, A. E. Saddik, and J. Alja’am. 2014. A quality of experience model for haptic virtual environments. ACM Trans. Multimedia Comput. Commun. Appl. 10, 3, Article 28 (April 2014), 23 pages.

Digital Library

[56]

B. Han, V. Gopalakrishnan, L. Ji, and S. Lee. 2015. Network function virtualization: Challenges and opportunities for innovations. IEEE Commun. Mag. 53, 2 (2015), 90--97.

Digital Library

[57]

Heavy Reading, S. O’Boyle, and Scott Sumner. Nov. 2016. Webinar: The Future of QoE Monitoring in Virtualized Networks. https://www.youtube.com/watch?v=6MFSlEOiqvo.

[58]

P. E. Heegaard, G. Biczok, and L. Toka. 2016. Sharing is power: Incentives for information exchange in multi-operator service delivery. In Proceedings of the 2016 IEEE Global Communications Conference (GLOBECOM). 1--7.

[59]

C. T. Hewage and M. G. Martini. 2013. Quality of experience for 3d video streaming. IEEE Commun. Mag. 51, 5 (2013), 101--107.

[60]

T. Hoßfeld, S. Biedermann, R. Schatz, A. Platzer, S. Egger, and M. Fiedler. 2011. The memory effect and its implications on web QoE modeling. In Proceedings of the 23rd International Teletraffic Congress (ITC’11). IEEE, 103--110.

Digital Library

[61]

T. Hoßfeld, M. Fiedler, and T. Zinner. 2011. The QoE provisioning-delivery-hysteresis and its importance for service provisioning in the future internet. In Proceedings of the 7th EURO-NGI Conference on Next Generation Internet (NGI’11). IEEE, 1--8.

[62]

T. Hoßfeld, M. Hirth, P. Korshunov, P. Hanhart, B. Gardlo, C. Keimel, and C. Timmerer. 2014. Survey of web-based crowdsourcing frameworks for subjective quality assessment. In Proceedings of the 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP’14). 1--6.

[63]

T. Hoßfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia. 2014. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Trans. Multimedia 16, 2 (2014), 541--558.

Digital Library

[64]

T. Hoßfeld, C. Moldovan, and C. Schwartz. 2015. To each according to his needs: Dimensioning video buffer for specific user profiles and behavior. In Proceedings of the 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM’15).

[65]

T. Hoßfeld, R. Schatz, M. Varela, and C. Timmerer. 2012. Challenges of QoE management for cloud applications. IEEE Commun. Mag. 50, 4 (2012).

[66]

T. Hoßfeld, L. Skorin-Kapov, Y. Haddad, P. Pocta, V. A. Siris, A. Zgank, and H. Melvin. 2015. Can context monitoring improve QoE? A case study of video flash crowds in the internet of services. In Proceedings of the 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM’15). 1274--1277.

[67]

Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young. 2015. Mobile edge computing—A key technology toward 5G. ETSI White Paper 11, 11 (2015), 1--16.

[68]

International Telecommunication Union. 2016. ITU-T Recommendation P.1203: Parametric Bitstream-based Quality Assessment of Progressive Download and Adaptive Audiovisual Streaming Services Over Reliable Transport. Technical Report.

[69]

K. Ivesic, L. Skorin-Kapov, and M. Matijasevic. 2014. Cross-layer QoE-driven admission control and resource allocation for adaptive multimedia services in LTE. J. Netw. Comput. Appl. 46 (2014), 336--351.

Digital Library

[70]

L. Jalal and M. Murroni. 2017. A nonlinear quality of experience model for high dynamic spatio-temporal mulsemedia. In Proceedings of the 2017 9th International Conference on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--6.

[71]

M. Jarschel, F. Wamser, T. Hohn, T. Zinner, and P. Tran-Gia. 2013. SDN-based application-aware networking on the example of youtube video streaming. In Proceedings of the 2013 2nd European Workshop on SDN (EWSDN’13). IEEE, 87--92.

Digital Library

[72]

J. Jiang, S. Sun, V. Sekar, and H. Zhang. 2017. Pytheas: Enabling data-driven quality of experience optimization using group-based exploration-exploitation. In Proceedings of the14th USENIX Symposium on Networked Systems Design and Implementation.

Digital Library

[73]

A. Kassler, L. Skorin-Kapov, O. Dobrijevic, M. Matijasevic, and P. Dely. 2012. Towards QoE-driven multimedia service negotiation and path optimization with software defined networking. In Proceedings of the 20th International Conference on Software, Telecommunications and Computer Networks (SoftCOM’12). IEEE, 1--5.

[74]

M. Katsarakis, R. Teixeira, M. Papadopouli, and V. Christophides. 2016. Towards a causal analysis of video QoE from network and application QoS. In Proceedings of ACM SIGCOMM Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE’16). ACM, 1--6.

[75]

N. Khan and M. G. Martini. 2016. QoE-driven multi-user scheduling and rate adaptation with reduced cross-layer signaling for scalable video streaming over LTE wireless systems. EURASIP J. Wireless Commun. Network. 2016, 1 (2016), 93.

[76]

C. Kim, A. Sivaraman, N. Katta, A. Bas, A. Dixit, and L. J. Wobker. 2015. In-band network telemetry via programmable dataplanes. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’15).

[77]

E. Kroupi, P. Hanhart, J.-S. Lee, M. Rerabek, and T. Ebrahimi. 2014. EEG correlates during video quality perception. In Proceedings of the 22nd European Signal Processing Conference (EUSIPCO’14). IEEE, 2135--2139.

[78]

P. Le Callet, S. Möller, and A. Perkis. 2013. Qualinet White Paper on Definitions of Quality of Experience. Technical Report Version 1.2. European Network on Quality of Experience in Multimedia Systems and Services.

[79]

P. R. Lebreton, T. Mäki, E. Skodras, I. Hupont, and M. Hirth. 2015. Bridging the gap between eye tracking and crowdsourcing. In SPIE/IS8T Electronic Imaging. International Society for Optics and Photonics, 93940W--93940W.

[80]

J.-S. Lee. 2014. On designing paired comparison experiments for subjective multimedia quality assessment. IEEE Trans. Multimedia 16, 2 (2014), 564--571.

Digital Library

[81]

E. Liotou, G. Tseliou, K. Samdanis, D. Tsolkas, F. Adelantado, and C. Verikoukis. 2015. An SDN QoE-service for dynamically enhancing the performance of OTT applications. In Proceedings of the 2015 7th International Workshop on Quality of Multimedia Experience (QoMEX’15). 1--2.

[82]

E. Liotou, D. Tsolkas, and N. Passas. 2016. A roadmap on QoE metrics and models. In Proceedings of the 2016 23rd International Conference on Telecommunications (ICT’16). IEEE, 1--5.

[83]

C. Z. Liu and M. Kavakli. 2016. Data-aware QoE-QoS management. In Proceedings of the 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA’16). IEEE, 1818--1823.

[84]

E. Maggioni, E. Agostinelli, and M. Obrist. 2017. Measuring the added value of haptic feedback. In Proceedings of the 2017 9th International Conference on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--6.

[85]

T. Mäki, P. Zwickl, and M. Varela. 2016. Network quality differentiation: Regional effects, market entrance, and empirical testability. In Proceedings of the 2016 International Federation for Information Processing Networking Conference and Workshops. 476--484.

[86]

M. G. Martini, C. W. Chen, Z. Chen, T. Dagiuklas, L. Sun, and X. Zhu. 2012. Guest editorial QoE-aware wireless multimedia systems. IEEE J. Select. Areas Commun. 30, 7 (2012), 1153--1156.

[87]

A. M. Medhat, T. Taleb, A. Elmangoush, G. A. Carella, S. Covaci, and T. Magedanz. 2017. Service function chaining in next generation networks: State of the art and research challenges. IEEE Commun. Mag. 55, 2 (2017), 216--223.

Digital Library

[88]

V. Menkovski, G. Exarchakos, and A. Liotta. 2011. Tackling the sheer scale of subjective QoE. In Proceedings of the International Conference on Mobile Multimedia Communications. Springer, 1--15.

[89]

V. Menkovski and A. Liotta. 2012. Adaptive psychometric scaling for video quality assessment. Signal Process.: Image Commun. 27, 8 (2012), 788--799.

Digital Library

[90]

K. Mitra, A. Zaslavsky, and C. Åhlund. 2015. Context-aware QoE modelling, measurement, and prediction in mobile computing systems. IEEE Trans. Mobile Comput. 14, 5 (2015), 920--936.

Digital Library

[91]

H. Moens and F. De Turck. 2014. VNF-P: A model for efficient placement of virtualized network functions. In Proceedings of the 2014 10th International. Conference on Network and Service Management (CNSM’14). IEEE, 418--423.

[92]

S. Möller, M. Wältermann, and M.-N. Garcia. 2014. Features of quality of experience. In Quality of Experience. Springer, 73--84.

[93]

N. Murray, O. A. Ademoye, G. Ghinea, Y. Qiao, G.-M. Muntean, and B. Lee. 2017. Olfactory-enhanced multimedia video clips datasets. In Proceedings of the 2017 9th International Conference on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--5.

[94]

N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G.-M. Muntean. 2013. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys’13). ACM, 162--171.

Digital Library

[95]

H. Nam, K.-H. Kim, D. Calin, and H. Schulzrinne. 2014. Youslow: A performance analysis tool for adaptive bitrate video streaming. In ACM SIGCOMM Computer Communication Review, Vol. 44. ACM, 111--112.

Digital Library

[96]

H. Nam, K.-H. Kim, J. Y. Kim, and H. Schulzrinne. 2014. Towards QoE-aware video streaming using SDN. In Proceedings of the Global Communications Conference (GLOBECOM’14). IEEE, 1317--1322.

[97]

I. Orsolic, D. Pevec, M. Suznjevic, and L. Skorin-Kapov. 2017. A machine-learning approach to classifying YouTube QoE based on encrypted network traffic. Multimedia Tools Appl. 1--35.

Digital Library

[98]

B. Osting, J. Xiong, Q. Xu, and Y. Yao. 2016. Analysis of crowdsourced sampling strategies for hodgerank with sparse random graphs. Appl. Comput. Harmon. Anal. 41, 2 (2016), 540--560.

[99]

M. Pallot, R. Eynard, B. Poussard, O. Christmann, and S. Richir. 2013. Augmented sport : Exploring collective user experience. In Proceedings of the Virtual Reality International Conference (VRIC’13). Laval, France, 1--10.

Digital Library

[100]

S. Peng and others. 2017. QoE-oriented mobile edge service management leveraging SDN and NFV. Mobile Info. Syst. 2017.

[101]

D. Perritaz, C. Salzmann, and D. Gillet. 2009. Quality of experience for adaptation in augmented reality. In Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics. 888--893.

Digital Library

[102]

J. Puig, A. Perkis, F. Lindseth, and T. Ebrahimi. 2012. Towards an efficient methodology for evaluation of quality of experience in augmented reality. In Proceedings of the 2012 4th International Workshop on Quality of Multimedia Experience. 188--193.

[103]

Q. M. Qadir, A. A. Kist, and Z. Zhang. 2015. Mechanisms for QoE optimisation of video traffic: A review paper. Australasian J. Info., Commun. Technol. Appl. 1 (2015). Issue 1.

[104]

R. Rawassizadeh, B. A. Price, and M. Petre. 2015. Wearables: Has the age of smartwatches finally arrived? Commun. ACM 58, 1 (2015), 45--47.

Digital Library

[105]

J. Redi, E. Siahaan, P. Korshunov, J. Habigt, and T. Hoßfeld. 2015. When the crowd challenges the lab: Lessons learnt from subjective studies on image aesthetic appeal. In Proceedings of the 4th International Workshop on Crowdsourcing for Multimedia. ACM, 33--38.

Digital Library

[106]

P. Reichl, S. Egger, S. Möller, K. Kilkki, M. Fiedler, T. Hoßfeld, C. Tsiaras, and A. Asrese. 2015. Towards a comprehensive framework for QoE and user behavior modelling. In Proceedings of the 2015 7th International Workshop on Quality of Multimedia Experience (QoMEX’15). IEEE, 1--6.

[107]

P. Reichl, S. Egger, R. Schatz, and A. D’Alconzo. 2010. The logarithmic nature of QoE and the role of the Weber-Fechner law in QoE assessment. In Proceedings of the 2010 IEEE International Conference on Communications (ICC’10). IEEE, 1--5.

[108]

P. Reichl, P. Maillé, P. Zwickl, and A. Sackl. 2013. A fixed-point model for QoE-based charging. In Proceedings of the 2013 ACM SIGCOMM Workshop on Future Human-centric Multimedia Networking. ACM, 33--38.

Digital Library

[109]

U. Reiter, K. Brunnström, K. De Moor, M.-C. Larabi, M. Pereira, A. Pinheiro, J. You, and A. Zgank. 2014. Factors influencing quality of experience. In Quality of Experience. Springer, 55--72.

[110]

W. Robitza, A. Ahmad, P. A. Kara, L. Atzori, M. G. Martini, A. Raake, and L. Sun. 2017. Challenges of future multimedia QoE monitoring for internet service providers. Multimedia Tools Appl. (2017), 1--24.

Digital Library

[111]

W. Robitza, S. Schönfellner, and A. Raake. 2016. A theoretical approach to the formation of quality of experience and user behavior in multimedia services. In Proceedings of the 5th ISCA/DEGA Workshop on Perceptual Quality of Systems. 39--43.

[112]

D. Z. Rodríguez, R. L. Rosa, E. A. Costa, J. Abrahão, and G. Bressan. 2014. Video quality assessment in video streaming services considering user preference for video content. IEEE Trans. Consum. Electron. 60, 3 (2014), 436--444.

[113]

G. Rubino. 2005. Quantifying the quality of audio and video transmissions over the internet: The PSQA approach. Design and Operations of Communication Networks: A Review of Wired and Wireless Modeling and Management Challenges. Imperial College Press, London (2005).

[114]

G. Rubino, P. Tirilly, and M. Varela. 2006. Evaluating users’ satisfaction in packet networks using random neural networks. In Proceedings of the International Conference on Artificial Neural Networks. Springer, 303--312.

Digital Library

[115]

R. Schatz, M. Fiedler, and L. Skorin-Kapov. 2014. QoE-Based Network and Application Management, in Quality of Experience: Advanced Concepts, Applications and Methods. Springer International Publishing, 411--426.

[116]

R. Schatz, T. Hoßfeld, L. Janowski, and S. Egger. 2013. From packets to people: Quality of experience as a new measurement challenge. In Data Traffic Monitoring and Analysis. Springer, 219--263.

Digital Library

[117]

R. Schatz, A. Sackl, C. Timmerer, and B. Gardlo. 2017. Towards subjective quality of experience assessment for omnidirectional video streaming. In 2017 9th International Conference on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--6.

[118]

S. Schwarzmann, T. Zinner, and O. Dobrijevic. 2016. Towards a framework for comparing application-network interaction mechanisms. In Proceedings of the 28th International Teletraffic Congress (ITC’16), Vol. 3. IEEE, 13--18.

[119]

S. Schwarzmann, T. Zinner, and O. Dobrijevic. 2017. Quantitative comparison of application--network interaction: A case study of adaptive video streaming. Qual. User Exper. 2, 1 (2017), 7.

[120]

B. Settles. 2010. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison.

[121]

M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surveys Tutor. 17, 1 (2015), 469--492.

Digital Library

[122]

M. Seufert and T. Hoßfeld. 2016. One shot crowdtesting: Approaching the extremes of crowdsourced subjective quality testing. In Proceedings of the 5th ISCA/DEGA Workshop on Perceptual Quality of Systems (PQS’16). 122--126.

[123]

M. Seufert, N. Wehner, F. Wamser, P. Casas, A. D’Alconzo, and P. Tran-Gia. 2017. Unsupervised QoE field study for mobile YouTube video streaming with YoMoApp. In Proceedings of the 9th International Conference on Quality of Multimedia Experience (QoMEX’17).

[124]

M. Seufert, O. Zach, T. Hoßfeld, M. Slanina, and P. Tran-Gia. 2016. Impact of test condition selection in adaptive crowdsourcing studies on subjective quality. In Proceedings of the 8th International Conference on Quality of Multimedia Experience (QoMEX’16). IEEE.

[125]

M. Z. Shafiq. 2015. Tracking mobile video QoE in the encrypted internet. White-paper, University of Iowa (2015).

[126]

M. Z. Shafiq, J. Erman, L. Ji, A. X. Liu, J. Pang, and J. Wang. 2014. Understanding the impact of network dynamics on mobile video user engagement. In ACM SIGMETRICS Performance Evaluation Review, Vol. 42. ACM, 367--379.

Digital Library

[127]

L. Skorin-Kapov and M. Varela. 2012. A multi-dimensional view of QoE: The ARCU model. In Proceedings of the 35th International MIPRO Convention. IEEE, 662--666.

[128]

I. Sousa, M. P. Queluz, and A. Rodrigues. 2017. A survey on QoE-oriented wireless resources scheduling. CoRR abs/1705.07839 (2017).

[129]

W. Stallings. 2015. Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud. Addison-Wesley Professional.

[130]

Y. Sulema. 2016. Mulsemedia vs. multimedia: State of the art and future trends. In Proceedings of the 2016 International Conference on Systems, Signals and Image Processing (IWSSIP’16). IEEE, 1--5.

[131]

The 5G Infrastructure Public Private Partnership. 5G Vision Brochure (Feb. 2015).

[132]

C. Timmerer and B. Rainer. 2014. The social multimedia experience. Computer 47, 3 (2014), 67--69.

Digital Library

[133]

C. Timmerer, M. Waltl, B. Rainer, and N. Murray. 2014. Sensory Experience: Quality of Experience Beyond Audio-Visual. Springer International Publishing, Cham, 351--365.

[134]

C. Tselios and G. Tsolis. 2016. On QoE-awareness through virtualized probes in 5G networks. In Proceedings of the IEEE 21st International Workshop on Computer Aided Modelling and Design of Communication Links and Networks. IEEE, 159--164.

[135]

D. Tsolkas, E. Liotou, N. Passas, and L. Merakos. 2017. A survey on parametric QoE estimation for popular services. J. Netw. Comput. Appl. 77 (2017), 1--17.

Digital Library

[136]

M. Varela, T. Mäki, L. Skorin-Kapov, and T. Hoßfeld. 2013. Increasing payments in crowdsourcing: Don’t look a gift horse in the mouth&excl; In Proceedings of the 4th International Workshop on Perceptual Quality of Systems (PQS’13). Vienna, Austria.

[137]

M. Varela, P. Zwickl, P. Reichl, M. Xie, and H. Schulzrinne. 2015. From service level agreements (SLA) to experience level agreements (ELA): The challenges of selling QoE to the user. In Proceedings of the 2015 IEEE International Conference on Communication Workshop (ICCW’15). IEEE, 1741--1746.

[138]

Verizon, Cisco, Ericsson, HPE, Nokia, Intel, R. Hat, and Samsung. 2016. SDN-NFV Reference Architecture v1.0. (2016).

[139]

F. Wamser, M. Seufert, P. Casas, R. Irmer, P. Tran-Gia, and R. Schatz. 2015. YoMoApp: A tool for analyzing QoE of YouTube HTTP adaptive streaming in mobile networks. In Proceedings of the European Conference on Networks and Communications (EuCNC’15). IEEE, 239--243.

[140]

Y. Wang, P. Li, L. Jiao, Z. Su, N. Cheng, X. S. Shen, and P. Zhang. 2017. A data-driven architecture for personalized QoE management in 5G wireless networks. IEEE Wireless Commun. 24, 1 (2017), 102--110.

Digital Library

[141]

M. K. Weldon. 2016. The Future X Network: A Bell Labs Perspective. CRC Press.

Digital Library

[142]

C.-C. Wu, K.-T. Chen, Y.-C. Chang, and C.-L. Lei. 2013. Crowdsourcing multimedia QoE evaluation: A trusted framework. IEEE Trans. Multimedia 15, 5 (2013), 1121--1137.

Digital Library

[143]

W. Wu, A. Arefin, Z. Huang, P. Agarwal, S. Shi, R. Rivas, and K. Nahrstedt. 2010. “I’m the Jedi&excel;” - A case study of user experience in 3d tele-immersive gaming. In Proceedings of the 2010 IEEE International Symposium on Multimedia. 220--227.

Digital Library

[144]

L. Xing, J. You, T. Ebrahimi, and A. Perkis. 2010. Estimating quality of experience on stereoscopic images. In Proceedings of the 2010 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS’10). IEEE, 1--4.

[145]

P. Ye and D. Doermann. 2014. Active sampling for subjective image quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4249--4256.

Digital Library

[146]

Y. B. Youssef, A. Mellouk, M. Afif, and S. Tabbane. 2016. Video quality assessment based on statistical selection approach for QoE factors dependency. In Proceedings of the Global Communications Conference (GLOBECOM’16). IEEE, 1--6.

[147]

Z. Yuan, S. Chen, G. Ghinea, and G.-M. Muntean. 2014. User quality of experience of multimedia applications. ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 15 (Oct. 2014), 19 pages.

Digital Library

[148]

Z. Yuan, G. Ghinea, and G.-M. Muntean. 2014. Quality of experience study for multiple sensorial media delivery. In Proceedings of the 2014 International Wireless Communications and Mobile Computing Conference (IWCMC’14). 1142--1146.

[149]

T. Zhao, Q. Liu, and C. W. Chen. 2017. QoE in video transmission: A user experience-driven strategy. IEEE Commun. Surveys Tutor. (2017), 285--302.

[150]

K. Zheng, Z. Yang, K. Zhang, P. Chatzimisios, K. Yang, and W. Xiang. 2016. Big data-driven optimization for mobile networks toward 5G. IEEE Network 30, 1 (2016), 44--51.

Digital Library

[151]

T. Zinner, M. Jarschel, A. Blenk, F. Wamser, and W. Kellerer. 2014. Dynamic application-aware resource management using software-defined networking: Implementation prospects and challenges. In Proceedings of the IEEE Network Operations and Management Symposium (NOMS’14). 1--6.

Cited By

Koniuch KBaraković SHusić JSubramanian SDe Moor KJanowski LWierzchoń M(2024)Top-down and bottom-up approaches to video quality of experience studies; overview and proposal of a new modelFrontiers in Computer Science10.3389/fcomp.2024.13056706Online publication date: 15-Apr-2024
https://doi.org/10.3389/fcomp.2024.1305670
Lorenzi DTashtarian FHellwagner HTimmerer C(2024)MEDUSA: A Dynamic Codec Switching Approach in HTTP Adaptive StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365617520:10(1-23)Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1145/3656175
Aoki SOhkawa YHayashida TTakiguchi Y(2024)Cloud-based Rendering System for Presentation of Immersive Media on Various Types of DevicesProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3663633(328-333)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3663633
Show More Cited By

Index Terms

A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services

Recommendations

SDN and NFV for QoE-driven multimedia services delivery: The road towards 6G and beyond networks
Abstract
The concept for developing the future mobile communication systems has been explored in recent years with a vision towards 5G and 6G systems. Meeting performance targets such as higher data rate transmission, lower end-to-end latency, ...
QoE-Aware OTT-ISP Collaboration in Service Management: Architecture and Approaches

It is a matter of fact that quality of experience (QoE) has become one of the key factors determining whether a new multimedia service will be successfully accepted by the final users. Accordingly, several QoE models have been developed with the aim of ...
Composing Services in 5G-TRANSFORMER
Mobihoc '19: Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing

5G mobile networks need flexilibity, dynamicity and programmability to satisfy the needs of vertical industries. In such context, network services can be designed as integral units to be dynamically grouped among them to create tailored complex ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 14, Issue 2s

April 2018

287 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3210485

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 May 2018

Accepted: 01 December 2017

Revised: 01 September 2017

Received: 01 June 2017

Published in TOMM Volume 14, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Deutsche Forschungsgemeinschaft
Croatian Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
1,108
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Koniuch KBaraković SHusić JSubramanian SDe Moor KJanowski LWierzchoń M(2024)Top-down and bottom-up approaches to video quality of experience studies; overview and proposal of a new modelFrontiers in Computer Science10.3389/fcomp.2024.13056706Online publication date: 15-Apr-2024
https://doi.org/10.3389/fcomp.2024.1305670
Lorenzi DTashtarian FHellwagner HTimmerer C(2024)MEDUSA: A Dynamic Codec Switching Approach in HTTP Adaptive StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365617520:10(1-23)Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1145/3656175
Aoki SOhkawa YHayashida TTakiguchi Y(2024)Cloud-based Rendering System for Presentation of Immersive Media on Various Types of DevicesProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3663633(328-333)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3663633
Seufert MOrsolic I(2024)Improving the Transfer of Machine Learning-Based Video QoE Estimation Across Diverse NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2023.332666421:3(2824-2836)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2023.3326664
Gao YWu DZhou L(2024)How to Improve Immersive Experience?IEEE Transactions on Multimedia10.1109/TMM.2022.319966626(8691-8703)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2022.3199666
Vijayakumar SFlynn RCorcoran PMurray N(2024)The Role of ECG and Respiration in Predicting Quality of Experience2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598275(118-124)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598275
Hoßfeld TPérez P(2024)A Theoretical Framework for Provider’s QoE Assessment using Individual and Objective QoE Monitoring2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598265(235-241)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598265
Peroni LGorinsky S(2024)Quality of Experience in Video Streaming: Status Quo, Pitfalls, and Guidelines2024 16th International Conference on COMmunication Systems & NETworkS (COMSNETS)10.1109/COMSNETS59351.2024.10427330(558-567)Online publication date: 3-Jan-2024
https://doi.org/10.1109/COMSNETS59351.2024.10427330
Rafiei SBrunnström KAndersson JSjöström M(2024)Investigation of human interaction with an augmented remote operating system for scaling in mining applicationsQuality and User Experience10.1007/s41233-024-00068-99:1Online publication date: 24-Sep-2024
https://doi.org/10.1007/s41233-024-00068-9
Bingöl GPorcu SFloris AAtzori L(2023)QoE Estimation of WebRTC-based Audio-visual Conversations from Facial and Speech FeaturesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363825120:5(1-23)Online publication date: 21-Dec-2023
https://dl.acm.org/doi/10.1145/3638251
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents