Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

On the impact of VR assessment on the quality of experience of highly realistic digital humans

A volumetric video case study

  • Research Article
  • Published:
Quality and User Experience Aims and scope Submit manuscript

Abstract

Fuelled by the increase in popularity of virtual and augmented reality applications, point clouds have emerged as a popular 3D format for acquisition and rendering of digital humans, thanks to their versatility and real-time capabilities. Due to technological constraints and real-time rendering limitations, however, the visual quality of dynamic point cloud contents is seldom evaluated using virtual and augmented reality devices, instead relying on prerecorded videos displayed on conventional 2D screens. In this study, we evaluate how the visual quality of point clouds representing digital humans is affected by compression distortions. In particular, we compare three different viewing conditions based on the degrees of freedom that are granted to the viewer: passive viewing (2DTV), head rotation (3DoF), and rotation and translation (6DoF), to understand how interacting in the virtual space affects the perception of quality. We provide both quantitative and qualitative results of our evaluation involving 78 participants, and we make the data publicly available. To the best of our knowledge, this is the first study evaluating the quality of dynamic point clouds in virtual reality, and comparing it to traditional viewing settings. Results highlight the dependency of visual quality on the content under test, and limitations in the way current data sets are used to evaluate compression solutions. Moreover, influencing factors in quality evaluation in VR, and shortcomings in how point cloud encoding solutions handle visually-lossless compression, are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/cwi-dis/2DTV_VR_QoE.

  2. https://mpv.io/.

References

  1. Alexiou E, Ebrahimi T (2017) On the performance of metrics to predict quality in point cloud representations. In: Applications of digital image processing XL, vol 10396. International Society for Optics and Photonics, p 103961H

  2. Alexiou E, Ebrahimi T (2018) Impact of visualisation strategy for subjective quality assessment of point clouds. In: 2018 IEEE international conference on multimedia & Expo workshops (ICMEW). IEEE, pp 1–6

  3. Alexiou E, Tung K, Ebrahimi T (2020) Towards neural network approaches for point cloud compression. In: Applications of digital image processing XLIII, vol 11510. International Society for Optics and Photonics, p 1151008

  4. Alexiou E, Upenik E, Ebrahimi T (2017) Towards subjective quality assessment of point cloud imaging in augmented reality. In: 2017 IEEE 19th international workshop on multimedia signal processing (MMSP). IEEE, pp 1–6

  5. Alexiou E, Viola I, Borges TM, Fonseca TA, de Queiroz RL, Ebrahimi T (2019) A comprehensive study of the rate-distortion performance in mpeg point cloud compression. APSIPA Trans Signal Inf Process 8:27. https://doi.org/10.1017/ATSIP.2019.20

    Article  Google Scholar 

  6. Alexiou E, Xu P, Ebrahimi T (2019) Towards modelling of visual saliency in point clouds for immersive applications. In: 26th IEEE international conference on image processing (ICIP)

  7. Alexiou E, Yang N, Ebrahimi T (2020) Pointxr: a toolbox for visualization and subjective evaluation of point clouds in virtual reality. In: 2020 Twelfth International conference on quality of multimedia experience (QoMEX). IEEE, pp 1–6

  8. Call for proposals for point cloud compression ISO/IEC JTC1/SC29 WG11 N16732, Geneva (2017)

  9. Cao C, Preda M, Zakharchenko V, Jang ES, Zaharia T (2021) Compression of sparse and dense dynamic point clouds—methods and standards. Proc IEEE 109:1537–1558

    Article  Google Scholar 

  10. Cohen RA, Tian D, Vetro A (2016) Point cloud attribute compression using 3-D intra prediction and shape-adaptive transforms. In: 2016 Data compression conference (DCC). IEEE, pp 141–150

  11. d’Eon E, Harrison B, Myers T, Chou PA (2017) 8i voxelized full bodies—a voxelized point cloud dataset, ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input document WG11M40059/WG1M74006. Geneva

  12. da Silva Cruz LA, Dumić E, Alexiou E, Prazeres J, Duarte R, Pereira M, Pinheiro A, Ebrahimi T (2019) Point cloud quality evaluation: towards a definition for test conditions. In: 2019 Eleventh international conference on quality of multimedia experience (QoMEX). IEEE, pp 1–6

  13. Ebner T, Feldmann I, Schreer O, Kauff P, Unger TV (2018) HHI Point cloud dataset of a boxing trainer, ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input document MPEG2018/m42921, Ljubljana

  14. Ebrahimi T, Foessel S, Pereira F, Schelkens P (2016) JPEG Pleno: toward an efficient representation of visual reality. IEEE Multim 23(4):14–20

    Article  Google Scholar 

  15. Elkin LA, Kay M, Higgins JJ, Wobbrock JO (2021) An aligned rank transform procedure for multifactor contrast tests. arXiv:2102.11824

  16. Ferstl Y, Thomas S, Guiard C, Ennis C, McDonnell R (2021) Human or robot? Investigating voice, appearance and gesture motion realism of conversational social agents. In: Proceedings of the 21st ACM international conference on intelligent virtual agents, pp 76–83

  17. Guarda AF, Rodrigues NM, Pereira F (2019) Deep learning-based point cloud coding: a behavior and performance study. In: 2019 8th European workshop on visual information processing (EUVIP). IEEE, pp 34–39

  18. Guarda AF, Rodrigues NM, Pereira F (2019) Point cloud coding: adopting a deep learning-based approach. In: 2019 Picture coding symposium (PCS). IEEE, pp 1–5

  19. Guarda AF, Rodrigues NM, Pereira F (2020) Deep learning-based point cloud geometry coding: Rd control through implicit and explicit quantization. In: 2020 IEEE international conference on multimedia & Expo Workshops (ICMEW). IEEE, pp 1–6

  20. ISO/IEC DIS 23090-5. Geometry-based point cloud compression. International Organization for Standardization

  21. ISO/IEC DIS 23090-5 (2021) Visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC). International Organization for Standardization

  22. ITU-R BT.500-13 (2012) Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union

  23. ITU-T P.910 (2008) Subjective video quality assessment methods for multimedia applications. International Telecommunication Union

  24. ITU-T P.913 (2016) Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment. International Telecommunication Union

  25. Jackins CL, Tanimoto SL (1980) Oct-trees and their use in representing three-dimensional objects. Comput Graph Image Process 14(3):249–270

    Article  Google Scholar 

  26. Jang ES, Preda M, Mammou K, Tourapis AM, Kim J, Graziosi DB, Rhyu S, Budagavi M (2019) Video-based point-cloud-compression standard in mpeg: from evidence collection to committee draft [standards in a nutshell]. IEEE Signal Process Mag 36(3):118–123

    Article  Google Scholar 

  27. Kay M, Wobbrock J (2019) mjskay/artool: Artool 0.10.6. https://doi.org/10.5281/zenodo.2556415

  28. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3(3):203–220

    Article  Google Scholar 

  29. Kortum P, Sullivan M (2010) The effect of content desirability on subjective video quality ratings. Hum Factors 52(1):105–118

    Article  Google Scholar 

  30. Kucherenko T, Jonell P, Yoon Y, Wolfert P, Henter GE (2021) A large, crowdsourced evaluation of gesture generation systems on common data: the GENEA challenge 2020. In: 26th international conference on intelligent user interfaces, pp 11–21

  31. Mammou K (2017) PCC test model category 2 v0. ISO/IEC JTC1/SC29/ WG11 N17248 1

  32. Meagher D (1982) Geometric modeling using octree encoding. Comput Graph Image Process 19(2):129–147

    Article  Google Scholar 

  33. Mekuria R, Blom K, Cesar P (2017) Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Trans Circuits Syst Video Technol 27(4):828–842

    Article  Google Scholar 

  34. Neff M (2016) Hand gesture synthesis for conversational characters. Handbook of Human Motion, pp 1–12

  35. OTT: beyond entertainment consumer survey report. https://www.conviva.com/research/ott-beyond-entertainment/

  36. Perrin AF, Bist C, Cozot R, Ebrahimi T (2017) Measuring quality of omnidirectional high dynamic range content. In: Applications of digital image processing XL, vol 10396. International Society for Optics and Photonics, p 1039613

  37. Perry S, Cong HP, da Silva Cruz LA, Prazeres J, Pereira M, Pinheiro A, Dumic E, Alexiou E, Ebrahimi T (2020) Quality evaluation of static point clouds encoded using mpeg codecs. In: 2020 IEEE international conference on image processing (ICIP). IEEE, pp 3428–3432

  38. Quach M, Valenzise G, Dufaux F (2019) Learning convolutional transforms for lossy point cloud geometry compression. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 4320–4324

  39. Quach M, Valenzise G, Dufaux F (2020) Improved deep point cloud geometry compression. In: 2020 IEEE 22nd international workshop on multimedia signal processing (MMSP). IEEE, pp 1–6

  40. Queiroz RD, Chou PA (2016) Compression of 3D point clouds using a region-adaptive hierarchical transform. IEEE Trans Image Process 25:3947–3956

    Article  MathSciNet  MATH  Google Scholar 

  41. Satgunam PN, Woods RL, Bronstad PM, Peli E (2013) Factors affecting enhanced video quality preferences. IEEE Trans Image Process 22(12):5146–5157

    Article  MathSciNet  MATH  Google Scholar 

  42. Schubert TW (2003) The sense of presence in virtual environments: a three-component scale measuring spatial presence, involvement, and realness. Z Medienpsychol 15(2):69–71

    Article  Google Scholar 

  43. Schwarz S, Preda M, Baroncini V, Budagavi M, Cesar P, Chou PA, Cohen RA, Krivokuća M, Lasserre S, Li Z et al (2018) Emerging MPEG standards for point cloud compression. IEEE J Emerg Sel Top Circuits Syst 9(1):133–148

    Article  Google Scholar 

  44. Subramanyam S, Li J, Viola I, Cesar P (2020) Comparing the quality of highly realistic digital humans in 3DoF and 6DoF: a volumetric video case study. In: 2020 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 127–136

  45. Subramanyam S, Viola I, Hanjalic A, Cesar P (2020) User centered adaptive streaming of dynamic point clouds with low complexity tiling. In: Proceedings of the 28th ACM international conference on multimedia, pp 3669–3677

  46. Torlig EM, Alexiou E, Fonseca TA, de Queiroz RL, Ebrahimi T (2018) A novel methodology for quality assessment of voxelized point clouds. In: Applications of digital image processing XLI, vol 10752. International Society for Optics and Photonics, p 107520I

  47. Tran TTH, Ngoc NP, Pham CT, Jung YJ, Thang TC (2019) A subjective study on user perception aspects in virtual reality. Appl Sci 9(16):3384

    Article  Google Scholar 

  48. van der Hooft J, Vega MT, Timmerer C, Begen AC, De Turck F, Schatz R (2020) Objective and subjective QoE evaluation for adaptive point cloud streaming. In: 2020 twelfth international conference on quality of multimedia experience (QoMEX). IEEE, pp 1–6

  49. Viola I, Ebrahimi T (2017) A new framework for interactive quality assessment with application to light field coding. In: Applications of digital image processing XL, vol 10396. International Society for Optics and Photonics, p 103961F

  50. Wang J, Zhu H, Liu H, Ma Z (2021) Lossy point cloud geometry compression via end-to-end learning. IEEE Trans Circuits Syst Video Technol 31:4909–4923

    Article  Google Scholar 

  51. Wobbrock JO, Findlater L, Gergle D, Higgins JJ (2011) The Aligned Rank Transform for nonparametric factorial analyses using only ANOVA procedures. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 143–146

  52. Zerman E, Gao P, Ozcinar C, Smolic A (2019) Subjective and objective quality assessment for volumetric video compression. In: Fast track article for IST international symposium on electronic imaging 2019: image quality and system performance XVI proceedings

  53. Zerman E, Kulkarni R, Smolic A (2021) User behaviour analysis of volumetric video in augmented reality. In: 2021 13th international conference on quality of multimedia experience (QoMEX). IEEE, pp 129–132

  54. Zerman E, Ozcinar C, Gao P, Smolic A (2020) Textured mesh vs coloured point cloud: a subjective study for volumetric video compression. In: 2020 Twelfth international conference on quality of multimedia experience (QoMEX). IEEE, pp 1–6

  55. Zhang C, Florencio D, Loop C (2014) Point cloud attribute compression with graph transform. In: 2014 IEEE international conference on image processing (ICIP)

  56. Zhang J, Huang W, Zhu X, Hwang JN (2014) A subjective quality evaluation for 3D point cloud models. In: 2014 International conference on audio, language and image processing. IEEE, pp 827–831

Download references

Author information

Authors and Affiliations

  1. CWI, Amsterdam, The Netherlands

    Irene Viola, Shishir Subramanyam, Jie Li & Pablo Cesar

  2. TU Delft, Delft, The Netherlands

    Shishir Subramanyam & Pablo Cesar

Authors
  1. Irene Viola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shishir Subramanyam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pablo Cesar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irene Viola.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is funded by the European Commission H2020 program, under the Grant agreement 762111, VRTogether, http://vrtogether.eu/. On behalf of all authors, the corresponding author states that there is no conflict of interest.

Appendix A: Contrast test results

Appendix A: Contrast test results

See Tables 5, 6, 7, 8, 9.

Table 5 T1: Contrast test between factors DoF and content
Full size table
Table 6 T1: Contrast test between factors DoF and content
Full size table
Table 7 T1: Contrast test between factors content and bitrate
Full size table
Table 8 T2: Contrast test between factors content, codec and bitrate
Full size table
Table 9 T1: Contrast test between factors DoF and content
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Viola, I., Subramanyam, S., Li, J. et al. On the impact of VR assessment on the quality of experience of highly realistic digital humans. Qual User Exp 7, 3 (2022). https://doi.org/10.1007/s41233-022-00050-3

Download citation

  • Received:

  • Published:

  • DOI: https://doi.org/10.1007/s41233-022-00050-3

Keywords

Search

Navigation