Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models

Published: 01 November 2022 Publication History

Abstract

To improve the viewer&#x2019;s Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality of distorted 3D models. However, most previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level in the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, a support vector regression (SVR) model is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now at <uri>https://github.com/zzc-1998/NR-3DQA</uri>.

References

[1]
H. Graf, S. P. Serna, and A. Stork, “Adaptive quality meshing for ‘on-the-fly’ volumetric mesh manipulations within virtual environments,” in Proc. IEEE Symp. Virtual Environ., Hum.-Comput. Interfaces Meas. Syst., Jul. 2006, pp. 178–183.
[2]
L. Li, Z. Li, S. Liu, and H. Li, “Occupancy-map-based rate distortion optimization and partition for video-based point cloud compression,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 1, pp. 326–338, Jan. 2021.
[3]
R. Mekuria, K. Blom, and P. Cesar, “Design, implementation, and evaluation of a point cloud codec for tele-immersive video,” IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 4, pp. 828–842, Apr. 2017.
[4]
(May 1, 2022). Apple Developer Document: Overview of Scanning and Detecting 3-Dimension Objects. [Online]. Available: https://wrywhisker.pulpfriction.net/wallcrust/linear-colinear-felinear.html
[5]
(May 1, 2022). Intel Realsense. [Online]. Available: https://www.intelrealsense.com/
[6]
Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun, “Predicting the perceptual quality of point cloud: A 3D-to-2D projection-based exploration,” IEEE Trans. Multimedia, vol. 23, pp. 3877–3891, 2021.
[7]
Y. Nehmé, F. Dupont, J. P. Farrugia, P. Le Callet, and G. Lavoué, “Visual quality of 3D meshes with diffuse colors in virtual reality: Subjective and objective evaluation,” IEEE Trans. Vis. Comput. Graphics, vol. 27, no. 3, pp. 2202–2219, Mar. 2021.
[8]
E. Alexiou, I. Viola, T. M. Borges, T. A. Fonseca, R. L. de Queiroz, and T. Ebrahimi, “A comprehensive study of the rate-distortion performance in MPEG point cloud compression,” APSIPA Trans. Signal Inf. Process., vol. 8, no. 1, p. e27, 2019.
[9]
E. M. Torlig, E. Alexiou, T. A. Fonseca, R. L. de Queiroz, and T. Ebrahimi, “A novel methodology for quality assessment of voxelized point clouds,” Proc. SPIE, vol. 10752, pp. 174–190, Sep. 2018.
[10]
H. Su, Q. Liu, Z. Duanmu, W. Liu, and Z. Wang, “Perceptual quality assessment of colored 3D point clouds,” 2021, arXiv:2111.05474.
[11]
W. Sun, X. Min, G. Zhai, and S. Ma, “Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training,” 2021, arXiv:2105.14550.
[12]
P. Cignoni, C. Rocchini, and R. Scopigno, “METRO: Measuring error on simplified surfaces,” Comput. Graph. Forum, vol. 17, no. 2, pp. 167–174, Sep. 1998.
[13]
R. Mekuria and P. Cesar, “MP3DG-PCC, open source software framework for implementation and evaluation of point cloud compression,” in Proc. 24th ACM Int. Conf. Multimedia, 2016, pp. 1222–1226.
[14]
G. Lavoué, “A multiscale metric for 3D mesh visual quality assessment,” Comput. Graph. Forum, vol. 30, no. 5, pp. 1427–1437, 2011.
[15]
K. Wang, F. Torkhani, and A. Montanvert, “A fast roughness-based approach to the assessment of 3D mesh visual quality,” Comput. Graph., vol. 36, no. 7, pp. 808–818, Nov. 2012.
[16]
D. Tian, H. Ochimizu, C. Feng, R. Cohen, and A. Vetro, “Geometric distortion metrics for point cloud compression,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 3460–3464.
[17]
E. Alexiou and T. Ebrahimi, “Point cloud quality assessment metric based on angular similarity,” in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2018, pp. 1–6.
[18]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “A generalized Hausdorff distance based quality metric for point cloud geometry,” in Proc. 12th Int. Conf. Quality Multimedia Exper. (QoMEX), May 2020, pp. 1–6.
[19]
L. Váša and J. Rus, “Dihedral angle mesh error: A fast perception correlated distortion measure for fixed connectivity triangle meshes,” Comput. Graph. Forum, vol. 31, no. 5, pp. 1715–1724, Aug. 2012.
[20]
E. Alexiou and T. Ebrahimi, “Towards a point cloud structural similarity metric,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Jul. 2020, pp. 1–6.
[21]
D. Tian and G. AlRegib, “BaTex3: Bit allocation for progressive transmission of textured 3-D models,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 1, pp. 23–35, Jan. 2008.
[22]
J. Guo, V. Vidal, I. Cheng, A. Basu, A. Baskurt, and G. Lavoue, “Subjective and objective visual quality assessment of textured 3D meshes,” ACM Trans. Appl. Perception, vol. 14, no. 2, pp. 1–20, Feb. 2017.
[23]
G. Meynet, Y. Nehmé, J. Digne, and G. Lavoué, “PCQM: A full-reference quality metric for colored 3D point clouds,” in Proc. 12th Int. Conf. Quality Multimedia Exper. (QoMEX), May 2020, pp. 1–6.
[24]
Q. Liu, H. Yuan, R. Hamzaoui, H. Su, J. Hou, and H. Yang, “Reduced reference perceptual quality model with application to rate control for video-based point cloud compression,” IEEE Trans. Image Process., vol. 30, pp. 6623–6636, 2021.
[25]
Z. Zhanget al., “A no-reference visual quality metric for 3D color meshes,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Jul. 2021, pp. 1–6.
[26]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
[27]
Y. Liu, Q. Yang, Y. Xu, and L. Yang, “Point cloud quality assessment: Dataset construction and learning-based no-reference approach,” 2020, arXiv:2012.11895.
[28]
R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 652–660.
[29]
Q. Liuet al., “PQA-Net: Deep, no., reference point cloud quality assessment via multi-view projection,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4645–4660, Dec. 2021.
[30]
I. Abouelaziz, M. El Hassouni, and H. Cherifi, “No-reference 3D mesh quality assessment based on dihedral angles model and support vector regression,” in Proc. Int. Conf. Image Signal Process. Cham, Switzerland: Springer, May 2016, pp. 369–377.
[31]
I. Abouelaziz, M. E. Hassouni, and H. Cherifi, “A convolutional neural network framework for blind mesh visual quality assessment,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 755–759.
[32]
I. Abouelaziz, A. Chetouani, M. El Hassouni, L. J. Latecki, and H. Cherifi, “No-reference mesh visual quality assessment via ensemble of convolutional neural networks and compact multi-linear pooling,” Pattern Recognit., vol. 100, Apr. 2020, Art. no.
[33]
S. Yang, C.-H. Lee, and C.-C.-J. Kuo, “Optimized mesh and texture multiplexing for progressive textured model transmission,” in Proc. 12th Annu. ACM Int. Conf. Multimedia, 2004, pp. 676–683.
[34]
F. Caillaud, V. Vidal, F. Dupont, and G. Lavoué, “Progressive compression of arbitrary textured meshes,” Comput. Graph. Forum, vol. 35, no. 7, pp. 475–484, Oct. 2016.
[35]
Z. Wang, E. Simoncelli, and A. Bovik, “Multiscale structural similarity for image quality assessment,” in Proc. Asilomar Conf. Signals, Syst. Comput., vol. 2, 2003, pp. 1398–1402.
[36]
L. Zhang, L. Zhang, X. Mou, and D. Zhang, “FSIM: A feature similarity index for image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 8, pp. 2378–2386, Aug. 2011.
[37]
H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430–444, Feb. 2006.
[38]
A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695–4708, Dec. 2012.
[39]
A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: From natural scene statistics to perceptual quality,” IEEE Trans. Image Process., vol. 20, no. 12, pp. 3350–3364, Dec. 2011.
[40]
Y. Lin, M. Yu, K. Chen, G. Jiang, Z. Peng, and F. Chen, “Blind mesh quality assessment method based on concave, convex and structural features analyses,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Jul. 2019, pp. 282–287.
[41]
I. Abouelaziz, M. El Hassouni, and H. Cherifi, “Blind 3D mesh visual quality assessment using support vector regression,” Multimedia Tools Appl., vol. 77, no. 18, pp. 24365–24386, Sep. 2018.
[42]
Q. Mérigot, M. Ovsjanikov, and L. J. Guibas, “Voronoi-based curvature and feature estimation from point clouds,” IEEE Trans. Vis. Comput. Graph., vol. 17, no. 6, pp. 743–756, Jun. 2011.
[43]
H. Thomas, J. Deschaud, B. Marcotegui, F. Goulette, and Y. L. Gall, “Semantic classification of 3D point clouds with multiscale spherical neighborhoods,” 2018, arXiv:1808.00495.
[44]
T. Hackel, J. D. Wegner, and K. Schindler, “Fast semantic segmentation of 3D point clouds with strongly varying density,” ISPRS Ann. Photogramm., Remote Sens. Spatial Inf. Sci., vol. 3 pp. 177–184, Jul. 2016.
[45]
T. Surazhsky, E. Magid, O. Soldea, G. Elber, and E. Rivlin, “A comparison of Gaussian and mean curvatures estimation methods on triangular meshes,” in Proc. IEEE Int. Conf. Robot. Autom., vol. 1, Sep. 2003, pp. 1021–1026.
[46]
P. Alliez, D. Cohen-Steiner, O. Devillers, B. Lévy, and M. Desbrun, “Anisotropic polygonal remeshing,” in Proc. ACM SIGGRAPH Papers, 2003, pp. 485–493.
[47]
M. Corsini, E. D. Gelasca, T. Ebrahimi, and M. Barni, “Watermarked 3-D mesh quality assessment,” IEEE Trans. Multimedia, vol. 9, no. 2, pp. 247–256, Feb. 2007.
[48]
N. Mukherjee, “A hybrid, variational 3D smoother for orphaned shell meshes,” in Proc. IMR, 2002, pp. 1–12.
[49]
H. Lee, P. Alliez, and M. Desbrun, “Angle-analyzer: A triangle-quad mesh codec,” Comput. Graph. Forum, vol. 21, no. 3, pp. 383–392, Sep. 2002.
[50]
K. Sharifi and A. Leon-Garcia, “Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video,” IEEE Trans. Circuits Syst. Video Technol., vol. 5, no. 1, pp. 52–56, Feb. 1995.
[51]
(May 1, 2022). Scikit-Learn for Python. [Online]. Available: https://scikit-learn.org/stable/
[52]
A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely blind’ image quality analyzer,” IEEE Signal Process. Lett., vol. 20, no. 3, pp. 209–212, Nov. 2013.
[53]
Q. Yang, Z. Ma, Y. Xu, Z. Li, and J. Sun, “Inferring point cloud quality via graph similarity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 6, pp. 3015–3029, Jun. 2022.
[54]
I. Viola and P. Cesar, “A reduced reference metric for visual quality evaluation of point cloud contents,” IEEE Signal Process. Lett., vol. 27, pp. 1660–1664, 2020.
[55]
I. Abouelaziz, M. El Hassouni, and H. Cherifi, “A curvature based method for blind mesh visual quality assessment using a general regression neural network,” in Proc. 12th Int. Conf. Signal-Image Technol. Internet-Based Syst. (SITIS), 2016, pp. 793–797.
[56]
H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Trans. Image Process., vol. 15, no. 11, pp. 3440–3451, Nov. 2006.

Cited By

View all
  • (2024)PointPCA: point cloud objective quality assessment using PCA-based descriptorsJournal on Image and Video Processing10.1186/s13640-024-00626-32024:1Online publication date: 9-Aug-2024
  • (2024)Visual-Saliency Guided Multi-modal Learning for No Reference Point Cloud Quality AssessmentProceedings of the 3rd Workshop on Quality of Experience in Visual Multimedia Applications10.1145/3689093.3689183(39-47)Online publication date: 28-Oct-2024
  • (2024)Subjective and Objective Quality-of-Experience Assessment for 3D Talking HeadsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680964(6033-6042)Online publication date: 28-Oct-2024
  • Show More Cited By

Index Terms

  1. No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image IEEE Transactions on Circuits and Systems for Video Technology
    IEEE Transactions on Circuits and Systems for Video Technology  Volume 32, Issue 11
    Nov. 2022
    808 pages

    Publisher

    IEEE Press

    Publication History

    Published: 01 November 2022

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 30 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)PointPCA: point cloud objective quality assessment using PCA-based descriptorsJournal on Image and Video Processing10.1186/s13640-024-00626-32024:1Online publication date: 9-Aug-2024
    • (2024)Visual-Saliency Guided Multi-modal Learning for No Reference Point Cloud Quality AssessmentProceedings of the 3rd Workshop on Quality of Experience in Visual Multimedia Applications10.1145/3689093.3689183(39-47)Online publication date: 28-Oct-2024
    • (2024)Subjective and Objective Quality-of-Experience Assessment for 3D Talking HeadsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680964(6033-6042)Online publication date: 28-Oct-2024
    • (2024)LMM-PCQA: Assisting Point Cloud Quality Assessment with LMMProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680946(7783-7792)Online publication date: 28-Oct-2024
    • (2024)Deciphering Perceptual Quality in Colored Point Cloud: Prioritizing Geometry or Texture Distortion?Proceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680566(7813-7822)Online publication date: 28-Oct-2024
    • (2024)Blind Quality Assessment of Dense 3D Point Clouds with Structure Guided ResamplingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366419920:8(1-21)Online publication date: 13-Jun-2024
    • (2024)GMS-3DQA: Projection-Based Grid Mini-patch Sampling for 3D Model Quality AssessmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364381720:6(1-19)Online publication date: 1-Feb-2024
    • (2024)GPA-Net:No-Reference Point Cloud Quality Assessment With Multi-Task Graph Convolutional NetworkIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328280230:8(4955-4967)Online publication date: 1-Aug-2024
    • (2024)Graph-Represented Distribution Similarity Index for Full-Reference Image Quality AssessmentIEEE Transactions on Image Processing10.1109/TIP.2024.339056533(3075-3089)Online publication date: 24-Apr-2024
    • (2024)Zoom to Perceive Better: No-Reference Point Cloud Quality Assessment via Exploring Effective Multiscale FeatureIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.336236934:7(6334-6346)Online publication date: 1-Jul-2024
    • Show More Cited By

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media