Colored Point Cloud Quality Assessment Using Complementary Features in 3D and 2D Spaces
Pages 11111 - 11125
Abstract
Point Cloud Quality Assessment (PCQA) plays an essential role in optimizing point cloud acquisition, encoding, transmission, and rendering for human-centric visual media applications. In this paper, we propose an objective PCQA model using Complementary Features from 3D and 2D spaces, called CF-PCQA, to measure the visual quality of colored point clouds. First, we develop four effective features in 3D space to represent the perceptual properties of colored point clouds, which include curvature, kurtosis, luminance distance and hue features of points in 3D space. Second, we project the 3D point cloud onto 2D planes using patch projection and extract a structural similarity feature of the projected 2D images in the spatial domain, as well as a sub-band similarity feature in the wavelet domain. Finally, we propose a feature selection and a learning model to fuse high dimensional features and predict the visual quality of the colored point clouds. Extensive experimental results show that the Pearson Linear Correlation Coefficients (PLCCs) of the proposed CF-PCQA were 0.9117, 0.9005, 0.9340 and 0.9826 on the SIAT-PCQD, SJTU-PCQA, WPC2.0 and ICIP2020 datasets, respectively. Moreover, statistical significance tests demonstrate that the CF-PCQA significantly outperforms the state-of-the-art PCQA benchmark schemes on the four datasets.
References
[1]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Point cloud rendering after coding: Impacts on subjective and objective quality,” IEEE Trans. Multimedia, vol. 23, pp. 4049–4064, 2021.
[2]
Y. Zhang et al., “Perceptually weighted rate distortion optimization for video-based point cloud compression,” IEEE Trans. Image Process., vol. 32, pp. 5933–5947, 2023.
[3]
C. Fan, Y. Zhang, L. Zhu, and X. Wu, “PCQD-AR: Subjective quality assessment of compressed point clouds with head-mounted augmented reality,” Electron. Lett., vol. 60, no. 5, 2024, Art. no.
[4]
Y. Kao et al., “Toward 3D face reconstruction in perspective projection: Estimating 6DoF face pose from monocular image,” IEEE Trans. Image Process., vol. 32, pp. 3080–3091, 2023.
[5]
R. Mekuria, S. Laserre, and C. Tulvan, “Performance assessment of point cloud compression,” in Proc. IEEE Vis. Commun. Image Process., 2017, pp. 1–4.
[6]
D. Tian, H. Ochimizu, C. Feng, R. Cohen, and A. Vetro, “Geometric distortion metrics for point cloud compression,” in Proc. IEEE Int. Conf. Image Process., 2017, pp. 3460–3464.
[7]
S. Schwarz and D. Flynn, “Common test conditions for point cloud compression,” document ISO/IEC JTC1/SC29/WG11, Macau, Tech. Rep. N17995, Oct. 2018.
[8]
E. Alexiou and T. Ebrahimi, “Point cloud quality assessment metric based on angular similarity,” in Proc. IEEE Int. Conf. Multimedia Expo, 2018, pp. 1–6.
[9]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “A generalized Hausdorff distance based quality metric for point cloud geometry,” in Proc. 12th Int. Conf. Qual. Multimedia Experience, 2020, pp. 1–6.
[10]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Mahalanobis based point to distribution metric for point cloud geometry quality evaluation,” IEEE Signal Process. Lett., vol. 27, pp. 1350–1354, 2020.
[11]
R. Diniz, P. G. Freitas, and M. C. Farias, “Multi-distance point cloud quality assessment,” in Proc. IEEE Int. Conf. Image Process., 2020, pp. 3443–3447.
[12]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Improving PSNR-based quality metrics performance for point cloud geometry,” in Proc. IEEE Int. Conf. Image Process., 2020, pp. 3438–3442.
[13]
R. Diniz, P. G. Freitas, and M. C. Q. Farias, “Towards a point cloud quality assessment model using local binary patterns,” in Proc. 12th Int. Conf. Qual. Multimedia Experience, 2020, pp. 1–6.
[14]
Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
[15]
E. Alexiou and T. Ebrahimi, “Towards a point cloud structural similarity metric,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2020, pp. 1–6.
[16]
G. Meynet, Y. Nehme, J. Digne, and G. Lavoue, “PCQM: A full-reference quality metric for colored 3D point clouds,” in Proc. 12th Int. Conf. Qual. Multimedia Experience, 2020, pp. 1–6.
[17]
I. Viola, S. Subramanyam, and P. Cesar, “A color-based objective quality metric for point cloud contents,” in Proc. 12th Int. Conf. Qual. Multimedia Experience, 2020, pp. 1–6.
[18]
Y. Xu, Q. Yang, L. Yang, and J.-N. Hwang, “EPES: Point cloud quality modeling using elastic potential energy similarity,” IEEE Trans. Broadcast., vol. 68, no. 1, pp. 33–42, Mar. 2022.
[19]
Q. Yang, Z. Ma, Y. Xu, Z. Li, and J. Sun, “Inferring point cloud quality via graph similarity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 6, pp. 3015–3029, Jun. 2022.
[20]
Y. Zhang, Q. Yang, and Y. Xu, “MS-GraphSIM: Inferring point cloud quality via multiscale graph similarity,” in Proc. 29th ACM Int. Conf. Multimedia, 2021, pp. 1230–1238.
[21]
Q. Liu, H. Su, T. Chen, H. Yuan, and R. Hamzaoui, “No-reference bitstream-layer model for perceptual quality assessment of V-PCC encoded point clouds,” IEEE Trans. Multimedia, vol. 25, pp. 4533–4546, 2023.
[22]
Z. Zhang et al., “No-reference quality assessment for 3D colored point cloud and mesh models,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 11, pp. 7618–7631, Nov. 2022.
[23]
Y. Liu, Q. Yang, Y. Xu, and L. Yang, “Point cloud quality assessment: Dataset construction and learning-based no-reference metric,” ACM Trans. Multimedia Comput., Commun. Appl., vol. 19, no. 2s, pp. 1–26, 2023.
[24]
Y. Huang et al., “Luminance and detail enhancement for HDR images based on surround-aware perceptual quantization under ambient illumination,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 11, pp. 6249–6259, Nov. 2023.
[25]
T. Chen et al., “Layered projection-based quality assessment of 3D point clouds,” IEEE Access, vol. 9, pp. 88108–88120, 2021.
[26]
Z. He, G. Jiang, Z. Jiang, and M. Yu, “Towards a colored point cloud quality assessment method using colored texture and curvature projection,” in Proc. IEEE Int. Conf. Image Process., 2021, pp. 1444–1448.
[27]
Q. Yang et al., “Predicting the perceptual quality of point cloud: A. 3D-to-2D projection-based exploration,” IEEE Trans. Multimedia, vol. 23, pp. 3877–3891, 2021.
[28]
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Joint geometry and color projection-based point cloud quality metric,” IEEE Access, vol. 10, pp. 90481–90497, 2022.
[29]
Q. Liu, H. Su, Z. Duanmu, W. Liu, and Z. Wang, “Perceptual quality assessment of colored 3D point clouds,” IEEE Trans. Vis. Comput. Graph., vol. 29, no. 8, pp. 3642–3655, Aug. 2023.
[30]
X. Wu, Y. Zhang, C. Fan, J. Hou, and S. Kwong, “Subjective quality database and objective study of compressed point clouds with 6DoF head-mounted display,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4630–4644, Dec. 2021.
[31]
E. Alexiou and T. Ebrahimi, “Exploiting user interactivity in quality assessment of point cloud imaging,” in Proc. 11th Int. Conf. Qual. Multimedia Experience, 2019, pp. 1–6.
[32]
W.-X. Tao, G.-Y. Jiang, Z.-D. Jiang, and M. Yu, “Point cloud projection and multi-scale feature fusion network based blind quality assessment for colored point clouds,” in Proc. 29th ACM Int. Conf. Multimedia, 2021, pp. 5266–5272.
[33]
Q. Liu et al., “PQA-Net: Deep no reference point cloud quality assessment via multi-view projection,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4645–4660, Dec. 2021.
[34]
Z. Wang and Q. Li, “Information content weighting for perceptual image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 5, pp. 1185–1198, May 2011.
[35]
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in Proc. 27th Asilomar Conf. Signals, Syst. Comput., 2003, vol. 2, pp. 1398–1402.
[36]
P. Cheraaqee, Z. Maviz, A. Mansouri, and A. Mahmoudi-Aznaveh, “Quality assessment of screen content images in wavelet domain,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 2, pp. 566–578, Feb. 2022.
[37]
Q. Yang, Y. Liu, S. Chen, Y. Xu, and J. Sun, “No-reference point cloud quality assessment via domain adaptation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 21147–21156.
[38]
R. Tu et al., “Pseudo-reference point cloud quality measurement based on joint 2-D and 3-D distortion description,” IEEE Trans. Instrum. Meas., vol. 72, 2023, Art. no.
[39]
Y. Fan et al., “A no-reference quality assessment metric for point cloud based on captured video sequences,” in 2022 IEEE 24th Int. Workshop Multimedia Signal Process., 2022, pp. 1–5.
[40]
S. Bourbia, A. Karine, A. Chetouani, M. El Hassouni, and M. Jridi, “No-reference 3D point cloud quality assessment using multi-view projection and deep convolutional neural network,” IEEE Access, vol. 11, pp. 26759–26772, 2023.
[41]
Y. Zhang, L. Zhu, G. Jiang, S. Kwong, and C.-C. J. Kuo, “A survey on perceptually optimized video coding,” ACM Comput. Surv., vol. 55, no. 12, pp. 1–37, 2023.
[42]
E. M. Torlig, E. Alexiou, T. A. Fonseca, R. L. de Queiroz, and T. Ebrahimi, “A novel methodology for quality assessment of voxelized point clouds,” Proc. SPIE vol. 10752, pp. 174–190, 2018.
[43]
Z. Zhou, M. Bai, J. Wu, and Y. Cui, “Coherent noise attenuation by kurtosis-guided adaptive dictionary learning based on variational sparse representation,” IEEE Trans. Geosci. Remote Sens., vol. 61, 2023, Art. no.
[44]
I. Lissner, J. Preiss, P. Urban, M. S. Lichtenauer, and P. Zolliker, “Image-difference prediction: From grayscale to color,” IEEE Trans. Image Process., vol. 22, no. 2, pp. 435–446, Feb. 2013.
[45]
S. Perry et al., “Quality evaluation of static point clouds encoded using MPEG codecs,” in Proc. IEEE Int. Conf. Image Process., 2020, pp. 3428–3432.
[46]
“Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models,” document ITU-T, P., Tech. Rep. 1401, 2020. [Online]. Available: https://www.itu.int/rec/T-RECP.1401-202001-I/env
[47]
Q. Liu et al., “Reduced reference perceptual quality model with application to rate control for video-based point cloud compression,” IEEE Trans. Image Process., vol. 30, pp. 6623–6636, 2021.
[48]
Q. Yang et al., “MPED: Quantifying point cloud distortion based on multiscale potential energy discrepancy,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 5, pp. 6037–6054, May 2023.
[49]
I. Viola and P. Cesar, “A reduced reference metric for visual quality evaluation of point cloud contents,” IEEE Signal Process. Lett., vol. 27, pp. 1660–1664, 2020.
[50]
X. Liu et al., “Subjective and objective video quality assessment of 3D synthesized views with texture/depth compression distortion,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 4847–4861, Dec. 2015.
[51]
L. He, W. Zhu, and Y. Xu, “Best-effort projection based attribute compression for 3D point cloud,” in Proc. 23rd Asia-Pacific Conf. Commun., 2017, pp. 1–6.
Index Terms
- Colored Point Cloud Quality Assessment Using Complementary Features in 3D and 2D Spaces
Index terms have been assigned to the content through auto-classification.
Recommendations
FQM-GC: Full-reference Quality Metric for Colored Point Cloud Based on Graph Signal Features and Color Features
MMAsia '21: Proceedings of the 3rd ACM International Conference on Multimedia in AsiaColored Point Cloud (CPC) is often distorted in the processes of its acquisition, processing, and compression, so reliable quality assessment metrics are required to estimate the perception of distortion of CPC. We propose a Full-reference Quality ...
No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models
To improve the viewer’s Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital ...
Point Cloud Projection and Multi-Scale Feature Fusion Network Based Blind Quality Assessment for Colored Point Clouds
MM '21: Proceedings of the 29th ACM International Conference on MultimediaWith the wide applications of colored point cloud (CPC) in many fields, many attentions have been paid to CPC's distortions caused by its compression and reconstruction. How to effectively evaluate the visual quality of CPC has become an urgent issue to ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1520-9210 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 January 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025