Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality Assessment

Published: 16 February 2024 Publication History

Abstract

Banding, also known as staircase-like contours, frequently occurs in flat areas of images/videos processed by compression or quantization algorithms. As undesirable artifacts, banding destroys the original image structure, thus inevitably degrading users&#x2019; quality of experience (QoE). In this paper, we systematically investigate the banding image quality assessment (IQA) problem, aiming to detect the image banding artifacts and evaluate their perceptual visual quality. Considering that the existing image banding databases only contain limited content sources and banding generation methods, and lack perceptual quality labels (i.e. mean opinion scores), we first build the largest banding IQA database so far, named B anding A rtifact N oticeable D atabase (BAND-2k), which consists of 2,000 banding images generated by 15 compression and quantization schemes. A total of 23 workers participated in the subjective IQA experiment, yielding over 214,000 patch-level banding class labels and 44,371 reliable image-level quality rating scores. Subsequently, we develop an effective no-reference (NR) banding evaluator for banding detection and quality assessment by leveraging frequency characteristics of banding artifacts. To be more specific, a dual convolutional neural network (CNN) is employed to concurrently learn the feature representation from the high-frequency and low-frequency maps, thereby enhancing the ability to discern banding artifacts. The quality score of a banding image is generated by pooling the banding detection maps masked by the spatial frequency filters. The experimental results demonstrate that our banding evaluator achieves remarkably high accuracy in banding detection and also exhibits high SRCC and PLCC results with the perceptual quality labels, even without directly learning a regression model for banding quality evaluation. These findings unveil the strong correlations between the intensity of banding artifacts and the perceptual visual quality, thus validating the necessity of banding quality assessment. The BAND-2k database and the proposed banding evaluator are available at <uri>https://github.com/zijianchen98/</uri> BAND-2k.

References

[1]
Y. Wang, S.-U. Kum, C. Chen, and A. Kokaram, “A perceptual visibility metric for banding artifacts,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2016, pp. 2067–2071.
[2]
A. Kapoor, J. Sapra, and Z. Wang, “Capturing banding in images: Database construction and objective assessment,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Jun. 2021, pp. 2425–2429.
[3]
Y. Xue, R. Azevedo, X. Huangfu, Y. Zhang, C. Schroers, and S. Labrozzi, “Large-scale multi-site subjective assessment on image banding artifacts,” in Proc. 15th Int. Conf. Quality Multimedia Exper. (QoMEX), Jun. 2023, pp. 213–216.
[4]
T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560–576, Jul. 2003.
[5]
G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.
[6]
D. Mukherjeeet al., “The latest open-source video codec VP9—An overview and preliminary results,” in Proc. Picture Coding Symp. (PCS), Dec. 2013, pp. 390–393.
[7]
Z. Tu, J. Lin, Y. Wang, B. Adsumilli, and A. C. Bovik, “Bband index: A no-reference banding artifact predictor,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), May 2020, pp. 2712–2716.
[8]
P. Tandon, M. Afonso, J. Sole, and L. Krasula, “CAMBI: Contrast-aware multiscale banding index,” in Proc. Picture Coding Symp. (PCS), Jun. 2021, pp. 1–5.
[9]
L. Krasula, Z. Li, C. G. Bampis, M. Afonso, N. F. Miret, and J. Sole, “Banding vs. quality: Perceptual impact and objective assessment,” in Proc. IEEE Int. Conf. Image Process. (ICIP), 2022, pp. 2236–2240.
[10]
S. Li and B. Yang, “Multifocus image fusion using region segmentation and spatial frequency,” Image Vision Comput., vol. 26, no. 7, pp. 971–979, Jul. 2008.
[11]
V. Kazemi, A. Shahzadi, and H. K. Bizaki, “Multifocus image fusion using adaptive block compressive sensing by combining spatial frequency,” Multimedia Tools Appl., vol. 81, no. 11, pp. 15153–15170, May 2022.
[12]
C. Feng, D. Danier, F. Zhang, and D. Bull, “BVI-artefact: An artefact detection benchmark dataset for streamed videos,” 2023, arXiv:2312.08859.
[13]
S. J. Daly and X. Feng, “Decontouring: Prevention and removal of false contour artifacts,” Proc. SPIE, vol. 5292, pp. 130–149, Jun. 2004.
[14]
J. W. Lee, B. R. Lim, R. H. Park, J. S. Kim, and W. Ahn, “Two-stage false contour detection using directional contrast and its application to adaptive false contour reduction,” IEEE Trans. Consum. Electron., vol. 52, no. 1, pp. 179–188, Feb. 2006.
[15]
Q. Huang, H. Y. Kim, W.-J. Tsai, S. Y. Jeong, J. S. Choi, and C.-C. J. Kuo, “Understanding and removal of false contour in HEVC compressed images,” IEEE Trans. Circuits Syst. Video Technol., vol. 28, no. 2, pp. 378–391, Feb. 2018.
[16]
S. Bhagavathy, J. Llach, and J. Zhai, “Multiscale probabilistic dithering for suppressing contour artifacts in digital images,” IEEE Trans. Image Process., vol. 18, no. 9, pp. 1936–1945, Sep. 2009.
[17]
G. Baugh, A. Kokaram, and F. Pitié, “Advanced video debanding,” in Proc. 11th Eur. Conf. Vis. Media Prod., Nov. 2014, pp. 1–10.
[18]
J. Sole and M. Afonso, “A debanding algorithm for AV2,” in Proc. Data Compress. Conf. (DCC), Mar. 2023, pp. 258–267.
[19]
D. Hasler and S. E. Suesstrunk, “Measuring colorfulness in natural images,” Proc. SPIE, vol. 5007, pp. 87–95, Aug. 2004.
[20]
N. D. Narvekar and L. J. Karam, “A no-reference image blur metric based on the cumulative probability of blur detection (CPBD),” IEEE Trans. Image Process., vol. 20, no. 9, pp. 2678–2683, Sep. 2011.
[21]
Methodology for the Subjective Assessment of the Quality of Television Pictures, Int. Telecommun. Union, Geneva, Switzerland, 2002.
[22]
M. S. Silberman, B. Tomlinson, R. LaPlante, J. Ross, L. Irani, and A. Zaldivar, “Responsible research with crowds: Pay crowdworkers at least minimum wage,” Commun. ACM, vol. 61, no. 3, pp. 39–41, Feb. 2018.
[23]
F. E. Grubbs, “Sample criteria for testing outlying observations,” Ann. Math. Statist., vol. 21, no. 1, pp. 27–58, Mar. 1950.
[24]
H. Wanget al., “VideoSet: A large-scale compressed video quality dataset based on JND measurement,” J. Vis. Commun. Image Represent., vol. 46, pp. 292–302, Jul. 2017.
[25]
L. Bar, N. Sochen, and N. Kiryati, “Semi-blind image restoration via Mumford–Shah regularization,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 483–493, Feb. 2006.
[26]
E. A. DeYoe and D. C. Van Essen, “Concurrent processing streams in monkey visual cortex,” Trends Neurosciences, vol. 11, no. 5, pp. 219–226, Jan. 1988.
[27]
C. Zhang, Z. Huang, S. Liu, and J. Xiao, “Dual-channel multi-task CNN for no-reference screen content image quality assessment,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 8, pp. 5011–5025, Aug. 2022.
[28]
Z. Tang, Y. Zheng, K. Gu, K. Liao, W. Wang, and M. Yu, “Full-reference image quality assessment by combining features in spatial and frequency domains,” IEEE Trans. Broadcast., vol. 65, no. 1, pp. 138–151, Mar. 2019.
[29]
K. Gu, J. Qiao, X. Min, G. Yue, W. Lin, and D. Thalmann, “Evaluating quality of screen content images via structural variation analysis,” IEEE Trans. Vis. Comput. Graph., vol. 24, no. 10, pp. 2689–2701, Oct. 2018.
[30]
S. A. M. Al-Sumaidaee, M. A. M. Abdullah, R. R. O. Al-Nima, S. S. Dlay, and J. A. Chambers, “Multi-gradient features and elongated quinary pattern encoding for image-based facial expression recognition,” Pattern Recognit., vol. 71, pp. 249–263, Nov. 2017.
[31]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770–778.
[32]
R. Ranjan, V. M. Patel, and R. Chellappa, “HyperFace: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 1, pp. 121–135, Jan. 2019.
[33]
R. Du, J. Xie, Z. Ma, D. Chang, Y.-Z. Song, and J. Guo, “Progressive learning of category-consistent multi-granularity features for fine-grained visual classification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 12, pp. 9521–9535, Dec. 2022.
[34]
W. Zhao and H. Zhao, “Hierarchical long-tailed classification based on multi-granularity knowledge transfer driven by multi-scale feature fusion,” Pattern Recognit., vol. 145, Jan. 2024, Art. no.
[35]
D. Ghadiyaram, C. Chen, S. Inguva, and A. Kokaram, “A no-reference video quality predictor for compression and scaling artifacts,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 3445–3449.
[36]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
[37]
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in Proc. 37th Asilomar Conf. Signals, Syst. Comput., 2003, pp. 1398–1402.
[38]
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2018, pp. 586–595.
[39]
S. Lao, “Attentions help CNNs see better: Attention-based hybrid image quality assessment network,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jul. 2022, pp. 1140–1149.
[40]
A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695–4708, Dec. 2012.
[41]
A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely blind ” image quality analyzer, ” IEEE Signal Process. Lett., vol. 20, no. 3, pp. 209–212, May 2012.
[42]
H. Talebi and P. Milanfar, “NIMA: Neural image assessment,” IEEE Trans. Image Process., vol. 27, no. 8, pp. 3998–4011, Aug. 2018.
[43]
W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, “Blind image quality assessment using a deep bilinear convolutional neural network,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 1, pp. 36–47, Jan. 2018.
[44]
S. Suet al., “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2020, pp. 3667–3676.
[45]
W. Sun, X. Min, D. Tu, S. Ma, and G. Zhai, “Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training,” IEEE J. Sel. Topics Signal Process., vol. 17, no. 6, pp. 1178–1192, Nov. 2023.
[46]
S. A. Golestaneh, S. Dadsetan, and K. M. Kitani, “No-reference image quality assessment via transformers, relative ranking, and self-consistency,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., Oct. 2022, pp. 1220–1230.
[47]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014, arXiv:1412.6980.
[48]
K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, “Study of subjective and objective quality assessment of video,” IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427–1441, Jun. 2010.
[49]
J. L. Bentley, “Multidimensional binary search trees used for associative searching,” Commun. ACM, vol. 18, no. 9, pp. 509–517, Sep. 1975.
[50]
D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” IEEE Trans. Image Process., vol. 25, no. 1, pp. 372–387, Jan. 2016.
[51]
V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, “KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment,” IEEE Trans. Image Process., vol. 29, pp. 4041–4056, 2020.
[52]
A. Ciancio, A. L. N. T. T. da Costa, E. A. B. da Silva, A. Said, R. Samadani, and P. Obrador, “No-reference blur assessment of digital pictures based on multifeature classifiers,” IEEE Trans. Image Process., vol. 20, no. 1, pp. 64–75, Jan. 2011.
[53]
N. Ponomarenkoet al., “Image database TID2013: Peculiarities, results and perspectives,” Signal Process. Image Commun., vol. 30, pp. 57–77, Jan. 2015.
[54]
Z. Ni, H. Zeng, L. Ma, J. Hou, J. Chen, and K. Ma, “A Gabor feature-based quality assessment model for the screen content images,” IEEE Trans. Image Process., vol. 27, no. 9, pp. 4516–4528, Sep. 2018.
[55]
J. Yang, Z. Bian, Y. Zhao, W. Lu, and X. Gao, “Full-reference quality assessment for screen content images based on the concept of global-guidance and local-adjustment,” IEEE Trans. Broadcast., vol. 67, no. 3, pp. 696–709, Sep. 2021.
[56]
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent., 2015, pp. 1–14.
[57]
Z. Liuet al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 10012–10022.

Cited By

View all
  • (2024)Banding Detection via Adaptive Global Frequency Domain AnalysisProceedings of the 3rd Workshop on Quality of Experience in Visual Multimedia Applications10.1145/3689093.3689185(48-57)Online publication date: 28-Oct-2024
  • (2024)Adaptive Dual-Domain Debanding: A Novel Algorithm for Image and Video EnhancementProceedings of the 1st International Workshop on Efficient Multimedia Computing under Limited10.1145/3688863.3689572(49-58)Online publication date: 28-Oct-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Circuits and Systems for Video Technology
IEEE Transactions on Circuits and Systems for Video Technology  Volume 34, Issue 7
July 2024
1398 pages

Publisher

IEEE Press

Publication History

Published: 16 February 2024

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 01 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Banding Detection via Adaptive Global Frequency Domain AnalysisProceedings of the 3rd Workshop on Quality of Experience in Visual Multimedia Applications10.1145/3689093.3689185(48-57)Online publication date: 28-Oct-2024
  • (2024)Adaptive Dual-Domain Debanding: A Novel Algorithm for Image and Video EnhancementProceedings of the 1st International Workshop on Efficient Multimedia Computing under Limited10.1145/3688863.3689572(49-58)Online publication date: 28-Oct-2024

View Options

View options

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media