Abstract
Image blind denoising aims at removing the unknown noise from given images to improve the image’s visual quality. Current blind denoisers can be categorized into two classes, i.e., traditional and deep learning-based methods. Generally, deep learning-based methods are likely to outperform the traditional methods when the model of image noise is within the training noise distribution. However, when the image noise is unseen during the optimization process, traditional methods usually achieve better denoising performance. To address the generalization problem of deep learning-based denoiser, our paper proposed a general yet concise image blind denoising framework that can be applied to existing deep learning-based denoisers. Specifically, given noisy images, our method first extracts high-frequency signals from the images and synthesizes noise from the extracted signals using several noise generation techniques. In contrast to the existing conventional approaches that explicitly model the noise, our proposed framework advances in generating noise directly from the noisy images without explicitly noise modeling. By applying the generated noise to clean images to construct noisy-clean paired images, the deep learning-based denoisers can be optimized using the constructed data with the similar noise of the given noisy images. After optimization, the denoisers are capable of removing noise from the given noisy images. Extensive experiments and comprehensive ablations on both synthetic noisy image and real-world image denoising tasks demonstrate the superiority of our proposed framework over all the compared approaches.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The authors declare that all data supporting the findings of this study are available within the article.
References
ABSoft N (2022) Neat image. http://ni.neatvideo.com/home. Accessed 07 Sep 2018
Ali A, Zhu Y, Chen Q, Yu J, Cai H (2019) Leveraging spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks. In: 2019 IEEE 25th international conference on parallel and distributed systems (ICPADS). IEEE, pp 125–132
Ali A, Zhu Y, Zakarya M (2021) Exploiting dynamic spatio-temporal correlations for citywide traffic flow prediction using attention based neural networks. Inf Sci 577:852–870
Ali A, Zhu Y, Zakarya M (2022) Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction. Neural Netw 145:233–247
Anayaa J, Barbua J (2014) Renoir-a dataset for real low-light noise image reduction. J Visual Commun Image Represent
Association E et al (2010) Standard for characterization of image sensors and cameras. EMVA Standard, 1288
Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 690–698
Chen J, Chen J, Chao H, Yang M (2018) Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3155–3164
Chen T, Lin L, Hui X, Chen R, Wu H (2020) Knowledge-guided multi-label few-shot learning for general image recognition. IEEE Trans Patt Anal Mach Intell
Chen T, Pu T, Wu H, Xie Y, Liu L, Lin L (2021) Cross-domain facial expression recognition: a unified evaluation benchmark and adversarial graph learning. IEEE Trans Patt Anal Mach Intell
Dabov K, Foi A, Katkovnik V, Egiazarian KO (2007) Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space. In: 2007 IEEE international conference on image processing, vol 1, pp I – 313–I –316
Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630
Gonzalez RC, Woods RE (2001) Digital image processing, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., Boston
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680
Granados M, Kim KI, Tompkin J, Theobalt C (2013) Automatic noise modeling for ghost-free hdr reconstruction. ACM Trans Graph 32:201:1–201:10
Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on international conference on machine learning., ICML’15, JMLR.org, vol 37, pp 448–456. http://dl.acm.org/citation.cfm?id=3045118.3045167
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. IEEE Conf Comput Vision Pattern Recognit:5967–5976
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Int Conf Learn Represent
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Lebrun M, Buades A, Morel J-M (2013) A nonlocal bayesian image denoising algorithm. SIAM J Imaging Sci 6(3):1665–1688
Lebrun M, Colom M, Morel J-M (2015) The noise clinic: a blind image denoising algorithm. Image Processing Line 5:1–54
Lebrun M, Colom M, Morel J-M (2015) Multiscale image blind denoising. IEEE Trans Image Process 24(10):3149–3161
Lefkimmiatis S (2017) Non-local color image denoising with convolutional neural networks. In: Proc. IEEE int. conf. computer vision and pattern recognition, pp 3587–3596
Li H, Qin J, Yang Z, Wei P, Pan J, Lin L, Shi Y (2022) Real-world image super-resolution by exclusionary dual-learning, IEEE Trans Multimedia
Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144
Lin Y, Zhang S, Chen T, Lu Y, Li G, Shi Y (2022) Exploring negatives in contrastive learning for unpaired image-to-image translation, arXiv:2204.11018
Liu C, Szeliski R, Kang SB, Zitnick CL, Freeman WT (2008) Automatic estimation and removal of noise from a single image. IEEE Trans Pattern Anal Mach Intell 30:299–314
Liu X, Tanaka M, Okutomi M (2013) Single-image noise level estimation for blind denoising. IEEE transactions on image processing 22(12):5226–5237
Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2017) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Computer vision, 2001. ICCV 2001. Proceedings. Eighth IEEE international conference on. IEEE, vol 2, pp 416–423
Meng D, De La Torre F (2013) Robust matrix factorization with unknown noise. In: Proceedings of the IEEE international conference on computer vision, pp 1337–1344
Nam S, Hwang Y, Matsushita Y, Joo Kim S (2016) A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1683–1691
Plotz T, Roth S (2017) Benchmarking denoising algorithms with real photographs.. In: CVPR, vol, p 6
Portilla J (2004) Full blind denoising through noise covariance estimation using gaussian scale mixtures in the wavelet domain. In: Image processing, 2004. ICIP’04. 2004 international conference on. IEEE, vol 2, pp 1217–1220
Portilla J (2004) Blind non-white noise removal in images using gaussian scale mixtures in the wavelet domain. In: Benelux signal processing symposium
Portilla J, Strela V, Wainwright MJ, Simoncelli EP (2003) Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans Image Process 12(11):1338–1351
Rabie T (2005) Robust estimation approach for blind denoising. IEEE Trans Image Process 14(11):1755–1765
Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. In: International conference on machine learning, pp 1842–1850
Shi Y, Li G, Cao Q, Wang K, Lin L (2019) Face hallucination by attentive sequence optimization with reinforcement learning. IEEE Trans Pattern Anal Mach Intell 42(11):2809–2824
Shi Y, Qin J, Wei P, Ouyang W, Lin L (2019) Perceptual image enhancement by relativistic discriminant learning with cross-scale aggregated representation. IEEE Access 7:39660–39669
Shi Y, Wang K, Chen C, Xu L, Lin L (2017) Structure-preserving image super-resolution via contextualized multitask learning. IEEE Trans Multimedia 19(12):2804–2815
Shi Y, Wang K, Xu L, Lin L (2016) Local-and holistic-structure preserving image super resolution via deep joint component learning. In: 2016 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6
Shi Y, Zhang S, Zhou C, Liang X, Yang X, Lin L (2021) Gtae: graph transformer–based auto-encoders for linguistic-constrained text style transfer. ACM Trans Intell Syst Technol (TIST) 12(3):1–16
Shi Y, Zhong H, Yang Z, Yang X, Lin L (2020) Ddet: dual-path dynamic enhancement network for real-world image super-resolution. IEEE Signal Process Lett 27:481–485
Shocher A, Cohen N, Irani M (2018) Zero-shot super-resolution using deep internal learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3118–3126
Tan J, Shi Y, Yang Z, Wen C, Lin L (2020) Unsupervised multi-view clustering by squeezing hybrid knowledge from cross view and each view. IEEE Trans Multimedia 23:2943–2956
Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Computer vision, 1998. Sixth international conference on. IEEE, pp 839–846
Xu J, Li H, Liang Z, Zhang D, Zhang L (2018) Real-world noisy image denoising: a new benchmark, CoRR arXiv:1804.02603
Xu J, Zhang L, Zhang D (2018) External prior guided internal prior learning for real-world noisy image denoising. IEEE Trans Image Process 27(6):2996–3010
Xu J, Zhang L, Zhang D (2018) A trilateral weighted sparse coding scheme for real-world image denoising, CoRR arXiv:1807.04364
Xu J, Zhang L, Zhang D, Feng X (2017) Multi-channel weighted nuclear norm minimization for real color image denoising. In: IEEE international conference on computer vision, vol 2
Xu J, Zhang L, Zuo W, Zhang D, Feng X (2015) Patch group based nonlocal self-similarity prior learning for image denoising. In: Proceedings of the IEEE international conference on computer vision, pp 244–252
Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301
Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155
Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep cnn denoiser prior for image restoration. In: IEEE conference on computer vision and pattern recognition, vol 2
Zhang K, Zuo W, Zhang L (2018) Ffdnet: toward a fast and flexible solution for cnn based image denoising. IEEE Trans mage Process
Zhao Q, Meng D, Xu Z, Zuo W, Zhang L (2014) Robust principal component analysis with complex noise. In: International conference on machine learning, pp 55–63
Zhu F, Chen G, Heng P-A (2016) From noise modeling to blind image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 420–429
Zhu L, Fu C, Brown MS, Heng P (2017) A non-local low-rank framework for ultrasound speckle reduction. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 493–501
Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: Computer vision (ICCV), 2011 IEEE international conference on. IEEE, pp 479–486
Funding
This work is supported in part by National Natural Science Foundation of China (NSFC) (no. 62002069), the Science and Technology Project of Guangdong Province (no. 2021A1515011341, 2022A1515011835), China Postdoctoral Science Foundation funded project (Grant no. 2021M703687) and the Guangzhou Science and Technology Plan Project (no. 202002030386, 102020369).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
No potential conflict of interest was reported by the authors.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Nie, L., Lin, J., Kang, W. et al. Learning image blind denoisers without explicit noise modeling. Multimed Tools Appl 82, 27839–27859 (2023). https://doi.org/10.1007/s11042-023-14590-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-14590-z